The other two home runs came in the third and sixth innings, when left fielder Carl Crawford and third baseman Willy Aybar, respectively, each hit one out of the park.
"We really pretty much lived by the long ball," manager Joe Maddon said. "We had some homers. We didn't have to do the speed game today -- we did it with the long ball."
By adding three home runs to their June total on the final day of the month, the Rays (44-35) set a new club record for long balls in a single month with 41. The team went 19-7 in June and moved ahead of Toronto (41-38) for third place in the American League East.
The record comes two days after the Rays became the fastest team in modern Major League history to record 100 home runs and 100 stolen bases in a season, doing so in 77 games.
This month, the two hitters at the top of the Rays' order have contributed in both categories, and Tuesday's win came thanks in large part to a blast of power from Upton and Crawford -- the team's leadoff and No. 2 batters. They've packed an impressive one-two punch for the Rays, hitting a combined .324 in June, while driving in 33 runs and contributing 10 home runs and 24 stolen bases.
Crawford's blast was his second in as many nights, and he did it by reaching down in the zone for a tough slider from Richmond (6-5) -- a pitch most batters would never expect to hit.
"He's been hot lately," Upton said. "That's the thing about Carl -- he doesn't necessarily hit the pitches you're supposed to hit. He hits the balls that nobody else hits. That's why he is who he is."
When asked about the pitch Crawford drove into the stands in right field, Maddon could only shake his head in awe.
"Carl's unique," Maddon said. "He does certain things, man. He's been playing for me as well as I've seen him play. He's really becoming a pretty good baseball player right now."
Crawford believes that with Upton continuing to heat up after a slow start to the year -- the center fielder batted only .204 through the end of May -- the tandem will only make life tougher and tougher for opposing pitchers.
"He's really starting to swing the bat good and I'm adjusting to the way it is when he's swinging the bat well," Crawford said of Upton. "Once we get really used to it, it should get even better."
Upton and Crawford's heroics have certainly not gone unnoticed by the man who hits behind them -- third baseman Evan Longoria.
"They're pretty good," Longoria said. "When either one of them gets on, it's a chance for a stolen base and most likely a stolen base. You know they're in scoring position right away for you. It makes it a lot easier for you as a hitter. You know that the pitcher's more worried about the guy that's on base than you, and you get a lot more fastballs and a lot more balls over the plate."
For Maddon, the three home runs were nice, but pitching has been a crucial part of the Rays' recent success -- and was on Tuesday as well.
"We did it with the long ball, and you do that because you pitch so well," Maddon said. "Any time you loook at a situation where we've been playing so well, for me it starts pretty much with pitching. It pretty much ends with pitching, too."
Since Aybar's home run did not come until the sixth inning and the Rays' fourth run until the eighth, Garza had to keep the Jays' offense at bay to preserve his team's narrow lead.
Garza (6-5) was not at his best, allowing seven hits and three walks over his seven-plus innings, but did his best without his best stuff, holding the Jays to a single run in the outing.
"Garz struggled on occasion a little bit with command," Maddon said. "It wasn't his sharpest outing, but his stuff is so good that ... he [can] put together a day where he isn't at his best and turn it into a win.
"That's when you're good -- when you're able to win when you're having a bad day."
Garza has been a Blue Jay nemesis since the beginning of 2008, posting a stellar 0.47 ERA in five starts against Toronto last year. While he was not as dominant on Tuesday as he has been against the Jays in the past, it was enough.
"Just happy that we won," Garza said. "It might've been ugly, but we got the job done. That's all that counts. As long as we've got that W."
However Garza got it done, his teammates give credit to the pitching staff for taking the pressure off the Rays' hitters, giving them the chance to win games whether by speed or power.
"The starting pitching has been unbelievable," Longoria said. "When you go four, five, six, seven innings of one-run or scoreless ball, it's not too tough on the offense to win a ballgame -- just keep grinding.
"The tempo and the momentum of the game is always in our favor when they're going out there and getting three quick outs and coming in."