With its third consecutive win, Tampa Bay (64-54) moved to three games back of first place in the American League Wild Card race, which is a tie between Boston and Texas.
Upton was a catalyst in the Rays' lineup en route to a World Series berth last year, while Burrell was a stable presence in the middle of the order for the champion Phillies. But they came into the series opener batting .236 and .220, respectively.
However, the end of the season is about seven weeks away, and Upton and Burrell have time to turn it around and help Tampa Bay make up those remaining three games.
"It's something kind of everybody's been waiting on all year," said Upton, who finished 3-for-4 with two RBIs but flied out in his last at-bat, when he needed a double for the cycle. "We just have to forget about everything that's happened up to this point, and hopefully me and [Burrell] can get going down the stretch. If we're going to make a run at this thing, we're definitely going to need to."
After Upton and Burrell came through in the sixth, the Rays had a 5-2 advantage. But that shrunk to one in the eighth, when Matt Wieters hit a two-run homer off Randy Choate to cap a nine-pitch at-bat.
In the ninth, J.P. Howell faced Wieters with two outs and runners on first and second. Howell got the rookie catcher to swing at three consecutive pitches to nail down his 13th save in 18 chances.
"We battled back," Orioles manager Dave Trembley said. "But earlier in the game, we had some opportunities. One in particular would've scored a run, and one run was the difference in the game."
Added Howell about the ninth: "Those guys are sitting on a lot of fastballs. They're trying to jump ship when it's a one-run game, so I tried to mix it up and keep it down."
Price left with the lead and gave up only two runs, but the 23-year-old rookie's pitch count was in triple digits by the end of the fifth. He ended up falling short of the sixth inning for the eighth time in 15 starts this season.
After throwing just nine pitches in the first inning, Price needed 48 to get through the next two. Sporting a 1-0 lead in the third, the left-hander gave up RBI singles to Adam Jones and Nolan Reimold to give the O's the lead.
Price, coming off a start that saw him throw four no-hit innings and then give up six runs in the next two frames, yielded seven hits and three walks while striking out five and throwing 102 pitches (59 strikes) in five innings to get the victory.
"We got a win; that's awesome. I went five innings; that's not what I wanted to do," said Price, who tied a club record by winning his fifth straight start at Tropicana Field. "I thought I had good stuff. I wish I could've commanded the zone a little bit better, but that's baseball."
Down, 2-1, in the fourth, Gabe Gross gave his team its second lead of the game with a two-run homer to right-center field.
In the sixth, Tampa Bay got some cushion and knocked Baltimore starter Jason Berken out of the game on the homers by Burrell, who returned to the lineup for the first time after leaving Friday's game with a neck flare-up, and Upton, who ended a span of 151 at-bats without going deep.
"I don't want to say that's the one thing that needs to occur for us to get back to the playoffs, but it certainly would help to get them looking like themselves," Rays manager Joe Maddon said of Upton and Burrell. "Pat's been steadily looking like that. B.J. has not. That's a real nice moment for [Upton] tonight. And, again, it's all about confidence. If he just gets his confidence going right now and gets that feeling and goes up there and knows he can drive a ball, it's going to make all the difference in the world."