The rally began with Jeff Larish's two-out double off Dane de la Rosa. Lucas May drew a walk to bring up Welch.
Jared Hughes struck out two in a perfect eighth inning to earn the win, with Tim Alderson earning the save.
The Rays had previously attempted a comeback of their own. Run-scoring doubles by Luke Scott and Chris Gimenez capped a three-run go-ahead rally in the sixth. Ben Zobrist's leadoff homer touched off that scoring against left-hander Mike Zagurski.
Alex Cobb went 6 1/3 innings, allowing five hits and three runs while walking three and striking out five.
John McDonald, the newest Pirates player, had snapped a 1-1 tie with an RBI single in the fourth, and pinch-hitter Garrett Jones' sacrifice fly later in the inning had given the Pirates a 3-1 lead.
Jeanmar Gomez went four innings in his start and allowed three hits and a run.
Tampa Bay took a 1-0 lead on Shelley Duncan's run-scoring infield grounder in the second, and Felix Pie tied it up for the Pirates with a run-scoring double in the third. Pie's eighth RBI also tied Andrew McCutchen for the team's spring lead.
Up next: David Price will start Saturday, when the Rays host the Twins in a 1:05 contest at Charlotte Sports Park, available live on MLB.TV. The reigning American League Cy Young Award winner will be making his first start in a Major League spring game since March 8. His past two outings have come in Minor League games that coincided with off-days for the Rays. Also slated to see action are Cesar Ramos, Kyle Farnsworth and Jake McGee.
Tom Singer is a reporter for MLB.com and writes an MLBlog Change for a Nickel. He can also be found on Twitter @Tom_Singer. This story was not subject to the approval of Major League Baseball or its clubs.