BOSTON -- Kevin Gausman felt like the Baltimore Orioles had the game stolen from them on Wednesday night.

The Orioles' starter was just 19 pitches into his day early in the second inning when he plunked Boston Red Sox shortstop Xander Bogaerts and was ejected by home plate umpire Sam Holbrook.

After Boston's Chris Sale plunked Baltimore's Manny Machado on Tuesday and the league held a conference call to quell the feud between the teams on Wednesday morning, the umpires were on high alert -- and Holbrook didn't hesitate to toss Gausman.

No warnings were given by the umpires before the game or after Gausman hit Bogaerts.

Boston proceeded to score four runs after that, two driven in by Josh Rutledge, and went on to secure a 4-2 win over Baltimore at Fenway Park.

"I didn't know who he (Holbrook) threw out at first. Just complete bush league, to be honest," a frustrated Gausman said.

Gasuman's 0-0 curveball to Bogaerts was clocked at 76 mph, but speed or pitch type didn't matter to Holbrook.

A visibly stunned Gausman pleaded his case with the umpire as O's catcher Caleb Joseph began to scream in Holbrook's face.

Baltimore skipper Buck Showalter also came out to argue, but neither he nor Joseph were thrown out.

"I told (Holbrook) right away, 'You're screwing our bullpen,'" Gausman said. "Pretty frustrating. For some reason, (Sale) can get away with throwing 98 behind (Machado's) leg (which) he's had two surgeries on, and I throw a curveball and he tosses me.

"First curveball I threw of the day. I was trying to throw one with a lot of depth and (it) just slipped out of my hand."

Gausman (1-3) was charged with a run, no hits and a walk in his one-plus innings.

Richard Bleier came on to pitch a career-high-matching four innings, allowing three runs (one earned), before Thursday's scheduled starter, Ubaldo Jimenez, tossed three scoreless innings.

Adam Jones was also ejected in the fifth inning for arguing balls and strikes. It was his first career ejection.

The last time two Orioles players were tossed in the same game was July 8, 2011, in Boston when Kevin Gregg and Jim Johnson were thrown out.

"It's unfortunate that those decisions come, but we have to deal with them now and it affects a lot of people," Showalter said. "... So you don't sit there and bleed; you do something about it."

Even Red Sox manager John Farrell was questioning Holbrook's decision.

"I think it caught us all a little bit off guard," Farrell said. "That's the best I can say right now. Surprising."

Drew Pomeranz tossed 5 1/3 two-run innings, striking out seven, for Boston (15-12), which has taken two of the first three in its four-game set with Baltimore (16-10).

"I would have liked to get through that sixth inning, but I gave us a good chance to win," said Pomeranz, who allowed five hits and walked two. "Bullpen came in and shut the door."

Craig Kimbrel struck out the side in the ninth for his AL-leading 10th save.

Chris Young and Dustin Pedroia each had an RBI for the Red Sox, with Pedroia (1-for-3) moving into a tie with Harry Hooper for eighth place on the team's career hits list, with 1,707.

J.J. Hardy and Trey Mancini each drove in a run for the Orioles, who have dropped four of the last six games. Boston improved to 4-4 on its 10-game homestand.

Boston built a four-run lead after five innings before Baltimore sliced the deficit in half in the sixth on Mancini's sacrifice fly and Hardy's RBI single to left.

Rutledge's line-drive single off Bleier in the second plated Boston's first run.

Rutledge entered the game in the first as a defensive replacement for Marco Hernandez, who strained his left shoulder fielding a ball at third base.

Hernandez said his shoulder popped out reaching for the ball, and he pushed it back in on the field. It's uncertain whether he will land on the disabled list.

Boston added three runs in the fourth on Young's one-run double to center, Rutledge's fielder's choice that scored Mitch Moreland and Pedroia's sacrifice fly.

NOTES: Boston issued a statement during Wednesday's game, saying it ejected a fan from the ballpark Tuesday night for directing a racial slur at another fan. The team said that fan is no longer welcome at Fenway Park. ... MLB commissioner Rob Manfred held a conference call with Orioles and Red Sox brass on Wednesday to discuss the teams' ongoing feud. Manfred warned both sides of discipline should they continue throwing at each other. "(The call was) to just basically address the last two weeks and to say enough is enough," Boston manager John Farrell said. Baltimore manager Buck Showalter was on the call but wouldn't share specifics.