David Pastrnak also had a power-play goal, Taylor Hall scored on a delayed penalty and backup goalie Jeremy Swayman, wearing new dark-tan pads, made a season-high 30 saves for Boston.
The NHL-best Bruins honored captain Patrice Bergeron for reaching 1,000 career points last month during an on-ice pregame ceremony.
''Never mind the Hall of Fame player he is, he's a Hall of Fame person and leader,'' Boston's first-year coach Jim Montgomery said. ''So much so that after the first I was not happy with the team. I was not very happy in between periods when I addressed the team. The second period had no impact.
''I said to Bergy with 40 seconds left in the second that: `It's your dressing room.' I just hope he doesn't want to be a head coach,'' Montgomery said, smiling. ''We came out and played our best period.''
Boston improved to 16-0-2 at TD Garden, with both losses coming in a shootout.
''I can't say it's always just me,'' Bergeron said, downplaying his talk to the team. ''I think there's a lot of guys that are vocal. I think you need that. ... Obviously, voices matter and is important, but I think your actions speak louder sometimes.''
Boone Jenner had a power-play goal, Patrik Laine scored and Daniil Tarasov stopped 35 shots for Columbus, which has lost three straight and six of eight.
Krejci's go-ahead goal late in the second was a bit of redemption. Columbus had tied the game 1-all while he was in the penalty box for high sticking. Just under four minutes later, he moved Boston ahead 2-1 on a one-timer from the left circle off a pass from Charlie McAvoy.
''You give up the lead and its tough to come back,'' said Jenner, who tied the game. ''You don't want to be down. You want to get that tying goal (again), but they were able to get a couple.''
Hall extended the margin to 3-1 at 12:41 of the third period and Tomas Nosek scored 19 seconds later.
Pastrnak's 20th goal made it 1-0 at 7:17 of the opening period, the seventh straight season he's reached that mark.
Joined by his wife, Stephanie, and their three small children, Bergeron was presented with crystal from the NHL by Johnny Bucyk, a gold stick from GM Don Sweeney, a check for $37,000 for a charity of his choice by President Cam Neely and gifts from the players by teammates Brad Marchand and Krejci.
He became the fourth player in Bruins history to reach the mark with the club, joining Hall of Famers Ray Bourque (1,506), Bucyk (1,339), and Phil Esposito (1,012) on Nov. 21 in a win at Tampa Bay. The three of them also congratulated him via a video on the Jumbotron.
''It was special,'' Bergeron said. ''I've been around for a long time in this city and played in front of these fans and I'm very appreciative of all the support I've had from them. It was nice moment. I was happy to share that with my family and teammates as well.''
There was also a video during a stoppage late in the first period with other NHL players congratulating Bergeron. That was greeted with a spattering of boos when Pittsburgh's Sidney Crosby and Washington's Alex Ovechkin were shown.
BLUE JACKETS' BITS
Star Johnny Gaudreau was held without a goal for the fifth time in the last six games. . Forward Eric Robinson returned to the lineup after missing the previous game due to an illness. . They're in a stretch of five of six on the road.
Blue Jackets: Host Dallas on Monday.
Bruins: Face Florida Monday in the fourth of a five-game homestand.
AP NHL: https://apnews.com/hub/NHL and https://twitter.com/AP-Sports
Copyright 2023 STATS LLC and Associated Press. Any commercial use or distribution without the express written consent of STATS LLC and Associated Press is strictly prohibited.