Yegor Chinakhov, Eric Robinson and Kent Johnson also scored for the Blue Jackets who won for the first time in three road games. Liam Foudy and Jack Roslovic each had two assists.
''It was amazing, very special for me,'' a beaming Tarasov said. ''The guys helped me a lot ... I just did my job.''
Artemi Panarin scored to give him a point in each of the Rangers' six games this season and Jaroslav Halak had 16 saves to fall to 0-2-0. New York lost its second straight at home after an overtime loss to San Jose on Thursday night.
''Not good enough, obviously,'' Rangers coach Gerard Gallant said. ''We were slow. I thought we were a step behind all night ... This was far from Halak's fault.''
Werenski opened the scoring with 2:06 left in the first period. Halak seemed to have stopped Werenski's shot but the puck skittered past the veteran goalie.
Peeke made it 2-0 at 1:42 of the second when his shot from the right faceoff circle eluded Halak for this first of the campaign.
Chinakhov's goal came at the 9-minute mark of the second for his first of the season to give Columbus a three-goal lead. Foudy, making his season debut, got his second assist of the night.
Panarin got his fourth of the season on a 5-on-3 power play with 1:16 left in the middle period to get the Rangers on the scoreboard.
Robinson added his first with 5:47 left in the third before Johnson capped the scoring 44 seconds later.
''We managed the game well in the third,? Blue Jackets coach Brad Larsen said. ''As a group, we calmed ourselves.''
Tarasov, a 23-year-old Russian was a third-round draft pick in 2017 by the Blue Jackets, made nine saves in the first period, 15 in the second and six in the third.
The 6-foot-5 goalie, who made a nifty glove save on Rangers defenseman Zac Jones at 6:40 of the second, was mobbed by his teammates at the final buzzer.
''Daniil was great for us,'' Larsen added. ''He was very calm and he made tough saves look easy.''
Tarasov has been backing up starter Elvis Merzlikins while Joonas Korpisalo recovers from hip surgery.
Halak, the 37-year-old netminder from Slovakia who joined the Rangers last summer to back up starter Igor Shesterkin, credited his opponent in the Blue Jackets' crease.
''I just got outplayed by their goalie. That wasn't my best game,'' Halak said. ''It's tough coming from behind. That's what happened in the third. They got a few chances and they scored on them.''
Rangers forward Filip Chytil did not return after a collision with Columbus forward Cole Sillinger early in the contest. Gallant said after the game that Chytil absorbed an accidental shoulder to the head and was ''day-to-day.''
Rangers defenseman Ryan Lindgren played in his 200th career NHL game.
Rangers: Host defending champion Colorado on Tuesday night.
Blue Jackets: Host Arizona on Tuesday night.
Copyright 2023 STATS LLC and Associated Press. Any commercial use or distribution without the express written consent of STATS LLC and Associated Press is strictly prohibited.