Pellegrino took over at St Mary's on a three-year deal last June, but Southampton announced in a statement that they had parted company with the 46-year-old with the club only one place and one point above the relegation zone.
"The club will look to appoint a new management team as soon as possible, with the search for a replacement already underway," the statement added.
Assistant coaches Carlos Compagnucci and Xavier Tamarit have also left the club.
A disastrous run of form that has seen the Saints win only one of their last 17 Premier League matches always appeared likely to end in Pellegrino's demise.
The latest 3-0 defeat at relegation rivals Newcastle on Saturday proved to be the final straw for the Southampton hierarchy, with the club in danger of being relegated from the top flight for the first time since being promoted in 2012.
"I observed some players who gave up and we cannot show this, to be honest," Pellegrino admitted after the humbling loss at St James' Park.
"You can lose, you can play badly, but you have to show another face on the pitch.
"For this reason, I would like to say sorry to our fans because we couldn't represent our club properly. This is the worst thing that we did today."
Former Blackburn, Manchester City and Stoke manager Mark Hughes has been installed as the early bookmakers' favourite to replace Pellegrino, with Marco Silva also linked with the job after being sacked by Watford in January.
Former Alaves coach Pellegrino did lead the south-coast club into the FA Cup quarter-finals, with the now-managerless side set to visit third-tier Wigan in the last eight on Sunday.
But Southampton had expected far better this season, after three consecutive top-eight finishes.
Frenchman Claude Puel was sacked after leading the team to eighth place and the League Cup final last term, and has since thrived after taking over at Leicester earlier this season.
Southampton, who sold centre-back Virgil van Dijk for a record-breaking £75 million ($102 million, 84 million euros) in January, have started leaking goals in recent weeks, conceding 19 in their last 10 Premier League outings.
But scoring has been their biggest problem all season, with Charlie Austin their top-scorer with only six goals in all competitions.
Fans have become increasingly frustrated with their team's defensive style of play, which had been one of the main reasons Puel was axed, with Leicester ironically now one of the more exciting sides to watch in the division.