Liverpool - who lost Adam Lallana to a first-half groin injury which has ruled him out of England's World Cup qualifiers against Malta and Slovenia - were below-par in south Wales as they trailed 1-0 at the break to an early Leroy Fer goal.
But Roberto Firmino and James Milner rewarded a much-improved second half performance to secure a 2-1 win and hand Liverpool a fourth straight Premier League victory.
Liverpool's Brazilian midfielder Roberto Firmino (C) celebrates scoring
Image credit: Reuters
"I was very angry. I have lost a lot of football games, but today makes no sense," Liverpool manager Klopp said.
"We were not ready and it is my responsibility. It was not good.
"The build-up was too static, no movement.
"We were never compact enough and we lost a lot of balls.
"We got better in the second half and when we let the ball roll it became difficult for Swansea.
"The second half was better, but still not brilliant."
Firmino levelled with his fourth goal of the season, a 54th-minute header, before Milner swept home his third penalty in two games six minutes from time.
Liverpool's win took them to within two points of leaders Manchester City - who play at Tottenham on Sunday - and was their fifth consecutive victory in all competitions.
But Klopp said Liverpool needed to show greater discipline after the international programme, the Reds returning from the two-week break with a home game against Manchester United on October 17.
"It was not a day we should lose," Klopp said.
"I didn't think this would be a bad day, there was no sign in training.
"We lost patience and were moaning about decisions. We need to stop that.
"We were angry with ourselves, but we got cooler.
"In my job I can see that, but not change a lot until half-time.
"It's important that when you are not brilliant, you are still able to win."
Swansea have not won a league game since the opening day of the season and speculation looks set to swirl around Guidolin's future.
Ryan Giggs and former United States manager Bob Bradley have been linked with taking over at the Liberty Stadium, and Guidolin accepted before the weekend that he could be out of a job if Swansea lost to Liverpool.
But the Italian insisted he was not bothered about speculation over his future and said: "This not my problem. My job is to work with the players and prepare the team for games.
"I am worried for the table, not my position. It is not good. We need more points.
"I am sure in the future this team can turn the situation around.
"The relationship between me and my players is perfect, they have good quality and characters.
"I don't know what happens tomorrow, or next week. But w e are working hard to improve and we will continue to do so."