With Jensen at the helm, Germany could only manage a 14th place finish at December's tournament in Hungary and Croatia.
"Already in Zagreb [at the Euros] and in the tournament's aftermath we have talked to the coaches and the players, analysing the games the team played in 2014," president of the German Handball Federation Bernhard Bauer said.
"Unfortunately, we weren't able to see any progress. That's why both parties in mutual consent decided to cancel Heine Jensen's contract."
Tokyo 2020 - France vs Russia - Handball - Olympic Highlights
Jensen took charge of Germany in early 2011, leading them to seventh at the European Championships in 2012 and then to the quarter-finals at the World Championships a year later.
Despite the split, the 37-year-old said he remains grateful for the opportunity to work with the national side during his four-year tenure.
"I really enjoyed and it was an honour for me working with this team and for the German Handball Federation, but following the disappointing EURO this team needs some new impetus," Jensen said.
"I'm thankful for the chance that I got as a young coach back in 2011.
"I wish the federation and the team all best for a possible Olympic qualification and the World Championship 2017 in Germany."
Tokyo 2020 - Norway vs Sweden - Handball - Olympic Highlights
Tokyo 2020 - France vs Denmark - Handball - Olympic Highlights