An FCF spokesman said a letter had been sent to soccer's world governing body FIFA announcing its decision.
"After analysing all the aspects involved in the organisation of the FIFA World Cup 2014, the Colombian Football Federation has decided to withdraw the candidature which it launched in December and which was very generously accepted at that time," said FCF president Luis Bedoya in the letter.
South America is due to stage the tournament under a rotation system introduced by FIFA. The other eight South American federations have all backed Brazil.
FIFA move to clear way for England bid
In Rio de Janeiro, Brazilian Football Confederation (CBF) president Ricardo Teixeira said the Colombian withdrawal would have little influence over his own country's bid.
"It's just as difficult as before," he told Brazilian television. "We have to do our homework otherwise FIFA will not give us the right to host the World Cup."
Last month, Colombia President Alvaro Uribe criticised FIFA president Sepp Blatter for dismissing the bid as a publicity stunt.
"Before he can say that, he has to get rid of all the mafia who have infected football, and respect Colombia," Uribe said.
However, Bedoya said he had been well received during a visit to FIFA's Zurich headquarters shortly afterwards.
"The pleasant reception which we received in March was very reassuring," he said.
The bid was controversial within Colombia, with many criticising the amount of money that would have to be spent to stage a World Cup.
The FCF was among the federations that agreed to back Brazil at a meeting in 2003 but broke ranks last year after Uribe said he wanted the country to launch a bid.
Brazil and Colombia knew they would have to invest heavily in new stadiums and infrastructure. Blatter has already suggested the tournament could "go north" if South America does not provide a suitable bid.
Security is also a major worry in both nations while Brazil is facing an air traffic control crisis which has caused widespread delays over the last few months.
Colombia was awarded the 1986 World Cup but pulled out two years before it was due to be played because of financial difficulties and Mexico hosted the tournament instead.
Brazil hosted the 1950 World Cup and also launched a bid for the 2006 event but withdrew two days before the final vote.
South Africa has been chosen to host the 2010 finals.
Benelux launches World Cup bid
Sven lost sleep over World Cup