The Welshman had trailed all day at the Belfast Waterfont, falling 6-3 behind at the start of the evening session, but stayed composed to win six out of the final eight frames, including the final two for a remarkable victory.
China's wonderful 17-year-old talent Yan is the youngest player to reach the final of a ranking event, but Ronnie O'Sullivan continues to hold the record as the youngest winner aged 17 at the UK Championship in 1993.
Northern Ireland Open
O'Sullivan wins final-frame shootout against Selby to reach final, Trump beats Sharav
It is Williams' first ranking tournament win since the German Masters in 2011, and is even more astonishing coming after he almost withdrew before the final with his wife Jo back in hospital fighting suspected viral meningitis.
"I was waiting on a phone call. She is back in hospital. I said to the officials no matter what the score is, if anything is wrong you come and tell me," said Williams.
"Tell me and I'll come out and make the decision. Thankfully, I haven't got that phone call to say anything was happening. I'll go back in the morning to the hospital, and spend however long it takes really to find out what the problem is.
I've had a lot of things going through my mind. I didn't know until this morning that I was able to play the final. I've feared six or eight months ago, I wouldn't win another tournament.
At the age of 42, it his 19th career ranking event and the first on UK soil since the 2003 LG Cup in Preston.
"I did say to myself a while ago, if I could get to 20 I would be over the moon. I'm on 19 now so just one more away. You never know," said Williams.
"I felt really good. The first eight frames were the worst I had played, but tonight I was back to playing really good again.
"He was just potting everything for fun. He is only 17 and is only one frame away from beating Ronnie's record. If he keeps improving he is going to be winning many tournaments."
Williams won the Alex Higgins trophy and a £70,000 cheque with a level of class and performance that brought him two world titles in 2000 and 2003.
Resuming 5-3 clear from the afternoon session, Yan threatened to blow away Williams as a career competitive high break of 137 saw him extend his lead to 6-3.
But Williams was unperturbed as he quickly regained the initiative as breaks of 62, 90 and 58 enabled him to restore parity at 6-6 heading for the mid-session interval.
With the tension palpable, Yan nudged himself 7-6 clear with a timely knock of 59, but Williams picked out a lovely red to a middle pocket on his way to drawing level again at 7-7.
Yan was in first in the 15th frame with a run of 60, but missing a tricky red to a middle pocket gave Williams the chance of a counter.
But he could not complete the clearance before a safety error late in the frame by Williams provided his opponent with the chance to move one up with a possible two remaining after he mopped up from yellow to blue.
Yan was first among the balls in the 16th frame, but could not capitalise on a brilliant long red as Williams emerged the dominant force to force the decider in the Northern Irish capital. He was again largely dominant despite Yan's best attempts to retrieve the situation via snookers.
"I am disappointed not to beat the record," said Yan after his first professional final. "But I will always have another chance so I will look to the next event.
"I really liked the venue, the crowd and the fans in Belfast. I'm looking forward to next year here.
"After this, I feel confident despite losing because I reached the final. I know how to handle this occasion so I will aim high for more finals rather than semi-finals."
FIRST SESSION REPORT
Teenager Yan Bingtao leads Mark Williams 5-3 after the opening session of the Northern Ireland Open final in Belfast.
Double world champion Williams made a late decision to play the final after his wife was taken back into hospital on Saturday with suspected viral meningitis.
But he was forced to play catch-up in the first session against 17-year-old Yan, who is chasing a piece of snooker history at the Belfast Waterfront.
If he can overcome Williams, Yan will replace Ronnie O'Sullivan by three months as the youngest winner of a ranking tournament.
O'Sullivan has held the record since lifting the UK Championship title at the age of 17 in 1993.
Yan led 2-0 and 3-1 with the highlight of the first four frames a break of 51 in the second frame.
A sparkling knock of 104 from Williams, 42, saw him claw back to 3-2 behind only for a 63 from Yan enough to restore a two-frame advantage at 4-2 clear.
Williams - chasing a first ranking win since the 2011 German Masters final and the first on UK soil since the 2003 LG Cup - played a lovely positional shot on his way to winning the seventh frame.
But he could not muster enough points during a visit to the table in the eight frame as the young Chinese player ended the afternoon two frames clear.
They play to a finish from 7pm with the first man to nine claiming the Alex Higgins trophy and a cheque for £70,000. The final is live on Eurosport and the Eurosport Player from 6:45pm as part of the Home Nations Series.
Northern Ireland Open
O'Sullivan sets up semi-final clash with Selby in Belfast