She later had an affair with Gordon Lightfoot. After the affair ended, Smith returned to Levon Helm and the circle who comprised The Band.
Smith became an employee and then mistress of Canadian singer-songwriter Gordon Lightfoot in the early- to mid-1970s.
The Smith-Lightfoot affair was volatile and illustrated in the lyrics of "Sundown", Lightfoot's Number One hit and most financially lucrative song. It reflects the dark feelings Lightfoot was experiencing at the time. Drinking too much and married to another woman, he on one occasion broke Smith's cheekbone in a fight. Lightfoot has stated of his three-year relationship with Smith, "I was sometimes crazy with jealousy".
Bluegrass musicians Bruce and Brian Good, The Good Brothers, who were one of Lightfoot's opening acts during that time, got fired by Lightfoot for "flirting" with Smith. Smith was cited in divorce papers, and shortly after his affair with Smith ended, Lightfoot was a party to the most expensive divorce settlement in Canadian history to that date.
In a 1975 interview, Lightfoot expanded upon Sundown and hinted at the worry he experienced in his relationship with Smith:
"All it is, is a thought about a situation where someone is wondering what his loved one is doing at the moment. He doesn't quite know where she is. He's not ready to give up on her, either, and that's about all I got to say about that."
Lightfoot gave another insight into his relationship with Smith in a 2000 interview when he remarked upon "Sundown" being:
"a back-alley kind of tune. It's based on infidelity -- I've seen both sides of that."
In 2008, Lightfoot gave an interview confirming that "Sundown" was written with his then-girlfriend in mind:
"I think my girlfriend was out with her friends one night at a bar while I was at home writing songs. I thought, 'I wonder what she’s doing with her friends at that bar!' It’s that kind of a feeling. 'Where is my true love tonight? What is my true love doing?'"