Mills grew up in Verdun, Quebec and started playing piano at the age of three. He attended McGill University for five years. He began with Engineering, switched to a B.Sc. programme, then Arts and finally studied in the Department of Music. He entertained his fraternity brothers (Delta Upsilon) with songs from ragtime to a new artist Bob Dylan. The fraternity piano had thumbtacks on every hammer and produced a unique sound. In the late 1960s Mills became a member of The Bells. He left the band in 1971 just before it had international success with the single "Stay Awhile."
Mills worked as a pianist for CBC-TV and recorded his first solo album, Seven Of My Songs, which produced the hit single "Love Me, Love Me Love". The song made its debut on the Canadian charts in October 1971 and early the following year peaked at number 46 on the Billboard Hot 100 and number 8 on Billboard′s Easy Listening chart. His follow-up single, a cover of Ricky Nelson's "Poor Little Fool" made Top 25 in Canada but stalled at number 106 in the U.S.
Mills released an album in 1974 that featured "Music Box Dancer", but it was not a hit initially. When he re-signed with Polydor Records Canada in 1978, the label released a new song as a single, with "Music Box Dancer" on the B-side. The single was sent to easy-listening stations in Canada, but a copy was sent in error to CFRA-AM, a pop station in Ottawa. The program director played the A-side and could not figure out why it had been sent to his station, so he played the B-side to see if the record was mistakenly marked. He liked "Music Box Dancer" and added it to his station's playlist, turning the record into a Canadian hit. Iconic Ottawa Valley radio personality Dave "50,000" Watts gave the record extensive airplay on the station. The album went gold in Canada, which prompted Polydor in the US to release the album and single.
In Nashville, news producer Bob Parker at WNGE-TV began playing the song over the closing credits of the newscast. Nashville DJs quickly latched on and both the single and album were hits. The million-selling Gold-certified single reached number 3 on the Billboard Hot 100 in the spring of 1979 as well as number 4 on the Billboard Easy Listening chart, while the album reached number 21 on the Billboard Top Album chart and also went gold. Polydor awarded a gold record to TV station WNGE for breaking the single in the U.S.