Concerned about his growing blindness, and not enjoying the social life in London, Bute retired to his estates on the Isle of Bute for the next six years. While recovering, Bute married his first wife, Lady Maria North, in 1818. Maria was one of the three daughters of the Earl of Guildford, and a wealthy heiress. £40,000 was settled on her at the time of her marriage and she was due to inherit a third of her father's extensive estates. Contemporaries considered Maria a kind and pleasant woman, but was often unwell and the marriage proved childless. In 1827 his father-in-law died and Maria inherited lands worth over £110,000.
By the aristocratic standards of the day, Bute lived a reclusive lifestyle. As a result of his personality and poor eyesight, he did not enjoy hunting, shooting, large social gatherings, nor did he like racehorses or gambling. His first wife's illnesses added to this sense of exclusion from wider aristocratic society.
In 1841 Lady Maria died, and Bute blamed his excessive focus on the dock programme for exacerbating his wife's illness.[18] As a result of the original marriage agreement, Bute continued to draw the incomes from his late wife's property for the remainder of his life, even though officially the estates would ultimately pass to Maria's sister, Lady Susan, on his own death.
Bute died in Cardiff on 18 March 1848, and was buried in Kirtling, alongside Maria, his first wife.