On 29 July 1833, he subsequently married Lady Charlotte Elizabeth Bertie. Together, they had five sons and five daughters, including:
Charlotte Maria Guest (1834–1902), who married Richard Du Cane (d. 1904), brother of Edmund Frederick Du Cane.
Ivor Bertie Guest (1835–1914), who married Lady Cornelia Henrietta Maria Spencer-Churchill (1847–1927), daughter of John Spencer-Churchill, 7th Duke of Marlborough.
Katharine Gwladys Guest (1837–1926), who married the Reverend Frederick Cecil Alderson (d. 1907), son of Sir Edward Hall Alderson.
Thomas Merthyr Guest (1838–1904), who married Lady Theodora Grosvenor (1840–1924), daughter of Richard Grosvenor, 2nd Marquess of Westminster.
Montague John Guest (1839–1909), a Liberal politician, who never married.
Augustus Frederick Guest (1840–1862), who died aged 21.
Arthur Edward Guest (1841–1898), a Conservative politician, who married Adeline Mary Chapman (d. 1931).
Mary Enid Evelyn Guest (1843–1912), who married Austen Henry Layard (1817–1894).
Constance Rhiannon Guest (1844–1916), who married Hon. Charles George Cornwallis Eliot (1839–1901), youngest son of Edward Eliot, 3rd Earl of St Germans.
Blanche Vere Guest (1847–1919), who married Edward Ponsonby, 8th Earl of Bessborough (1851–1920).
Guest died in 1852 and was buried in an iron coffin under a red granite slab in the local St John's Church, which had been built for him in 1827.