On 2 February 1899, Judith married Neville Stephen Lytton, the youngest son of the Earl of Lytton. The marriage took place in Cairo, and they later returned to England and moved into a house in the grounds of her parents' estate, Crabbet Park, near Crawley, filled with relics from Judith's great-grandfather, Lord Byron. The couple later had three children: Noel Anthony Scawen (1900–1985), Anne (later known as Lady Anne Lytton) (1901–1979) and Winifred (later known as Lady Winifrid Tryon) (1904–1985). She later became estranged from her husband, and the couple divorced in 1923. Neville soon remarried, but Lady Wentworth never did, focusing on managing Crabbet Park until her death.