He married Sarah Maria Chapman in 1876 but already had one child, Ruth (b.1873, Chatham). Ruth was later joined by Daisy (b.1880, Stepney) and Primrose (b.1884). Information about married life with Sadler was given by his wife (then living at 3 Skinner Street, Chatham, Kent) in a report taken by Chief Inspector Donald Swanson on 21st February 1891. They met in Chatham when James was a sailor and were married at St.John's Church, Chatham. After their marriage, they went to live in Elephant & Castle, South London and at this time, he was working as a labourer at Torr's Tea Warehouse, Cutler Street, Houndsditch. However, Sadler did not like crossing London Bridge and so they moved to a house in Buck's Row, Whitechapel. The house, situated somewhere near the middle of the Row, was shared with some Brush Makers who worked in the Minories. They later moved to 77 Tetley Street, Poplar, when Sadler briefly became a conductor. Whilst living there, Sarah took a knife from her husband, refusing to say where she had hidden it. From Poplar they moved to a corner shop in Hurley Road, Lower Kensington where Sadler ran a business as a greengrocer. From there they went to Manor Street, Walworth and at this time, Sadler was working at a bed warehouse in Cutler Street. They did not stay in Walworth long, moving to Colebrook Terrace, Bethnal Green. By 1888, they were living at 2 Johnson Street, Commercial Road and they separated in August of that year after a quarrel. On 15th August, Sarah Sadler left London to live with her mother in Chatham. Around March 1889, Sarah suddenly heard from her estranged husband who had written a letter suggesting they meet at Fenchurch Street Station on a Saturday. The meeting took place (although she was late) and they spent their time together looking round shop before staying at a Coffee House opposite Mile End Gate. Sadler had to work the following day (Sunday), but they met again at 5.00pm. They went to a public house in Whitechapel where, after having a few drinks, Sadler began to quarrel with some customers. Sarah walked out and waited for him outside and when he emerged they began to argue. She suggested they go their separate ways, but Sadler wanted to take her to a place 'where a woman had been murdered', something she was not that keen to do. Sarah ran away when they got near Whitechapel Church, but James caught up with her and offered to take her to an eelshop for a meal, in order to make up for the quarrel. After eating, they returned to the Coffee House in Mile End Gate where they stayed until the Monday morning. Mrs Sadler then went back to Chatham. Following the murder of Frances Coles on February 13, 1891, Sadler was made to give an account of himself as a possible suspect for the crime. James Sadler was arrested on Sunday 15th February 1891 for the murder of Frances Coles and charged the following day. Sadler received legal representation from the Seaman's Union who had collected some strong witnesses in his favour; as a result, charges against him were dropped on 2nd March before the case could go to court. By May of 1891, James and his wife were living together again at 121 Danbrook Road, Streatham where they kept a chandler's shop, apparently financed with money that Sadler received from the newspapers following his acquittal for the Coles murder. Sarah Sadler wrote to the police on 10th December 1891 regarding an assault on her by her husband. It was quickly investigated - reports state that Sadler had not only assaulted his wife and treated her cruelly, but that he also threatened to take her life. Mrs. Sadler did not press charges against her husband, but was grateful for the posting of a police officer in view of the house. The matter died off for several months until the elderly lodger James Moffatt wrote to the police in March 1892, once again concerned about James' aggressive behaviour towards his wife. After Sadler had threatened to murder Sarah ('cut her throat') on 9th May 1892, he was summonsed before Lambeth Police Court on 16th May where he was bound over to keep the peace. In January 1893, James Sadler informed the police that he had left the shop at Danbrook Road and had taken lodgings at 108 Faraday Street, Camberwell.