D’Oyly and his second wife Elizabeth Jane Ross moved to Patna in Bihar in 1821 when he became Opium Agent for the East India Company. Their house at Bankipore, a suburb of Patna, was a focus of artistic activity, and Elizabeth also painted as well as being a musician.