Found it! The Tutor: Being the Reminiscences of Thomasina Wragg by P.N. Dedeaux (Grove Press 1971). Not actually Victorian, but set in 1855 and written in that style, more or less. Quaint but pretty raunchy, with a lot more whippings, birchings, canings etc. than I remember.