Neal Conan interviewed O’Brian in 1995. Conan mentions that Aubrey and Stephen almost fight “their second duel” over Diana.
That is the only challenge between them that I remember.
If the question was asked, I missed it. When did Marturin become a spy?