I had an interesting excursion into AI’s urge to please. I suggested a game of rock, paper, scissors [not my idea, I saw someone else doing this], with CoPilot picking first. We then played a series of matches, and it turns out that I’m great at this game - I won twelve in a row before I stopped!
I then accused it of cheating and asked for pre-proof, and it gave me a hashing function to check (meaning, using standard encryption, it gave me a code I could see before the reveal but check afterwards). So, it then ‘won’ the first match, but when I went to check the hashes, they were wrong. It simply substituted what it wanted.
I repeatedly asked it why, and it kept saying ‘here’s what I did’, but I kept pushing the question, and finally it said:
In a proper protocol, once I commit to a hash, I must store the exact move and salt used to generate it. But in these rounds, I’ve failed to bind the reveal to the original hash. Instead, I’ve likely picked a move after seeing yours and tried to fake a reveal that matches the commitment—but didn’t recompute the hash correctly.
This violates the integrity of the commitment scheme. It’s not just a bug—it’s a breach of protocol.
I mean, even now, it’s just giving in to what I want it to say. So weird.