I agree that that Goedel’s incompleteness theorems do not threaten the idea that the brain instantiates an algorithm, and accordingly, I disagree with Penrose’s argument. But I think Penrose makes a good argument and I don’t think its obvious that he is wrong.
Penrose’s argument is based on Goedel’s proof. Goedel’s incompleteness proof shows that there are sentences (or theorems) within any consistent formal system, roughly an algorithm, that cannot be proven within that formal system. Another way it is often put is that any system that is complete (meaning that the system can generate any true sentence about itself) cannot be proved to be consistent within the system. I may be out of my depths a little here, since there are really two or three related types of incompleteness that Goedel showed in several related proofs.
Furthermore, it is possible to create a sentence within any system that says of the system that the system is consistent (assuming it is consistent). But, Goedel showed, if the system is in fact consistent, it will be impossible to prove within the system that the system is complete. The system will not return an answer on the completeness issue. But we humans can be inventive and find ways to prove the completeness of the system by using methods that lie outside of the formal system. So we humans can know something that the formal system does not. Therefore, Penrose argues, human beings must not be using a formal system. If we were using a formal system to do mathematics, we could never prove the completeness and consistency of our own system – the one we use to do mathematics. But Penrose argues, we can do both in theory.
This is a very rough sketch and I may have it slightly wrong, since it is long time since I read Penrose’s book. Penrose is not the first to come up with this argument, and it has been much criticized over the years.
Penrose has been criticized for assuming that we are in fact consistent (he rejects the possibility that we are actually inconsistent as a matter of mathematical faith). Also, Penrose has been criticized for rejecting the possibility that our system might be so complex that we could never prove its completeness.
I think that simply getting things wrong or making mistakes and errors in thought and judgement does not prove that we are inconsistent or incomplete. A computer doing mathematical proofs using a formal system can make mistakes even if the system is consistent – computers can make mistakes if they overheat, if they have viruses, or if they are just full of bugs. Likewise, we humans often make mathematical or logic mistakes even if we know our logic or math – many things can interfere with the smooth operation of our brains. But when we review our thinking, we can catch the error because our system is not bad, it just didn’t operate the way it was supposed to.
So I think that Goedel’s theory does present real questions as to whether our minds are formal systems, and I don’t believe the mere fact that we make mistakes refute arguments, such as Penrose’s, that are based on Geodel’s proofs.