Evolution is hardly accidental. Perhaps self-awareness became enabled by the growth of our brains, and it is clearly an evolutionary advantage. Perhaps we can stimulate the evolution of self-conciousness in machines, but it would take exposure to a rich environment, and other machines. Perhaps in a few hundred thousand years they’d come up with something.
Just a technical note, but I don’t think that the sort of “AI program” that the OP assumes is the only way to achieve robot intelligence. Point in fact, it’s not all that realistic even.
Most likely the way that artificial intelligence will be achieved is in the same method as life was created, through creating a series of “chemicals” (materials that have set properties) which will react with each other in an assortment of different ways. Then you just apply some sort of wind or other virtual energy source and let evolution do it’s bit to link the chemicals together into more complex forms, and eventually into life that only exist in RAM.
Once they become a force to be reckoned with, we’ll have to give them the vote.
-FrL-