I didn’t see the OP asking about artificial intelligence, per se, but rather about how far can computer programming go.
I look at it this way: Is an insect intelligent? Or does it simply have a very complex set of instructions that govern what it does, when it does it, and how it responds to various internal and external stimulii? I mean, a bee has this basic instruction that says, “Fly around until the scent detectors are stimulated by molecules of a certain type, change direction of flight such that the scent detectors get more stimulation, when an object is eventually found (the flower), crawl around until certain substances are found, fly back along the original route to the hive, perform certain actions that will relay the pertinent information about the found source of the desired substance to the other bees, deposit the substance at some prescribed location. Repeat.”
That’s highly simplified, but the basic point is that you don’t really need “intelligence” to be an insect. A bee doesn’t think, “It’s time to go look for nectar”, or “I don’t really feel like flying aorund right now, I think I’ll just kick back and watch some TV.”
I believe that it is well within the abilities of current software technology to write a program for an artificial bee that would be indistinguishable – at least as far as behavior goes – from a real bee. The reasons that an artificial bee couldn’t be made today are hardware limitations, not software-related. We don’t have anything that can store a large enough program in something the size of a bee, and we don’t have the motors and servors and power source that could make something that small.
Moving up from insects, I would ask: Is a cat really intelligent? Or does it just have an extremely complex set of instructions that take as input lots and lots of parameters (the state of the cat’s stomach, what’s coming into its eyes, its ears, its nose, etc.) and then govern the cat’s behavior? Cats certainly learn, but we’ve had “learning” computer programs for years, programs that change their outputs based on changing inputs.
A cat learns that if it’s outside and wants to come into the house, if it meows, the big creature on two legs will open the door and let it in. But if you’re not home, how long will the cat sit there meowing? Does it have the intelligence to conclude that, after two minutes of no response, the two-legged creature is not coming? Or does the “meow to get in” program just keep running and running until something else (hunger, sight of a bird, a dog in the yard) over-rides it?
I believe that, theoretically at least, we could write an artificial cat program. Obviously, it would be huge, gigantic, and more complex than any program ever written, and would take who know how many hundreds of software engineers and decades to write. But, as relates to the OP, I don’t think it is necessarily beyond the theoretical limits of programming.
So I’d say that we’ve got a long, long way to go before computer program hits any kind of limits. I think what will always limit us will be the hardware, not the software.