Somewhat pertinent to the question, research suggests that nest-building capability is the result of both instinct and experience. Instinct indisputably plays a large role, but individual birds of the same species build their nests differently, and they also get better at it with practice.
There is obviously no definitive answer to this question but the attribution of behavior to “instinct” as a catch-all term for common species-wide behaviors isn’t really a scientific observation. Insects and arachnoids obviously do things by “instinct”, i.e. behavior that is evolved into genetically coded patterns into what passes for their distributed cognitive structures but more sophisticated animals with more complex brains and that demonstrate learned behaviors have some combination of programmed behavior, affective (emotional) responses, and some degree of problem-solving cognition.
As recent as thirty years ago the dominant thinking in cognitive zoology was that most animals do not really have any capacity for sapience and only a very limited ability to experience very basic emotions such as fear or joy but not more complex feelings such as jealous or envy that would required an ability to recall prior behavior and anticipate future events. A lot of work since then has demonstrated that while humans have a nearly unique ability to conceptualize future events in detail and in the complex grammar of language beyond what great apes and elephants are capable of (the jury is still out on cetaceans), animals display the same general range and types of affective responses as humans. The work of Jaak Panksepp in particular showed that rats are capable of play, grief, anticipation, and suffer something akin to depression under stressful conditions that retards normally demonstrated cognitive capabilities. His The Archaeology of Mind: Neuroevolutionary Origins of Human Emotions goes into extensive detail on this topic.
It is not strictly pertinent to the question of the o.p. since studying brain structure doesn’t really tell us anything about what is going on cognitively any more than looking at a map tells you about traffic patterns but Suzana Herculano-Houzel‘s The Human Advantage: How Our Brains Became Remarkable is very instructive in illustrating how many of the assumptions about brain to body mass ratio, neuron counts, and structural comparisons between brains of different species has often been based on faulty assumptions, oft-repeated misunderstandings, and incorrect application of scaling and statistical techniques. (It was her work in stereology and “turning brains into soup” to get accurate counts of different types of neutrons that corrected the oft-repeated claim that humans have 100 billion neurons when we really have just 86 billion neurons, a difference that eclipses that of the total neuron count of the orangutan.) It is clear from the work that making gross comparisons of brain vs. body mass is only roughly useful between relatively closely related species, very poorly across orders, and not really at all in comparing species in different classes. Our conception of the world and anticipation of its workings is very different than that of birds, which actually have different perceptive rates and see in a different spectrum which goes into the ultraviolet.
Whether nesting birds are ‘thinking’ about bearing offspring is something I don’t think we can never really know, but dismissing out of hand the notion that they have some anticipation of specific future consequences is not beyond conception. Of course, much of our behavior as supposedly sapient cognitive beings is driven by affective responses and many cognitive scientists have hypothesized that most of what we perceive as rational decision making is actually post-hoc rationalization of behavior choices already made at some ‘instinctive’ level well below our perception of consciousness.
The question recalls the classic Thomas Nagel “What is it like to be a bat?” Bottom line is that the most intelligent person blind from birth has … a hard time … understanding what the experience of “red” actually is; our understanding even the qualia of other creatures is outside our ken.
The questions to be addressed that answering the OP requires two basic issues to be addressed to start -
deciding how we know whether or not another creature, even one extremely similar to ourselves, has sentience.
what is intelligence anyway and how is it quantifiable, defined in a way that is not self-serving as “what humans have” as we solve novel problem in domains that are salient to us.
The questions have importance beyond wondering what a nesting bird “thinks” - They answers, whatever “we” eventually decide they are, will be needed to inform policies as artificial intelligences become more complex and capable; would be important if life is ever contacted elsewhere in the universe … how can we know what is intelligent life if we cannot even decide what earth-bound life meets that standard?
Beavers react to the sound of water running. In the wild, adult and adolescent beavers react to the sound by patching leaks in their dam. A young beaver, raised in captivity, will react to the sound coming from a speaker on the floor of its cage by piling sticks on the speaker.
That’s the part we can’t answer exactly. Is the sound creating stress for them, i.e., they don’t like that sound? Or does the sound make them think/feel that sticks are needed to cover the sound up ? Do they understand at all that the purpose of the sticks is to create a dam or are they just following their instinctive reaction to a stress inducing sound?
Birds can do some complicated things but the skills of beavers are much more remarkable, their ability to find and cut down saplings and use them to build huge dams and create their very complicated lodges. Beavers spend considerable time with their young passing on their skills and customs, yet young beavers separated from others will still instinctively develop and use the same skills.
My favorite encounter with a beaver was years ago when I sat beside a mountain stream and noticed a few feet away from me the beaver-gnawed stump of a young tree, maybe 1-2" in diameter. Then I glanced up, and saw that about a foot above the stump, the tree continued, hovering in midair.
It rocked my world for an interminable, terrifying moment.
Until I looked further up, and saw the vines wrapped around the upper part of the tree, holding it in place.
That poor beaver, responding to instinct or desire or planning by gnawing through a tree, and watching the tree bounce a few inches up into the air and stay there instead of falling over. No instinct could prepare it for that moment.