How do theoretical physicists choose what to work on ? After worrying about this question through a Ph.D. and two postdocs, in my case I think we can say: for the most part, badly.
The other disclaimer about the following is that I’m only going to discuss the sorts of theoretical physicists you’d find in particle physics/quantum field theory. This is mainly so I can write with some authority, but it also seems to be what the OP has in mind as stereotypical examples of the breed. To include those working in fields like solid state or astronomy would change some of the specifics, but even with the restriction there’s still quite some diversity. As it is, I’m trying to cover people as different as, say, phenomenologists and string theorists, for whom strategies can be fairly different. There are also factors like age and position: the practical choices open to a grad student are likely to be narrower in some ways than those open to a tenured lecturer/professor.
Firstly, given that the replies have tended to emphasise funding, it’s worth saying at the outset that this isn’t really a major factor in peoples’ choices. It’s slightly country dependent, but for the most part you aren’t scrabbling after the next grant when you decide what to work on. In many cases you’re being paid a salary that is intended to be a living wage throughout the year. (A tenured position in the US being the notable example; it often technically doesn’t cover the summer and you’re expected to cover this with grants.) Things like office, library and computer are covered by the department, though they may be getting the money for this from some grant. About the only other major expense a theorist incurs is travel and conferences and for these you may be looking for grants. And since most research in the area isn’t planned far in advance, grants tend to be awarded on the basis that you say roughly what you’re interested in. You’re actually being assessed on your track record to date. This is very different to other fields, where the expectation is that you detail research programmes in advance. With some of theoretical physics, there’s the informal recognition that people can switch direction very fast in response to developments, so what they say on a grant application may become obsolete within months.
What you may be worrying about is your next job. In fact, if you’re a postdoc, you will be. I’ll say something about this pressure at the end.
Furthermore, since we’re only talking about particle theorists, explaining experimental data ain’t likely to be a factor either. Unfortunately, there really hasn’t been any new experimental results to explain in the last twenty years. True there’s lots of new data and at any time there’s usually at least one anomaly kicking around, but hitherto they’ve always gone away. The Standard Model (SM) is pretty damn robust. That doesn’t mean there’s no interaction between theory and experiment. Phenomenologists (basically, the theorists who talk to experimentalists) keep busy by looking for new tests of the SM and generating the predictions required, both from the SM and possible rivals. However, I’ve long observed that it’s not good for the health of the field that it’s trained a generation - and I include myself in this - of phenomenologists who’ve never seen anything truly unexpected in the data.
Nametag gets sort of close to the truth, though it’s nowhere near as narrow as that for theorists. One way of thinking of the issue is that you have a set of tools, a collection of techniques you can apply to a problem. Some of these are utterly general and will be known to everybody in the field. Some are tactical tricks you’ve found have worked in the past. But a few will be fairly specialised techniques that only you and a few others are really adept in. Usually you’ll have picked up at least one by the end of your Ph.D. It might be a training in how to calculate complicated QCD Feynman diagrams, the quirks of asymptotic series in QFT, large-N approximations, the details of knot theory or whatever. Most theorists are unlikely to stray too far from the application of their favoured tools. For some, it’s quite possible to make a good and useful career out of applying a specific tool to the same sort of problem. Someone who can calculate those QCD diagrams could respectably do nothing else, using contacts with experimentalists to decide what particular calculations would be most useful for comparing the QCD predictions to new and forthcoming data.
Otherwise, you’ve got a set of specialist hammers and you’re looking for nails. You read the literature in the hope you come across a paper where you recognise a “nail” suitable for your hammers. You talk to people in the hope that you have complementary problems/skills. You network at conferences. In general, the less obvious the relation of the hammer to the area people have previously written about, the more kudos you’ll get. For example, the 4 otherwise subjects/techniques I mentioned above weren’t random. To those of us worrying about any one of the first three, it was a delightful surprise when it was pointed out in the mid-90s that they were related to knot theory. We all knew a little knot theory (and a lot of Vaughan Jones gossip), but none of us had had the detailed knowledge to realise the connection. Hence we were impressed.
More generally, you go through papers on subjects you’re interested in. If the paper really interests you, you work through it in detail. Perhaps you see a better way of doing things. Or is there something like that that you can apply their ideas to.
Thus again and again you can take ideas from one place and apply them somewhere else.
Another tactic is questioning assumptions. You read a paper and ask yourself what are the consequences of that assumption being wrong. This can lead to incrediably rich stuff. For example, any quantum mechanics textbook will say that there are only two types of particles, fermions and bosons. Well, it was realised some time ago that this isn’t true. One way it isn’t is if you’re only restricted to two dimensions. On a plane all the standard arguments about particle statistics don’t work, so it turns out that other possibilities exist and some of these have turned out to be significant in solid state physics (where particles are often restricted to being in a plane). I know someone who has devoted much of his career to thinking up ever more recondite ways in which the textbook argument here can be wrong.
That’s really a variant of generalising. Rather than looking at what’s the most likely case, asking what’s the most general form of something possible. This is closely tied to the example of GUTs mentioned in the OP. The SM is one example of a non-abelian gauge theory. What would the other cases look like ? (Actually, that’s an example of a really old and dull research topic.)
Then there are obvious unanswered questions. Why is the top quark so much heavier than all the other quarks ? If this worries you, you might start thinking of how the top quark might be different from the others. Which leads to a whole slew of non-SM theories. (Actually, I’m not sure it is abnormally heavy, hence I personally find this an uninteresting question.) This sort of activity usually falls under the name “model building”. You cook up some variant of understood theories in such a way that it answers such a question.
Do model builders believe their models ? Hmm. A friend has a model in which quarks actually have integer charges. A very unorthodox notion that everybody else thinks went out in the 60s. I recently asked him, does he believe it ? His response was “it’s possible”. Which is enough for him to invest the effort in the idea. And, since he’s on the verge of tenure, that’s probably the right decision.
Then you get fads. Stuff everybody ought to be working on. Some can be pretty shallow, being little more than a need to include some particular buzzword in all your papers. But others can be profound. In general, the younger you are, the more susceptable to fads you’ll be. I was having dinner last Saturday with a friend who’s an active US string theorist and in the course of the evening he commented that he didn’t see why a particular very well-known string theorist was publishing on D-branes. Why ? The guy in question doesn’t need to make a reputation and the young and hungrey postdocs will beat him to the hot D-brane topics. However fashionable D-branes are at the moment, they’re only an interlude on the hunt and he’d be better off doing something riskier and more long-term.
So the answer to the OP is, no, you never sit down with the blank sheet of paper and create brilliant stuff. You’re always reacting to what other people have already done. If you’re really lucky or very good, in the course of doing this you stumble across something utterly new. But even then it’s not out of the blue: from my very, very limited experience, it’s weird stuff you conjure up while thinking about what’s gone before.
The final factor is taste. We all have personal prejudices about what’s interesting. I can see that string theory is elegant, but it’s always been too far removed from experiment for me ever to have wanted to work on it. And I think, that to be remotely effective as a theoretical physicist, you have to have the conviction of your tastes. In some small sense you’re betting against the universe. When you’re playing that sort of game, betting with funding agencies and tenure committees is a bit irrelevant. It’s difficult enough to convince yourself that what you’re doing is good.