(2) If my memory of the book serves, Gandalf thought that Thorin had a bit of a stiff neck, and needed someone with a different point of view to change the tone of the expedition. (Thorin’s obstinacy was underplayed in the film.) He had also known Bilbo’s ancestors, and knew that the Bagginses possessed more in terms of character than met the eye, even if Bilbo had yet to demonstrate it.
If you want to know why Gandalf gave a shit about the expedition in the first place, he was worried about the dragon putting the balance of power in the “evil” direction in that part of middle earth, and figured a thriving dwarf kingdom under the mountain could only be a good thing.
The dwarves needed another member of the party to avoid being 13 in number, and they also needed a burglar, the job Bilbo was specifically brought along to do. Since hobbits are small, quiet and nimble, presumably that also makes them good burglars. Gandalf thought Bilbo in particular was likely to make a good adventurer given his family history and wizardly insight into his character.
Bilbo represents the “common man” in the story, he’s the guy, comfy at home in his slippers, sitting down with a warm cup of tea, who gets ripped away on a wild adventure. He’s our eyes into the strange world. That’s really why he’s there, to build the story around; someone who’s a bit naive that we can relate to.
Other than that, yeah, to be the burglar, because hobbits are quiet and all. And then, maybe because Gandalf knew something more than he let on, which is certainly suggested later on.
I got the impression that Bilbo is acting as a lure. Gandalf isn’t exactly sure what he is fishing for but he thinks if he uses an innocent hobbit as bait then something will be drawn to it. Looks like he was correct.
Gandalf is really fond of hobbits. He’d probably been planning for centuries on centuries to utilize them in some way to defeat Sauron, which is why he had the rangers protect the Shire and keep the hobbits in relative peace throughout their generations.
Gandalf’s purpose in Middle Earth was, in large part, to encourage people to stand up against the darkness. His strong association with fire (and Fire) is symbolic of his role in kindling courage in the hearts of others–one at a time, if necessary. He knew from long association that a steady ember of that courage waited in the hearts of many hobbits, and in Bilbo in particular. His purpose, his reason for taking mortal form, his very nature impelled him to breathe on that ember and bring it to life.
Also, I daresay that he found it terribly amusing.
It’s not so much that he had knowledge, per se, that sending Bilbo along would allow the expedition to succeed. It would be more accurate to describe it as intuition, or a hunch. But the hunches of such beings as Gandalf are wiser than the learnings of we mortals.
Really, there wasn’t anyone else but a hobbit to send along with them. If they could have found a fourteenth dwarf, Thorin would have already done so. It’s not likely that Thorin would’ve accepted an elf, either. So the only other options left in Arnor were rangers (busy elsewhere), humans (apparently only at Bree… basically tall hobbits, anyway), or a hobbit like Bilbo.
Since rangers weren’t an option and there don’t seem to be many humans at Bree to pick from, the Shire had Gandalf’s best probability for finding someone suitable.
One of their considerations was not to have their party number “unlucky” thirteen. (Gandalf didn’t count since he wouldn’t be with them all of the way.) In any case, Rohan was far out of their way, and Laketown was at the very end of their journey. They didn’t encounter any real men between Bree and Laketown (Beorn not being an ordinary human.)
There were humans in Rohand and Laketown. The dwarves weren’t going to Rohan, not even near-to. And Laketown was at the end of their journey – they wanted a fourteenth so they wouldn’t have unlucky thirteen on their journey.
The only humans we know of left in Arnor are at Bree, or are rangers. There probably should be humans somewhere else around, JRRT just doesn’t ever show or mention them to us.