It’s just five miles off the British coast, and seems pretty well-developed: File:Isle of Wight OS OpenData map.png - Wikipedia
Wiki says it’s been discussed but not done: Isle of Wight - Wikipedia
Just not cost-effective? Some other reason?
I would say that there’s not enough call for it to justify the cost.
Just found this, too: Transport on the Isle of Wight - Wikipedia
Looks like there’s local opposition.
You pretty much answered your question with your opening statement: “It’s just five miles”
A pretty good distance for a bridge or tunnel. Doable, but not coming out of the rainy day fund.
Which means multiple billion of dollars for the least expensive link. What is the offsetting benefit that pays for this (and overcomes the objection that you’re fundamentally changing the character of the island)?
The Confederation Bridge is 12.9 km (8 miles) long and was built with special features for crossing ice-covered water, and it cost 1.3 billion Canadian dollars in 1997. It was built to join Prince Edward Island (pop. 141 000) to the mainland.
A link to the Isle of Wight would be shorter and in a less severe climate.
Isle of Wight has approximately the same population as PEI too. An interesting idea. First I want the Irish Sea Tunnel to be built. 
But people want to go to PEI. 
The toll on that Canadian bridge appears to be almost £29 although you only pay it going one way. Seems kinda steep, no?
It appears that the PEI climate and corresponding difficulties with water transport were a key argument for that bridge. A less severe climate means easier ferry service.
And it’s not clear at all that the whole project was a good idea. It seems the Fixed Link has been good for some sectors of the island’s economy, not so much for others. The government pays $44 million per year to the bridge consortium, which also charges tolls to cover their construction cost overruns, while struggling to suitably maintain the structure. In 2032, the whole aging, expensive thing will be handed over to the government. I wonder how people will look on it then.
I guess it depends on what a ferry trip would cost.
A fare point…
Are we talking about a bridge from the Isle to England, or to France? It might make a difference.
England. It would obviously make a HUGE difference if it were to France. Much farther away.
Would it be labeled a bridge to nowhere? Maybe there’s some political opposition for that kind of reason?
Yes, it’s too dear.
You’d have to scrimp and save…
They’ve probably heard stories about how Wight Trash will move in and lower property values.
Got it. 
If it were a toll bridge, you would need a ticket to Ryde.
Nobody really wants one enough to justify the cost of building it - and it wouldn’t be a cheap build either, as The Solent is very busy with large shipping at both ends.
Also, it would enable the verminous American grey squirrel to invade one of the last strongholds of the native red squirrel (although I doubt this is a deciding factor really).