(Insurance laws and policy language varies from state to state, etc.)
Commonly for the total loss of a vehicle you are owed the Fair Market Value (FMV) of the car. Meaning, if you had decided to sell it the day before the accident, how much could you have gotten? Normally this is easily enough determined by looking at current prices for comparable vehicles in your local area.
Aftermarket specialty items and the effect they have on the FMV of your car depend on a number of things such as what kind of car is it? Which parts you’ve replaced, how long ago, what “performance” is gained over the original equipment, and most importantly: how did your changes improve the overall desirability of the car in the regular market (as opposed to the “Cavalier on steroids fetishists of America” cult). In a lot of cases, the answer is that the net effect of your efforts was to create a machine that suited your need and preferences, but not necessarily those of the public at large. Harley Davidson motorcycles are a classic example of this. Few of these hit the road exactly as they were manufactured–the owners like to customize them and make the bike uniquely their own. Well, you can spnd thousands of dollars on the paint job, chrome package and pipes YOU like, but when it’s time to sell you either need to find your clone to buy it, or you’ll need to understand that what turns your crank doesn’t necessarily do it for someone else. Cars are the same way to some extent. You can take a $2,000 Civic and pump all the money you want into paint, spinner rims, rice cannon exhaust, titanium shift knobs, nitrous boosters, octane-reading fuel gauges, concert-replicating stereo gear…but still, it’s a $2,000 Civic. OK, maybe a $2,500 Civic. Because cars that are that highly individualized are going to be a turn off to most of the rest of the world apart from thieves. And even THEY will take it apart and sell the pieces.
But back to the question at hand: What’d ya do? Did you replace existing functional equipment with slightly better functioning equipment? A 1990 Mustang GT comes with an engine, replacing it with a brand new one for $8,000 doesn’t make it THAT much better than any other 1990 GT, it just means that the engine is in new condition–and that’s worth something to a potential buyer, but nowhere near $8k.
Your best bet here is to step away from your car for a second (This is a lot harder than it sounds) and approach your car as a buyer. How much would YOU pay for it? Be critical–how much are you going to care about a 3-year old exhaust system? Ceramic brake pads that have already seen 20k miles? Is it a hot-looking car? If so, do you think the previous owner drove it like it was precious? or did he drive it like it was a hot car? In which case, what do you think is left of the modifications? Maybe the owner has worn them out–or maybe worn other less obvious parts. Maybe the rear end is ready to fail because it’s not rated for the output of the high-end engine? All that.
If in the end you look at the $5,000 that the insurance is offering you, and you can’t buy anything comparable to the car you had, then you need to be able to explain to a disinterested 3rd party what a reasonable settlement would be. If you can do that then you’ve got a case. If it looks like you’ve got a case, then the suit would be directed at the party that hit your car. You name that person directly in a suit. Defense will be provided by his insurer. They will support their offer to the judge, you will present your argument to the judge, and a binding decision will be rendered.
What kind of car and what kind of mods are we talking about?