People might not want to come over to your house or apartment or ride in your car because of the smell. That could limit your social life, or your kids’ social lives. Your social life might be limited even further if you’re not willing to refrain from smoking in other people’s homes or cars.
Most cars I’ve rented have had a “no smoking” sign on the key ring. That limits your options when you’re renting a car, or means that you can’t smoke in the car while you’re driving a rental car.
Some hotels or hotel rooms are smoke-free. That limits your options when you travel.
You have to spend time outside in bad weather.
Airplanes and many other forms of mass transit don’t allow smoking. If you’re not willing to refrain from smoking for that long, that limits your travel options. Or it might just make air travel that much less pleasant for you than it already is for non-smokers.
If you’re looking for a roommate or want to rent out a room in your house or apartment, it could limit your options.
It would make renting a room, apartment, or house more difficult.
If you smoke in the car, and later want to sell the car, that will be a problem. I’m not sure if a dealer would consider that when evaluating a trade-in (my guess, though, is that they would), but if you were trying to sell the car yourself, it certainly would affect how easy your car would be to sell and how much money you’d get for it.
You almost certainly have a higher risk of a fire at home than a non-smoker does, since you have burning materials around much more often.
You take breaks from your work that a non-smoker wouldn’t. That means either you have lower productivity than a non-smoker, or work longer hours. Lower productivity and less time at your desk per day could affect your career, as could the social stigma of smoking. Working longer hours just sucks.
It’s legally and socially acceptable to discriminate against you, in a way that it wouldn’t be to discriminate against a religious person or (in some areas) a gay person. If someone says they don’t want to hire you because you’re a smoker, you have no legal recourse. It’s quite a different situation legally if someone tells me they don’t want to hire Jews. The distinctive smell of a smoker’s hair, clothes, and breath makes it hard for you to hide the fact that you smoke when you go on a job interview, and makes it easy for someone who doesn’t want to hire smokers to do so. It’s a lot harder to hide from a prospective employer than religious practices (at least some of them, a burqa or yarmulke would be harder to hide) or sexuality.
Speaking of hair, it will make your hair look worse if and when it turns white or gray. It will give your hair a yellowish tinge that most people don’t find attractive (this is one reason why old ladies used to get blue rinses- to cover the yellowish color in their hair from cigarette smoke).
