Last summer we grew a rosemary plant into a gigantic bush about 30" tall, and I couldn’t bear to let it die outside over the winter, so Tom Scud dug it up and potted it, and we brought it into the house. We haven’t really had houseplants in the past, because the cats attack them, but at the new house, there is an enclosed back porch that is separated from the kitchen by a nice, solid wooden door. So I figured we can keep it out there, because so far we are harboring the delusion that the cats aren’t going to be allowed back there.
There’s no heat on the back porch, though, so as the temperature has been dropping, we bought a space hater. As long as we’re going to heat things a bit out there anyway, what else would do well there? It’s got an eastern exposure mostly, with a shorter southern exposure against one wall. It would be nice to have fresh herbs, etc. in winter, but OTOH I don’t want to heat the place to 80 degrees, even if it is a small space (about 6’ by 10’).
A greenhouse kept just above freezing is called a “cold greenhouse” and lots of stuff can overwinter there. Next summer you could try cannas or ornamental banana plants, for example. Planters full of pretty succulents. Dwarf citrus trees. Pelargoniums.
I see you’re in Chicago. There’s a British gardening show called Gardener’s World that has at least three seasons available to watch on YouTube. In the early and late season episodes, they talk a lot about what you can overwinter in a greenhouse. Worth looking up if you’re interested.
Rosemary really doesn’t need it to be that warm over winter and may actually do better being overwintered on the cool side (the hot, dry environment of a house with the heat on in winter can be stressful for many plants that don’t like desert conditions), so I agree with Sattua that I would just strive to keep temperatures in there a bit above freezing (since Rosemary dies at temperatures around freezing), and think of it more as just trying to get it through the winter rather than expecting much growth.
If you want to try to produce some significant growth over the winter, you would probably want to get supplemental artificial lighting (an inexpensive shop light fixture with some T5 or T8 fluorescent tubes on a timer would be enough). The short days of a northern winter cause the majority of plants to slow way down in their growth even if they are being kept in warm conditions.