You’re a fellow engineer, aren’t you? Engineers think like that, but common customers for hardware who are non-technicians? Look at TV sets or cars, the bigger, the better and more expensive. Why not also computers? 
ETA: they also ignore the history of computing. Miniaturization has been going on since the early days of computers and has always been a driving factor in the biz.
I actually think the “smaller things should cost less” idea is a relatively recent one. Decades ago, smaller computers (particularly portable ones) were something of a novelty, and nobody would bat an eye at the higher cost. Remember that computers started as massive mainframes, and as they shrunk down to become consumer items, being small was a real necessity. Computers took up a lot of space, even PCs designed for home use, and people craved a smaller footprint and would pay a premium for it. I also don’t remember complaints about thinner screens being more expensive during the transition from CRTs, because again, the freedom from a smaller size was a luxury that consumers badly wanted.
Forward to today, and I think people take it for granted that devices will be conveniently small. Now, a larger size often correlates with a greater cost; larger laptops with larger screens cost more than smaller ones, as do larger monitors in general. Same with phones; if a smart phone comes in two sizes, the larger size inevitably costs more (assuming it’s the same model otherwise). Consumers have now been conditioned to expect the opposite from what they once did. Paying more for a smaller computer now seems counterintuitive.
(As a bit of background, I was a computer salesman around 1999-2000, and I distinctly remember the transition from CRTs and the desire for miniaturization from those days.)
Well, you’re not giving us a whole lot to work with. If you have more specific requirements, like a certain budget or certain things you do in Photoshop often (some more complex filters might take more power, for example, or some of the 3D rendering stuff if you ever use that), that’s one thing. But otherwise what you’re asking for is just any generic computer.
If the most taxing thing you ever do on your computer is a bit of Photoshop, I think you just need a relatively recent computer — of any sort. It can be small if you prefer that, but otherwise, for any given budget, a regular-sized desktop will usually give you better performance. But it may be easier to find cheaper mini PCs too. Refurb or not, it probably won’t matter.
From what you describe, I don’t think performance is going to be an issue at all, and because of that, size and heat also probably won’t make much of a difference. Just get whatever you want, that either looks good or feels good or has good support or is cheap enough… absent information to the contrary, what you’re describing is just what any recent-ish computer can do.
What is “on a budget” to you? $200? $500? $1000?
At the $200-ish price point, it’s going to be slim pickings… used/refurb might be the way to go there, but you’re basically left with whatever you can get. Don’t have high expectations about performance or quality or longevity. This is the bottom of the barrel. Maybe it’s worth considering the N100/N150 machines that @echoreply mentioned.
At the $500-ish price point you’ll have more options. There, I honestly don’t think you’ll find a better computer than the Mac Mini or Macbook Neo (Photoshop runs fine on a Mac too). But if you’re tied to Windows, well, just get whatever you can find in that same price range. That Geekom A6 Mini doesn’t look half bad if you don’t care about the thermals. Never heard of the brand before, but IMHO it doesn’t matter… the budget name-brand computers (Dell, HP, Lenovo, etc.) at that price point are generally overpriced crap with poor support and poor components and filled with adware and bloat. HP in particular I’ve had a string of bad to terrible experiences with, though other Dopers like them. But anyway, at least on paper, you get a lot more bang for the buck with the no-names, and the reviews seem good enough.
At $1000 ish, you have a LOT more options, and I think it warrants a moment’s pause to consider what you’re actually looking for in a machine. I would not spend this much on a “generic” computer unless you have a reason to — either you want a specific kind of build quality, certain components, a long in-home warranty, or some other specific criterion. Otherwise, you’re probably just wasting money that could be spent on better peripherals (like a really nice monitor) or a specific component upgrade (like a much bigger SSD). Or save it.
Refurbs are fine too as long as there’s warranty and support or you’re comfortable diagnosing issues yourself. I’ve owned many refurb machines and actually they’ve never given me issues; it’s the new computers that typically give me problems. The refurb ones usually get tested more thoroughly before being resold.
If you have a Microcenter near you, ask about their in-house Powerspec builds for basic home/office use (not their gaming ones). If you have a Best Buy near you, check out their stash of open-box computers (you might have to ask an employee; many stores don’t display them anymore, but they are often in the back). If you have a Costco nearby, check their in-store selections (which are usually cheaper than the online ones).
If all else fails, just read some reviews and look for a budget computer on your favorite online retailer and limit it by price range and good reviews. Your criteria are generic enough that really any affordable computer should be fine.
Side note: I also have a ~10 year old janky Windows laptop I can send you for free if you want that. Or a ~7 year old Intel i9 Macbook Pro laptop (well-built, solid machine with a good SSD and CPU and a great screen and speakers) you can have for $250 — I can put Windows on that too, it being the last generation of Macs that can still run Windows natively.