Have you considered a second hand DSLR? If you want to play around with the fundamentals, you really need something more than a point-and-shoot, but it won’t kill you not to have a zillion million pixels, a three inch screen, or video capability. The way technology advances, and obsoletes everything that came before it, you should have little trouble finding a 3, 4, 5 year old DSLR at your chosen price point.
I’m Nikon-centric, so something like a D70 or D50 would be about as old as you’d want to go. Six megapixels, internal focus motor for older auto-focus lenses, control options from full auto to fully manual, and great image quality. All you lose over the current generation is some resolution that you didn’t really need, some low light ability, and the screen on the back is pretty small. But, you know, if you want to get amongst it with the hands on fundamentals of photography, on a budget, you don’t need the latest and greatest.
Get yourself the kit lenses that came with those cameras (the 18-70mm, or the 18-55, and/or 55-200mm), which will either come with the camera, or you’ll pick up for cheap separately. Then get yourself a fast prime, one of the 50mm or 35mm f/2.0, 1.8 or 1.4 versions, and go from there. If it turns out you like photography, it’s not a huge leap from there to a current model camera and more exotic lenses. If you don’t like it, at least you’ve tried.
You can get point-and-shoots and crossover models that let you have a go at manual control, but a DSLR is probably what you really want. Any brand will do, I just have Nikons so that’s what I’m familiar with. Don’t go older than say 2004, or less than 6 MP. If you have a friend or acquaintance who is into photography, they probably have an older camera they no longer use, and could lend or sell to you. Or auction sites, camera stores, camera clubs…