Mildew is a good possibility, but let me suggest something else:
Is the stopper black? How about the ball floater (assuming it has one)? Those will eventually break down over time from being exposed to the water in the tank. Minerals in and the pH of the water cause them to break down. It may be a very long time before it happens, but it will happen, and the result is black material in the bowl. It’ll stick to the ceramic, because polymers do that, and depending on how hard the water is, they may bond with the ceramic by mixing with mineral deposits. You say they are easy to remove, so your water is probably not very hard.
If they cannot be completely removed, Lime-Away sprayed in the bowl should get rid of everything. Spray it over all the surfaces not immersed in water, wait about 10 minutes, and scrub.
Replace all the parts that are polymers, or “rubber.” That’ll mostly be the floater and the stopper. You’ll probably pay between $15-35 at Home Depot for replacements, they’ll come with directions, and take about 15 minutes to replace. Shut off the water to the toilet, and flush it to empty the tank. You should not need more than ordinary hand tools to replace them.
As far as cleaning out the little holes: I have had to clear out a toilet when we had extremely hard water, because we had well water-- we had to flush out the little holes about every six months with Lime-Away, or the toilet got too clogged with mineral deposits to work.
Here’s what we did. Do not laugh. This worked.
But a disposable douche bottle. I am not making this up. Empty the vinegar-&-water, or whatever. Rinse. Fill it with Lime-Away, if you think you need it, and stick the business end up the small hole in the toilet bowl on the bottom toward the front. Squeeze. The Lime-Away comes out all the little holes.
Repeat until it flushes clean. Usually took us twice. Then we flushed once with water-- just a regular flush with the handle-- then we filled the douche bottle with a 25% bleach solution, and douched the toilet again.
Before we did this, we had to replace a new toilet after just 2 years, because it got completely limed over. Started doing the every-six-months Lime-Away/bleach douching, and the next toilet lasted until we moved, which was more than 13 years.
If you don’t think you need Lime-Away, don’t worry about that; but a 25% bleach solution will kill mildew. Just keep flushing until the water runs clean.
Somebody is probably going to say “just put the bleach in the tank,” but if you do that, first of all, it’s not good for the polymers in the tank, and second, it won’t be very concentrated. You’ll have to use more, and spend more time. Plus, flushing by squeezing a douche bottle uses a lot more force than the force of gravity if you put the bleach in the tank. I am the voice of experience here.
Throw the douche bottle out. Don’t try to save it for the next time. The chemicals will break it down, and it will probably leak when you try to use it again-- at the least, it will lose its shape, and won’t work right.
Good luck!