Eh… kinda.
With the car off, the battery should read close to 12.6 volts. Around 12.5 to 12.6 is good. Down around 12.0 volts is pretty well discharged. If the alternator is working, the battery shouldn’t discharge this low, assuming that the battery is actually good.
After measuring the battery with the car off, now repeat the test with the car running. The voltage measured at the battery should go up by maybe a volt or so. Your 13.5 number is good, but it assumes that the battery voltage with the car off is 12.5 or so.
But basically, the quick and dirty test is to make sure that the voltage goes up by a volt or so, maybe a bit more if you rev the engine.
In other words, to test the alternator, you’re looking for the voltage to go up by at least a volt or so. You’re not necessarily looking at the actual voltage. If the battery voltage is 12.1 with the car off and it goes up to 13.1, it’s probably the battery that is bad and not the alternator.
Note that this test works well on an older car like your 2000 Civic. It’s not necessarily a valid test on a modern car with a newer fancy shmancy smart charging system.
If you don’t measure the battery voltage with the car off first, then you won’t really know if it’s the battery or the alternator that is dragging the voltage down.
They tend to undersize batteries in cars these days, partly to save costs and partly to save weight. Unfortunately, this also means that car batteries often don’t last very long. The battery gets discharged too far during normal starting and starts to chemically self-destruct.
A lead-acid battery is made up of lead and lead oxide plates with sulfuric acid and water in between them to act as the electrolyte. When the battery discharges, both the lead and the lead oxide plates turn into lead sulfate, and the sulfuric acid turns into water. When you charge the battery, this reaction reverses, and the plates go back to lead and lead oxide, and the water goes back to sulfuric acid. If the battery gets discharged too far, the lead sulfate can form
crystals that don’t break up when the battery is charged, leading to a loss in battery capacity. This is called battery “sulfation”.
If you put in a larger battery, the battery won’t get discharged as far during use, and your battery plates won’t end up a sulfated mess. Instead of replacing the battery every 3 or 4 years, you might not ever have to replace the battery over the life of the car.
Of course, this assumes that you can actually fit a larger battery into your car. Many cars these days are made with just a very teeny tiny space for the battery, and you physically can’t fit a larger one in its place.
Temperature extremes are bad for batteries. Cars are more difficult to start in very cold weather, which puts more of a strain on the battery. Hot weather is also bad, since it increases the self-discharge rate for batteries (a battery that is just sitting there will discharge on its own, hence “self-discharge”). If the battery sits too long at a low voltage (below 12.2 volts or so), sulfate crystals are more likely to form.
If the car is going to sit for any length of time, you can get battery tenders to keep the battery fully charged and prevent plate sulfation. I got an el-cheapo one from Harbor Freight for about $10 that works pretty well. You can probably find something similar on Amazon.