Leave out “ground” for the moment.
Electricity flows in a “circle”, or a circuit. The electricity goes out from the generator on one wire, and has to come back on another wire. Disconnect either wire, and you’ve got no electricity.
AC means “Alternating Current”, which means it makes a sine wave. It flows in one direction, slows down, then switches and flows in the other direction, slows down, then goes back in the first direction, etc. over and over again. It goes through a complete cycle (switching one direction then the other) 60 times per second in the U.S. In other areas, it switches 50 times per second, but it’s the same basic concept.
So, from a current flow standpoint, there really isn’t a difference between the “hot” and the “neutral” wire. In each wire, the current flows in one direction, and then the other. But whichever way the current is flowing, it needs to go out and back from the generator, so if you disconnect either wire, the current stops.
Now let’s get back to the concept of “ground”.
You can run an ungrounded system. In fact, hospitals do it all the time. It’s called an "isolated’ power system. The next time you are in a hospital, look for the red outlets. Those are isolated. Isolated systems are nice because you can touch either wire alone and not get shocked. You have to touch both wires to form a circuit, and then you get shocked.
Homes aren’t isolated. They are grounded. All they do is arbitrarily pick one of the wires, and call it “ground”. This wire is connected to the earth. Literally. I mean, they drive a copper rod into the dirt and attach a wire to it (in older homes it may just use the water pipe, but it’s the same thing). Now you don’t have an isolated system. To differentiate the wires, the one that is connected to the ground is called the “neutral” wire, and the one that isn’t is the “hot” wire, and they are given different colors. The reason for the names is obvious if you touch the wires while touching something that is electrically connected to the earth. If you touch the neutral wire, nothing much happens. But, if you touch the hot wire while you are touching earth, you complete the circuit and you get shocked.
So, you’re thinking, that’s stupid. Why not run the whole thing ungrounded, if it’s safer? Well, the reason is that it is awfully difficult to keep an isolated system isolated. Hospitals go through all kinds of efforts and even have yearly testing done on every single outlet to make sure they stay isolated. If you try to run a large electrical system (like to a bunch of houses) all isolated, you find that Mother Nature likes to randomly insert grounds everywhere. A tree branch can touch a line and short it to ground, for example. The power company simply doesn’t have the manpower to keep a large system like the U.S. electrical system ungrounded. Plus, you’ve got issues with lightning strikes and stuff like that. Having the system grounded makes it more immune to lightning, solar flares, and that sort of thing.
It used to be that homes only had two wires, hot and neutral. But this can cause you problems. If you’ve got some sort of electrical device or appliance and it has a metal case, you want to connect that case to the neutral since it is safe to touch. If you don’t, the hot wire could short to the case and you’d have this great big “hot” connection that could easily shock you. But what happens if there’s a fault on your neutral line? In that case the entire case could end up “hot” and again you’d have something dangerous. So, to protect against that, they added a third wire into the system, called the “ground”. This is a protective ground, and it never has current flowing through it. It’s a straight shot from the case of your device to earth ground. This way, in all of the different scenarios of a wire short circuiting to the case of your electrical device, you are much less likely to get shocked.
The earth does conduct electricity, so it is technically possible to take one wire from your generator and shove it into the ground, and take the neutral from your house and just shove it into the ground too. This saves you half of the cost of your wiring, since now you only need the “hot” wire. Power companies actually used to do this sometimes back in the early days, but it caused problems. Your “earth” connection isn’t always the greatest electrical connection, and varied with how wet the ground was and all sorts of other factors. So, these days, while everything is grounded (for residential circuits at least), the “earth” isn’t carrying the current back to the generator. The electricity isn’t going into the ground. It’s going back to the generator through the wires.
Physically, you have a transformer on the distribution line. One of these lines is connected to the earth, literally, through a grounding rod. All of the neutrals (the white wires) in your house are also connected to this grounding rod. All of the grounds (the green wires) are also connected to the grounding rod, at one point and one point only, and this point must be physically close to the grounding rod.
In older homes, the cold water pipe served as the grounding rod. This isn’t done any more.
Anything big and metal in your house is usually grounded. Even though we don’t use the water pipe as an earth ground, all of the water pipes in your home are grounded. You wouldn’t want them to “float” (not be connected to anything) since then a hot wire could short to them and you wouldn’t know it until you touched it and got shocked.