If we are talking about DC circuits, then the capacitor is almost always for filtering and voltage stability. Think of all of the little wires and connections as resistors. If a component draws an excessive amount of current suddenly, the voltage at that component will drop, possibly enough to cause a malfunction. Or, worse, the voltage change will cause an oscillation inside the part that feeds back into the current draw making the oscillation worse and suddenly your circuit is howling like a banshee.
Capacitors are small energy storage devices that will help to smooth out those fluctuations.
There are two basic types of capacitors. You have large electrolytic capacitors, which are used for heavy duty bulk filtering. These have large capacitance values, and they look like little tin cans, because you can basically think of them as kind of an electrical fruit roll-up. They have two metal foil sheets with a bunch of goo (the electrolyte) between them, and the whole thing is rolled up and shoved into a metal case. Electrolytic capacitors are great for high capacitance value, but at high frequencies they suck. So for higher frequencies, we use chip capacitors or small ceramic capacitors (mostly chip capacitors these days). Often in power supplies you will have big electrolytic capacitors paralleled with small chip or ceramic capacitors. The electrolytics do the bulk filtering, and the smaller capacitors do the high frequency stability.
In order to keep the power supply stable at each integrated circuit, I personally (as well as many other electronics designers) will just out of habit put a small capacitor between the positive and negative voltages right at the chip, keeping the circuit traces very short.
Sometimes you’ll put a resistor across the power supply to give capacitors (and maybe inductors or relays if you have any) a place to dump their stored energy once the power is shut off. Otherwise that stored energy can go elsewhere in the circuit, possibly somewhere bad. Often instead of just a simple resistor you’ll have a reverse biased diode in series with a resistor. The advantage of this is that no current is being wasted heating up the resistor while the power is on.
Similarly, high voltage circuits will often have bleeder resistors across energy storage devices, for similar reasons. Old cathode ray type televisions usually had bleeder resistors on the tube, and microwave ovens usually have a bleeder resistor across the big capacitors. Once the power is shut off, the energy in the capacitors gets turned into heat in the resistors. Without the resistors, those capacitors can store energy for quite some time, and could give you a nasty shock if you were to open up the microwave oven and touch them.
For AC circuits, you also have what is called a Metal Oxide Varistor, or MOV. Sometimes you’ll have these in DC circuits for similar reasons. An MOV has a fairly high resistance until you exceed its turn-on voltage, at which point its resistance drops significantly. The point of using these on power supplies is to clamp down on any over-voltages, thus suppressing electrical spikes by shorting them out quickly. The MOV is often destroyed when this happens, but the idea is you kill the MOV so that the circuit it is protecting survives.
If you have a question about anything specific, give us a link to the video or web page you are looking at.