In general, if the warning light is red, one should stop driving as soon as practicable. This is the color used for low oil pressure (possible consequence, engine ruined requiring very expensive overhaul or replacement), overheating (possible consequence, significant engine damage requiring expensive repair), and low brake fluid (possible consequence, serious bodily injury or death).
If the warning light is amber, it’s generally not urgent if there are no other obvious drivability symptoms (i.e., the car has normal smoothness and power). However, it’s potentially serious and should be investigated as soon as is convenient. Also note that a flashing amber light indicates engine misfiring which could disable the catalytic converter, which can be an expensive repair.
False alarms are always possible, but it’s not wise to assume them.
The “brake” warning light, even though it’s red, is often not indicating an urgent problem. It’s common for the brake fluid to get low due to normal wear on the pads, and many cars use the same light to indicate the parking brake is on. But do check to make sure there’s not actually a leak or master cylinder failure.
Lights (amber/yellow) that say “check engine” or “service engine soon” or show a pictogram of an engine do cause a fair amount of confusion. (I’ll call this light the CEL.) The CEL is related to the onboard computer, and comes on when some circuit monitored by the computer is giving a reading outside the expected range. Typically this will relate to drivability (engine performance, ignition, fuel, etc.) or emission controls. It doesn’t literally mean the engine should be checked or serviced - it means the computer trouble code memory should be read to see what malfunction needs to be corrected. Said malfunction can range from the innocuous (loose gas cap) to the potentially serious (misfire, incorrect fuel ratio). To add to the ambiguity, some older cars used this same light as a maintenance reminder every X miles.
It is a shame that some people seem to be unaware of the importance of warning lights and gauges. Of course, sometimes it’s not a big deal when a light comes on or a gauge reading indicates trouble, but sometimes it is a big deal, possibly a very big deal that can disable the car and/or cost a lot of money to fix if the warning is ignored. The cost of ignorance can be high.
ETA: I’m not aware of any vehicle that relies on the CEL to indicate low oil pressure or overheating. There’s either a gauge or a red light for those items.