COMPUTERS: How does the Hardware link to the OS to the Program?

Factual Questions
1

Hello. I am interested in a layman’s answer to the following:

How does the Hardware of a computer link up, ultimately, to the program, on a computer?

WHERE I AM AT: Ultimately everything is hardware. But for practical purposes, it would seem that the hardware links to the operating system (which is the master-program, or set of programs) that the other programs that the person uses rests on. Further, any time that a program links to the machine aspect, it is all in binary code. Am I somewhere close to the mark here?

ONE LAST QUESTION: If the hardware aspect DOES only read 1s and 0s from any program, then I would assume that the machine aspect is designed to read a prepackaged language (different sets of binary do different things to the hardware). If so, what is that language?

THANK YOU For your time! (Note: Not asking about quantum computers)

2

OK, hardware is anything you can touch.
Most computers are based upon what has become called the Von Neumann architecture. This breaks the hardware up unto key parts. The most importnat idea is the division between hardware that does stuff to information, (the processor) and hardware that maintains information upon which the processor acts. In addition you need hardware that can talk to the outside - Input/Output, and a way to couple them altogether - often called a Bus.

The most important realisation is that the information managed in the system can be code to control the processor (the program) and the data that you wish to act upon. This unification leads to important aspects of what a conventional computer can do.

The processor is a huge lump of digital logic that operates in a simple cycle.

  1. It fetches an instruction from memory.
  2. It decodes the instruction.
  3. This decoding presents other parts of the processor with internal commands of what to do. Typically these will:
  4. Fetch an operand (basically a slice of data), maybe fetch another one or two
  5. Apply an operation to the operands.
  6. Stores the result back into memory.
  7. Works out where to get the next instruction from.
    Repeat.

Say you have a program that involves a step that adds two numbers.
The processor reaches the location in memory where there is an instruction something like:
ADD $1230, $1234, $1230
This might mean something like, fetch the value in memory from location 1230, fetch the value from location 1234, add them together and put the result in location 1230. (This is glossing over a huge amount of stuff.)

That add instruction is itself just a numeric value. A huge bunch of such instructions is a program. You can have instructions that test data values, and jump to other parts of the code, and so on.

But so far it is of no use unless you have an ability to talk to the outside world. So there is additional hardware that bridges the gap between data managed by the processor and devices that can do stuff. There is a lot of variation in how this is done. But in the end the processor needs to be able to write instructions to the hardware that controls the devices, send data to the devices, and read data from them. A neat trick is to make the interface where you talk to the devices look like reading and writing data to memory (aka memory mapped IO). So there are magic locations in memory that cause IO devices to do stuff if you read or write to them.

So far the above might allow you to make a very simple device controller. Simple enough to run say a deskside clock. When you want a bigger and more general purpose computer, you make more demands of the software. If you want the computer to be able to do lots of things - like an ordinary PC, you need something to coordinate all the various aspects of operation, and schedule them so that each gets a chance to execute on the processor. In the beginning we talked of time-sharing systems. And there was a special programme that oversaw this. Over time this programme acquired more and more capability and became what we tend to call the operating system. It is the first program to run on a newly powered up computer, and its job is to control things.

Asking about binary code. It is binary because all modern computers use binary to represent data. Data values and code values are indistinguishable except in the way the processor uses them. The processor gets its instructions as data values specific to its design. Code for an x86 (is on a PC) is different to that that runs on an ARM (most smart phones).

The bit about it all being data is important as it leads to the idea of high level languages. Nobody wants to write code in binary. So we write programs that translate from a higher level human readable form into binary code. Initially these programs just let programmers write the binary instructions in an easy manner. But soon much higher level languages evolved. The key trick being that we write a special program (a compiler) that reads the high level program description and creates the binary instructions that can be run.

The big win came about when there became high level languages that were independent of the hardware they ran on.

3

If I mention device drivers, does that mean anything to you? Those are the programs that provide software interfaces to hardware devices.

4

To me? Sure. I have written a few.

This gets us to some of the more interesting magic of how a modern processor architecture works - although when I say modern there is a lot of scope for different ideas depends upon the use case of the architecture. Embedded device controllers provide a very different world to a PC.

The standard theology of what an OS provides tends to focus on providing an abstract machine atop the hardware. The key to interaction between the OS running on the hardware to oversee and control stuff, and user/client code running, and how to interact with devices - which operate asynchronous to the processor, are provided with a couple of common paradigms. Interrupts and exceptions. Either of these cause the processor to drop what it is doing, save the state of what it was doing, and then start to run a designated piece of code. In architectures that provide protection mechanisms, the operating mode usually switched to the most privileged mode (usually called kernel mode.) In kernel mode the processor will allow access to all of memory, devices, and enables certain reserved instructions that ordinary user mode processes cannot execute. The code that runs in kernel mode is often just called “the kernel”.

The difference between an exception and an interrupt is that exceptions are synchronous with program execution - and are caused by program execution. Interrupts occur asynchronously to program execution and are caused by hardware conditions. Perhaps the simplest interrupt is the one created by the hardware clock, which causes regular execution of the operating system interleaving with user code.

Devices can range across huge amounts of complexity. We could consider a trivial serial line input. We get 8 bit ASCII characters, one at a time. Every time you press a key, an interrupt is generated by the device. There is a mix of hardware and software that allows the operating system to work out which device caused the interrupt. This is specific to the exact hardware. The interrupt fires, the processor stores away what it was doing, and runs the interrupt hander code. First thing the handler does is work out where the interrupt came from. There may be special registers that it can examine (possibly available mapped into memory, possibly special hardware that requires special instructions to read). Once it works out what caused the interrupt, it calls the appropriate specific handler code. That code might simply read the 8 bit ASCII value from the device - again, the device might be made available as a memory location, or there might be a significant amount of messing about with reserved instructions that read the value from hardware. The handler then passes the read value up to where it is needed (which is a whole separate miracle of abstraction.) Much the same happens to write a character value to a device. However this time the action is initiated in user code. The user code calls a special routine that eventually causes an exception that causes the operating system interface to be executed. Same deal, the specific exception works out what action is needed, and in the case of device output will call the device code to perform the appropriate action. Since it is running within the operating system, in kernel mode, this code also has access to the appropriate hardware. It may be as trivial as writing the ASCII character into a specific location to cause the hardware to output the character. Commonly the operating system wants to know when the device has finished what it is doing, and so may also expect the device to cause another interrupt when it is done. Once all this is done the operating system will find the next available user process, restore its state and resume execution of it - dropping out of kernel mode as it does so. To reiterate - it is the hardware that provides the abstraction of these modes and controls transit between these modes. They are the magic that allows the OS to do its work.

This brings up the question of asynchronous IO. The above example has the IO occurring synchronously with the requests, with the requesting code blocked awaiting the device. The OS commonly provides the file system and uses device drivers that control disks or SSDs, or talks on the network. It doesn’t want to be blocked waiting for operations that take a long time. So it will make requests of the device, and expect to be interrupted by the device when the request is done. Internally the OS may maintain queues of requests, with many outstanding, also modern devices may support multiple outstanding requests. The OS is expected to manage the circus.

A lot of the above is operating systems 101 level. The level that worked on something like the PDP-11 I first did such stuff on. Things get more complex as you get much smarter devices, I haven’t mentioned buses, the manner in which half the devices in a modern PC are on the same die as the processor or the next door chip, nor any mention of the fabulously weird and specialised mechanisms seen in high end embedded control processors. We are in a world where devices themselves run embedded processors with their own operating systems.

Writing a real device driver is mostly about reading the device manuals, and the page upon page upon page of device register definitions and working out what to poke where to make what you want to have happen actually happen. Then dealing with the asynchronous nature of everything. Typically you end up with an event driven model of code that reacts to input and output and requests, and otherwise shovel control requests about. I haven’t mentioned Direct Memory Access (DMA) or DMA controllers. Most devices use DMA, and another part of device driver writing is setting up data transfer requests by setting DMA requests. DMA systems are like little devices in their own rite, and they exist to shovel data about in nice big chunks for you. I have also been out of the game for quite a while.

5

I was going to write something about this, having been a senior systems programmer for many years in Silicon Valley.

But Francis_Vaughan has covered it very well, I can’t fault anything he said… no need to repeat it.

If there are any further specific questions, I’m sure we can try to help.

6

I would add that a lot of the concepts here can be very abstract, and if the OP has a question about something specific that would aid his understanding (e.g. , “What happens down in the hardware when I tap a key to enter a question for the Straight Dope Message Board in my browser?”) that might be a good way to help. I find that people will often “get it” a little faster when ir’s something they can see with their eyes.

But, by and large, no beef with Francis_Vaughan’s answers. Bravo!

7

What may be confusing is that the vast majority of IO devices and other hardware are pretty much standardized. One of the functions of installing an OS like Windows is it has a large library of drivers for the most common hardware - the video, keyboard, USB, network port, mouse, camera, printers, etc. The chips that handle the IO are also pretty standard.

Long gone are the days of trying to detemine the, let’s say, network card and then have to download the driver for that non-standard card and go through the install process. (Run the install program, so the driver is there. then go into the Windows configurtion and tell windows it has such-and-such hardware, and the driver is that…)

There’s the added bonus that a lot of other USB devices (esp. printers) will have the driver built in and when you plug it in, part of the USB driver function is to interrogate newly found hardware and ask what it is, and then download its driver - if needed - specific to that device, from the device over USB.

Then of course, Windows is also often asking for driver updates along with all those other Windows updates it does regularly. So if there are fixes or tweaks, you get them.

8

Expanding on this, since the OP asked about binary code. In the actual hardware this would be encoded as a “word” that’s 64 bits long (or 32 bits or 16 bits depending on the actual hardware).
In this example it might look like:
0000000000001010 0000001011001110 0000001011010010 0000001011001110

Where:
0000000000001010 = the ADD instruction (I just made that up; I didn’t feel like looking up real opcodes)
0000001011001110 = the first operand, in this case 1230 represented as a 16 bit binary number.
0000001011010010 = the second operand, in this case 1234 represented as a 16 bit binary number.
0000001011001110 = the destination, in this case 1230 represented as a 16 bit binary number.

(This is just a made up example. It won’t necessarily be split up into four segments of equal length like that, but I was trying to keep things simple in this made up instruction set.)

9

Also consider the good old days, with TRS80 or Commodore computers, because the concept is not much different.

You would poke data into certain memry addresses - poke this number into there and the following character appears on line X position Y on the screen. Peek this address - the IO chip - and you can see what is the code for the key that is being pressed. The IO chip might have a memory location for the data to be sent, another to say “OK, now send” and another to say whether you mean send or receive, and another to show the result of the operation. (Plus for serial ports, a memory cell setting how fast to send or receive, etc. - things get complicated…)

The whole bit about “interrupts” is that some processes - like sending data on a modem or network - are time or event dependent, so instead of constantly checking (which slows everything else down) the processor waits for the “interrupt” signal to say “drop what you’re doing and handle this…”

Usually, these processes are more complex - poke the data into memory cell X, and then poke into Y to number that says “send that data over the serial port”. So the drivers essentially do what the operating system calls for - how to translate a higher level command - “display this line of text on the screen” (i.e. program line "print ‘Hello, World’ ") into the lower level “poke the following sequence one at a time into the following location following this process, so as to achieve the desired result.”

10

There is a very low level of operating system preinstalled in a computer. It is a program. It is written to tie the various hardware devices together and get them in a state to operate higher level programs such as the operating system. Even if you have not installed an operating system such as Windows or various Linux flavours, the computer is running this low level program. The BIOS is one visible interactive aspect of this low level program. It is contained in a chip on the motherboard. An operating system such as Windows has code that talks very directly to this lower level program. It also has code that lets other programs tell it what they want the lower level program to do. Most programming languages also have the ability to talk directly to the lower level system. But it is easier to write it to talk to the operating system.

Is this at all helpful? I can try again.

11

The BIOS is an interesting innovation. It is important to be clear it is a PC only thing. Other computers don’t have a BIOS, although some may include something with similar attributes.

The original idea of the BIOS was to provide a veneer of compatibility over the top of the hardware, so that the operating system proper didn’t have to worry about differences in the hardware between models and manufacturers. The early BIOS can be credited was being one of the key enablers of the success of the PC platform. So the BIOS provided a standardised interface to an expected basic set of hardware. The operating system could be generic, without a whole mess of different device drivers and requirements for messing with the hardware setting to make it all work. I would hesitate to call the BIOS an operating system. Modern computers use a reasonably sophisticated version of the basic BIOS idea, again providing the operating system and its boot loader (more on that) to operate without needing to know the specifics of the hardware.

When a system powers on, the system is presented with a problem. How to you load the operating system into memory when the capability to access devices and run programs is in the operating system. This is the bootstrap. Called, because it is like trying to lift yourself into the air by tugging on your bootstraps.

The bootstrap process can be really simple. On power up, a processor will start fetching instructions from a known location Say location 0. You arrange the hardware so that there is non-volatile memory with useful instructions located there. One of my favourite pieces of vintage computer hardware is the CDC 6600 deadstart panel. (I would kill to obtain one.) The picture is on its side.

It really is what it looks like. It is a panel of large toggle switches. Each switch is a bit. These set the bits of the first few locations in memory. There are enough locations to set instructions to load the first block of a disk into memory and jump into the loaded code. That is enough to set in motion the process of bringing the operating system up.

Back in these days the computers were highly bespoke, and every time you added new hardware you needed modify the operating system. Heck, even Unix was like that. Recompiling the operating system with new drivers, or setting various lookup tables was something every system manager did. (For instance, if you look under the hood at Unix devices, there are files in /dev - these files are special files - there is no underlying file - rather they are “device nodes” and they map to a major and minor device number. The major device number indicated the device driver to use, and was nothing more than an index into a table, and the minor device number told the device driver which actual device to use.)

Anyway, the process of bootstrapping is quite fun, and over time has become quite sophisticated, with multiple layers appearing. Perhaps the biggest change was when Sun Microsystems introduced the ability to bootstrap a computer - one with no disk at all - over the network.

It is still worth keeping in mind that the vast majority of computers we encounter in our lives now are embedded systems, and they don’t have anything like the complexity of a PC of PC like machine. They run a wide range of software, from simple programs that run of the bare hardware, through to systems that run embedded versions of Linux. And there are a few operating systems that only exist in the embedded space, or are directed at highly reliable operations.