Source Code and Compilers and Command Line Interfaces, Oh My!

So here I am (well, at least some of the time) in the brave new world of MacOS X, which looks (sort of) like Macintosh but lives and breathes and thinks in Unix. I have a thread in MPSIMS devoted to the everyday joys and frustrations of learning how to use it in the conventional end-user sense (settings and applications and so forth), but there’s another aspect of MacOS X I need to learn more about…

Mac OS X is FreeBSD at its core, and as with other flavors of Unix one of the attractions is supposed to be the availability of free software, lots of it, with pretty sophisticated functionality (albeit with pretty primitive interfaces most of the time). So, most of this free Unix software is available as source code; I would imagine that if I were to download some of it, I’d end up with folders full of files with names like “main.c” and “MakeApp *” and so on. OK, what now? Is there a freeware or shareware compiler that will compile for BSD Unix running on Mac hardware? Is there one already built into BSD? Or do I buy CodeWarrior or something?

Now let’s jump forward and assume I’m sitting here with my compiler…uh…well, what now? Select all the files in the Source Code folder and find a “Compile” command in a menu and sit back and watch it turn it into an executable program? Or does compiling from source code require knowing my way around C++ and my own computer’s architecture and making important changes first?

Finally, and most embarrassing, let’s jump forward again and assume that I now have my newly compiled Unix program sitting in a folder on my hard drive. I launch Terminal and I ls and cd my way over to the folder in question and…uh…well, now what? Does one launch Unix applications simply by typing their names at the command prompt, the way one would in DOS? (I tried that in Linux once and it didn’t seem to work).

See the “Development Tools” CD that came with your OS? Run the installer on that. Now, in addition to having Apple’s extremely sophisticated software development stuff, you will have all the standard Unix commandline development tools, including the BSD compiler and libraries.

Most source code you download will be in gzipped tar format (.tar.gz or .tgz) do uncompress these, do

tar -xzvf name_of_file.tar.gz

Once you have all the source files extracted, there is a 99% chance that there will be a file called README. It usually has build instructions. Most of the time, this consists of running a configuration script and then automake. It usually looks something like this

./configure (from the top of the source code directory)
make
make install

If all goes well, the app will compile and install. Keep in mind that more complex programs may not compile or configure properly because their configure scripts do not know how to set themselves up for Apple’s rather, errm, unique build environment.

A good place to find OSX ports of existing Unix software is http://www.versiontrakcer.com/

Assuming your PATH is configured correctly, once the app you want is installed, you run it by typing its name on the commandline, just like DOS. Try typing the first few characters or a program you know works and then pressing Tab to see something magical.

For Linux, at least, it’s rediculously easy. Just download something that says it’ll work with your OS and unzip it into a directory. Then read the README. It’ll tell you what to do in general, and if you need to do something specific to your OS. Mostly, it’ll just tell you to run these commands:
[ul]
[li]make[/li][li]make test[/li][li]make install[/li][/ul]
If you’re lucky, lots of strange things will scroll by and tell you that they’re happy and you’ll have a new program compiled and installed. Then you just type the name of it at the command line (assuming it’s been installed into a directory that’s in your $PATH). You could probably also go through a File Manager in your GUI and click on it.

If it’s a Major Sort Of Thing that you’re installing, most likely you don’t want to just run it; it might need to be called from another program, or run with special options. If you, for instance, make and install Apache, and then just type “httpd”, it’ll be pretty boring. Try installing a simple text-based game or something, and it’ll be more obvious what’s happening.

Also, when you type “ls”, you’re using a UNIX application by typing its name and viewing its output; it just doesn’t make any pretty windows (I don’t think; I’ve never use OS X).

Unix has a quirk when it comes to executing programs. It will only execute
them by name if the directory they’re in is in the PATH environment variable.

What this means for stuff you compile yourself is that may need to type
“./thisprog” instead of just “thisprog”. The “./” means “in the current
directory”. Just like typing “/bin/thatprog” would make the computer execute
the program named “thatprog” in the directory “/bin”, typing “./thisprog”
means "run the program “thisprog” in the directory “.”. The directory “.”
is an alias for the name of the current directory you’re at. You can
also type it in longhand. For instance, suppose you’re in “/home/mydir”.
You could type “/home/mydir/thisprog”.
You might want to pick up a beginner’s UNIX book. UNIX
has been around for a long time and has gotten very
complex. There’s a lot to learn. A good book is a
tremendous help. If you can find one published by
a company called O’Reilly and Associates, with some
kind of animal on the cover, those are generally really
good.
-Ben

I’d recommend you look at these two sites:

http://www.stepwise.com
http://www.macosxhints.com

Both of them have instructions and hints on compiling and installing various software packages, and using the command line interface.
A good way to start experimenting might be to try compiling the latest version of Apache. Full instructions are available on the Stepwise site. Stepwise also has a tracker for new releases of various 3rd party MacOS X and Unix programs, i.e. cool screensavers, utilities, etc.

One thing you should be aware of is that many unix packages are compiled with the intention of running under X Windows. This is the traditional Unix GUI, and is not a standard part of MacOS X. You could use X Windows apps with Tenon XTools (expensive) or wait for the XonX project to get a freeware version that is easy to install and use (they’re progressing rapidly). Without an XWin package, you’re pretty much stuck with text-interface CLI apps under Unix.

For general Unix stuff, I’d recommend you read some of the materials at http://www.informit.com, they have the full text of books like “Unix Unleashed” online with searchable indexes, and you’ll find them a godsend. The O’Reilly books are invaluable, I particularly recommend “Unix Power Tools” as the best general primer on the command line interface.

Many packages will split the welcome documentation into two files, README and INSTALL. INSTALL has the actual installation instructions; README has release notes, a description of the program, etc.

Unless you edit the Makefile or the configure program asks you for a path, ‘make install’ will put the application’s files in default directories, usually under /usr/local. The executable itself (or executables, there may be several) goes in /usr/local/bin, the configuration/data files go in /usr/local/lib or /usr/local/etc, and the man page goes somewhere under /usr/local/man. The README or INSTALL file should tell you how to change the default paths, but you don’t really need to change them.

Man(ual) pages are some of your best friends on Unix. ‘man ls’ will give you documentation for the ls command, including the behavior of every command line switch. Man pages are placed into numbered sections… section 1 is user commands, 3 is standard library functions, 5 is administrative commands, and so on. ‘man printf’ will show the page from the first section that matches… to force it to look in a certain section, use ‘man 3 printf’.

Most major applications will come with man pages; some will come with hypertext ‘info’ pages instead. You can often get basic help from the program itself by typing ‘ls --help’ or whatever.

Quoth ModernRonin2:

Usually, though, the path will include . , so no matter what directory you happen to be in, it’ll be in the path. If . is not in your path, then I would recommend putting it there: How to do this will depend on your setup.

And to answer the obvious unix-hater question of why “.” isn’t automatically in your path, I’ll point out that it can be a security risk. In many contexts, however, such as a machine a single person logs into, or one where all the users are trusted by each other (such as a work environment), it’s a completely acceptable risk, hence Chronos’ advice.

To clarify, the risk is that someone might put a trojan horse in a directory they have write access to (e.g. /tmp or their home directory), and if you cd in there some day and type (or mistype) a command with that name, you execute it. One way to reduce this risk significantly is to put “.” at the end of your path, so if you type “ls”, the command line interpreter will find /bin/ls before “./ls”.