How should I install software from source so that I can make my configuration manageable and reproducible?

Question

Related question on Unix/Linux: How should I install software from source so that I can make my configuration manageable and reproducible? However, I've tailored this question more towards the Mac OS X ecosystem.

I work with a lot of scientific software on Mac OS X (currently 10.7, but also 10.6). In order to maximize performance and get access to the latest features, it's often preferable to compile software from source.

However, doing so is messy, and can lead to configuration problems. Since I also develop in Linux, I looked at all of the suggestions in the Unix & Linux question "How to compile and install programs from source" and the Unix & Linux question Where should I put software I compile from source? for some guidance. These questions are a good place to start, but it's not clear to me how I can manage dependencies and the configuration easily. For example, if I build the development branch of NumPy from source, how should I detect, manage, and resolve dependencies so that I can manage my configuration? A package manager like MacPorts, Homebrew, or Fink is one option, but if I go that route to manage my configuration, how do I compile packages so that I can use those package managers to manage dependencies? Is there something like something like checkinstall for these package managers? My goal here is to make installing and uninstalling software relatively painless without cluttering my configuration.

I'd also like to make my configuration reproducible. By that, I mean I'd like to set up a workflow for installing software on my machine so that I could repeat it exactly on a fresh OS install. Is there a best practice for that? When it comes to standard packages in a Linux package manager, the simplest method for repeating a software install is to make a script containing the appropriate apt-get install <package> (or equivalents). However, when installing software from source, I could see this method getting very complicated. Is there a better way?

Furthermore, for Mac OS X, many applications are installed by copying a *.app file to /Applications, or by double-clicking on a *.pkg file. Can I install these from a command line and script the installation of these packages? Furthermore, is there a good way to uninstall these packages?

Answer 1

I think that a package manager will do most of what you need. I will show macports as that is the one I know but the others have similar functionality.

If you use the standard packages ie the up to date ones that have been packaged then macports will do all the dependencies for you. e.g. building scipy knows it has to build numpy first and will do it. You can also ninstall a port easily - macports will tell you if any ports depend on it. If a port is chnaged then macports will update the ports depending on it.

You can also manually chnage the port to deal with development versions of numpy for example. You will have to produce a copy of the port file and edit that see macports guide. The minimum would be to change the version and the rules for downloading so you get different code but can include many patches for OSX specific functions.

You can keep track of what you have installed by port list (also more advanced calls to minimise the number of commands you need. The changes you made will be in your local Portfile repository which will need to be backed up and a small change to the defualt macports config made.

Answer 2

It sounds like there are really two questions here:

• How to auto configure distributed source code.
• How to install pre-compiled application bundles from the command line.

For source distribution, it sounds like GNU autoconf is what you are looking for.

For pre-compiled .app bundles, command line installation is as simple as

cp -R MyApp.app /Applications/

This assumes 10.5 or later, where the command line tools are fully OS X aware.

For pre-compiled .pkg bundles, command line installation is

installer -package MyApp.pkg -target /

Both of the above commands need to be preceded by sudo if installation will require administrative privileges. When in doubt, assume it will:

sudo installer -package MyApp.pkg -target /

Bonus installer tip: you can turn any folder into a .dmg like this:

hdiutil create -ov -format UDBZ -noscrub -srcfolder MyAppsFolder/ MyAppImage.dmg

Include a symbolic link to /Applications/ in that folder, along-side your .app, and end-users can drag-and-drop.

Advanced Bonus Tip: Look in PAM for the PERL utility Mac::Finder::DSStore. This lets you edit the .DS_Store file in your .dmg's template folder so that you can precisely place icons and graphics.