Sunday, February 17, 2008

The art and limit of dependencies

If you want to get a job done, you need to list up the necessary things and tools, and procedures to use and apply them. In other words, you have many dependencies on them.

In a modern computer software development, no piece of software can withstand without any dependency to other pieces. You do not want to make a C program without the standard I/O library. Dependencies to the tools are also important and critical; if your codes include those written in FORTRAN, C, C++, and Java, you need the four compilers and language execution environments.

Computer programmers have been making the tools to automatically resolve the dependencies. make is a popular one, derived from the UNIX programming environment, which parses the rulesets called Makefile and determine whether if you need to rebuild a result from the source files, by comparing the timestamps. If make finds out one of the source files are newer than the result, it will invoke the command to rebuild the result.

Version control systems such as subversion, CVS, and RCS, are another good examples of dependency management tools. You can save the history of changes on a file, a directory, or a set of directories. You can make a software package by checking out a set of tagged files; you can even make multiple branches of a code.

FreeBSD operating system has its own dependency management system of externally-contributed programs called ports and packages. A port means a set of rules, configuration files, and source codes necessary to build a program. A package means the derivative of the port, built by another computer. Many essential parts of FreeBSD subsystems, including the X Window and Perl programming language, are installed as packages, because they are not considered as the core parts of FreeBSD.

I'm always pleasantly surprised when a very complicated port, such as Japanese version of LaTeX, a typesetting and documentation set of programs, can be built without major glitches, including the automatic installation of depending programs, such as Ghostscript. For most of the ports, FreeBSD volunteers are always doing the outstanding jobs.

I feel very much annoyed, however, when I have to untangle the web of dependencies when the installation/building of a software toolset from the scratch. Unfortunately, Xorg 7.3, a free implementation set of X Window programs, fonts, and tools, was not able to build in my environment. I had to copy three prebuilt sets of files: include files of C/C++, shared/static libraries, and the very basic fonts, to finish building the necessary programs.

In my case of Xorg kitbuilding, some very old fonts existing since late 1980s were not successfully compiled and converted into various ISO8859 part codesets. And I could not build the font handling libraries. This glitch killed the whole automatic compilation task, which was supposed to untangle the enormous list of dependencies. I also found that the target directory name was merged to /usr/local from the traditional /usr/X11R6, and I wasn't sure whether if I moved the old files and subdirectories under the old directory safely to the new directory.

Untangling the web of dependency is a very hard task, since the parameters you need to examine are scattering around all the directories, in all the ports, and you often also have to set an environment variable to do something extraordinary. While I can guess what I should do because I've done a lot of this kind of tasks, I never want to do this at any time because scanning your memory and poking around the files are very much mentally painful. In a recent hostile computer environment, vulnerabilities can easily sneak into such a complicated and unformalized tasks.

How do you manage your computers? Do you care about the dependencies of the tools, programs, configuration files, and other objects?