Introduction The New Locker-Organization Conventions ... Organization of Machine-Dependent Files ... ... The arch Directory ... ... Platform-Specific Directories ... ... Data-Specific Directories ... ... Symbolic Links for Binary Compatability ... ... Example ... Organization of Machine-Independent Files ... Use of Convenience Symlinks Addressing Backwards Compatibility ... Limitations of the Old bindir Scheme ... Backwards Compatability in the Software ... Backwards Compatability Symlinks Programs and Variables Supporting the New File-Organization Conventions ... The add Command (revised) ... The athdir Command (new) ... The bindir Variable (revised) ... The ATHENA_SYS Variable (new)
There are many possible ways to organize and deliver binaries for multiple architectures under a single filesystem. A good systematic file-organization scheme must provide a standard way to separate machine-dependent binaries into different directories for ease of use and maintenance. Generality, backwards compatibility, and neatness must also play a role.
For some time, Athena has had a set of conventions in place in support of locker file organization (the so-called "bindir" conventions). But various difficulties were involved in this scheme, and these conventions have been found to be intractable.
The Athena 7.7 Patch Release (January 1995) introduces a new and more flexible set of conventions (and related tools) for organizing files in a locker. These conventions are now the standard locker organization scheme supported on Athena.
The new conventions differ from previous conventions supported on Athena (but, for backward compatability, some of the old conventions will still work for previously-supported platforms).
If you maintain a development locker on Athena, you are encouraged to learn and follow the new conventions -- even if you rely on the old conventions for existing platforms, you will need to know the new conventions for any future ports.
This document outlines the new conventions, plus the new tools available in the Athena 7.7 Patch release in support of these conventions (e.g., the revised add alias, and the new athdir command).
The arch Directory. In order to avoid any sort of clutter in the top level directory of a locker, all machine-dependent directories are placed under a new directory (called arch) at the lop-level of the locker:
/mit/locker/arch/
Platform-Specific Directories. Under arch is a separate directory for each supported platform:
/mit/locker/arch/platform/
The name of each directory (platform) is defined by the AFS "@sys" value for that platform. For example, pmax_ul4 would be the appropriate name for a directory of files for DECstation 5000's running Ultrix 4.1; sun4m_53 would work for files for Sun SPARCclassics running SunOS 5.3, and so forth.
Note that, unlike the previous Athena convention, the directory name in the new convention does not include the letters "bin" at the end of it.
Data-Specific Directories. Under each platform-specific directory are subdirectories holding specific types of machine-dependent data:
/mit/locker/arch/platform/data-directory/
There are some standard data-directory names, which reflect the type of data the directory holds. Some commonly-used names include:
bin (for binaries)
lib (for libraries)
man (for machine-dependent manual pages)
etc
build
Symbolic Links for Binary Compatability. Finally, for some "platforms", binaries compiled on one platform run fine on another similar platform, even though the AFS "@sys" values are distinct. Because of this "binary compatibility" across multiple AFS "@sys" values, you may want to include symbolic links for some platform-specific directories rather than create different (but basically redundant) directories for every possible platform.
For example, if you already have a directory for sun4m_53 files, you need not establish whole new directories for sun4c_51 or sun4c_53 files, because users on the latter platforms can use binaries and other files compiled for the sun4m_53 platform. Instead of creating whole new directories, you can simply create two symbolic links at the same directory level as other platform-specific directories and point to the existing sun4m_53 directory:
athena% ln -s sun4m_53 sun4c_51 athena% ln -s sun4m_53 sun4c_53
(Of course, this approach only applies for platforms that are truly compatible.)
Example. A locker containing only libraries and binaries, reflecting the current Athena platforms, and organized according to the conventions outlined above might look schematically as follows:
/mit/locker/arch/
pmax_ul4/
bin/
lib/
rs_aix32/
bin/
lib/
sgi_52/
bin/
lib/
sun4m_53/
bin/
lib/
sun4c_51 -> sun4m_53 [symbolic link]
sun4c_53 -> sun4m_53 [symbolic link]
Many files, such as manual pages or data files, may be the same for all machine architectures. Currently, the only defined convention for such files applies only to manual pages. That convention is simply that, in the top level directory of a locker there is a directory called "man" that follows the conventions for manual directories followed by most flavors of Unix:
/mit/locker/man/
(In the new conventions, as noted above, if you have machine-specific manual pages, you can place them in a data subdirectory of the appropriate platform directory. The revised add command supports this possibility when modifying the MANPATH, in that it checks first for the existence of the machine-specific directory:
/mit/locker/arch/platform/man
Only if this fails does add try the machine-independent directory /mit/locker/man, as a fallback.)
Typically, users employ the add command to set things up so that they never have to know a full pathname to access platform-appropriate binaries from a given locker. (For example, a user can enter "add sipb; Mosaic &" to run the Mosaic program without ever having to know its full pathname within the sipb locker.)
Nevertheless, there are times when a user will bypass the convenience provided by add and type the full pathname anyway (e.g., the user does not add the locker in the first place and must use a full AFS pathname to access the binary; or the user has added the locker but has been given an instruction for running the binary that lists its explicit pathname, starting /mit/locker/..)
For the convenience of such users (and for such uses), it is very helpful for the locker maintainer to include a symbolic link at the top level of a development locker that can take users directly to the data-directory appropriate for their platform (this can be especially useful for binary or library directories, for example).
The general form for creating such a convenience symlink is as follows (this assumes you are at the top level of the locker, e.g., at /mit/locker):
athena% ln -s arch/@sys/data-directory data-directory
For example, the most common use will be to create a link for binary files:
athena% attach locker athena% cd /mit/locker athena% ln -s arch/@sys/bin bin
This creates a directory at the top level of the locker with the name of the data-directory type you identified:
/mit/locker/bin -> arch/@sys/bin
Note that the string @sys in the ln command is the literal string "@sys" and not the value of @sys. (The string "@sys" should never be used literally, except in convenience symlinks as described here. To obtain the value that @sys might return, use the environment variable value string "$ATHENA_SYS" instead.)
The new file-naming conventions cannot simply replace the pre-existing scheme entirely: too many services rely on the old way of doing things. Nevertheless, developers are encouraged to move to the new scheme and establish structures for backwards compatability, rather than merely continue in the old scheme. Apart from the fact that new ports will only be able to use the new scheme, there are also compelling reasons the new scheme should be adopted even for existing platforms.
This section describes limitations of the old scheme, and suggests how to incorporate provisions for backwards compatability while adopting the new framework.
The file layout conventions on Athena before the Athena 7.7 Patch release (i.e., the old bindir scheme) included standards for binary and manual directories only, and no suggestions for hierarchies to avoid clutter or other general hints. This lack of generality alone might be sufficient reason for an overhaul, but a more specific issue loomed: how to support multiple configurations on the same machine type.
The main convention of the old scheme was that binary directories should be placed at the top level of the locker, with names derived from the output of the machtype command, as follows:
/mit/locker/`machtype`bin/
In practice, this produced the following directories:
/mit/locker/
decmipsbin/ [Digital DECstation running Ultrix 4.2A]
rsaixbin/ [IBM RS/6000 running AIX 3.2.3]
sun4bin/ [Sun SPARCclassic running Solaris 2.3]
vaxbin/ [Digital VAXstation - no longer in use]
rtbin/ [IBM RT workstation - no longer in use]
This scheme proved limited. Most importantly, the bindir naming procedure does not allow for any flexibility to specify individual platform "flavors" for the same machine type. (A platform type is identified by a specification of both the machine type and a particular operating system. The machtype command, and the shell bindir variable based on it, only identify the machine type, not the full platform type.)
For example, the decmipsbin directories are assumed to cover any DECstation, without regard to operating system -- but binaries compiled under different operating systems are often incompatible, even if compiled for the same machine type. This naming scheme thus required that all DECstations had to run the same operating system (currently Ultrix 4.2A) or users would not be able to share binaries.
In other words, there was no provision within the bindir naming scheme for developers to create binaries that would run on the same machine type under a different operating system configuration from what was currently accepted as "standard". This forced "standardization" could be undertaken in the public clusters, where Athena controlled the machines, but it kept innovation at bay by forcing forward-looking developers into non-standard practices.
Developers could always work around the limitations, but in doing so they broke the conventions that Athena supported, and had to reinvent the wheel repeatedly. The new naming conventions introduced in the Athena 7.7 Patch release have been developed to be broad enough to allow "nonstandard" development to proceed in a way fully consistent with "standard" practices.
One obvious solution to the limitations described above would be to change the value that machtype returns, to reflect platform types rather than machine types. For future platforms, that might make sense, but for existing platforms, it could not work: the machtype command, and the bindir variable based on it, are relied upon too heavily in their current form. Many shell scripts depend on the old output of machtype, and almost every development locker follows the old bindir naming scheme.
It would be impractical and burdensome to require everyone to change everything to conform to the new scheme. Even if every developer could be forced to spend the time and energy to revise their binary directories to conform to the new arch/platform/bin scheme, there are many old lockers without designated maintainers that use the old `machtype`bin convention and it would be unrealistic (as well as a misuse of resources) to require anyone to update them.
But even with all these obstacles surmounted, there would still be no practical way to update the entire world simultaneously: enacting a new scheme across the board would have effectively meant shutting down huge chunks of Athena for months (i.e., each piece would be basically unusable until it was converted to the new scheme).
Rather than require any such a ghastly scenario to occur, the Athena 7.7 Patch release provides provisions for backwards compatability such that anything conforming to the old scheme will still function in the new release. In adding directories to a search path, for example, the revised add command will look first for new-style directories -- but will still try old-style directories if no new-style ones are found. Similarly, the athdir command (rather than machtype) is now used to define the bindir variable, upon which directory names are based -- but machtype is still available, and athdir (and hence $bindir) gives old-style values when appropriate. (These commands are described elsewhere.)
To support backwards compatability in lockers that are being updated to follow the new naming conventions, developers are encouraged to put in symbolic links that point from the old-style binary directory names to the appropriate new-style directores.
For binary directories (the most likely case), this takes the form of entering a command of the following form at the top level of the locker:
athena% ln -s arch/platform/bin old-style-bindir-name
For example, if you have already created a platform directory called "pmax_ul4" (for DECstation/Ultix 4.2A development work) under your arch directory, and put appropriate data-directories (such as bin) under it, you can provide a compatability symlink at the top level of the locker as follows:
athena% attach locker athena% cd /mit/locker athena% ln -s arch/pmax_ul4/bin decmipsbin
This creates a link at the top level of the locker as follows:
/mit/locker/decmipsbin -> arch/pmax_ul4/bin
(Note that, if at some later point, the platform directory you want to point users to changes, you can simply remake the link to point to the new directory.)
You should set up symbolic links for any of the old-style directory names for which you have established new-style binary directories:
decmipsbin [Digital DECstation running Ultrix 4.2A] rsaixbin [IBM RS/6000 running AIX 3.2.3] sun4bin [Sun SPARCclassic running Solaris 2.3] vaxbin [Digital VAXstation] rtbin [IBM RT workstation]
In practice, you can probably ignore vaxbin and rtbin, as these machines are not supported on Athena anyway. Also, new platforms, such as SGI, need not provide any symlinks, as they do not have an old bindir value to worry about.
Note that these compatability links are a service not for standard users but for users who somehow employ the old naming scheme (e.g., they are using a shell script that uses the output of machtype directly, or in which the old-style names were hard-coded) -- users who attach a locker using the add command will not need these links, as add will find the new-style directory correctly and put that right in your search path.
There are four elements in the Athena 7.7 Patch release that are available to help you work well within the new locker file-naming conventions:
Each of these system tools is described below.
The add command (see add(1) for details on use), for binary directories, initially checks for the existence of the new style binary directory. If it finds it, it adds that to the user's search path. If not, it falls back to the old machtype based convention. Similiarly, in order to more easily support machine-dependent manual pages, it checks for an arch/platform/man directory before falling back to the traditional man directory.
The athdir command is in some ways a generalization of the add command. The most important functionality it provides is as a replacement for the use of the /mit/locker/`machtype`bin string. `athdir /mit/locker` should now be used instead, and will work correctly whether old or new directory conventions are in use in that locker. athdir is also potentially useful for finding library or include directories from inside of makefiles. See athdir(1) for details.
athdir is intended to entirely replace the old `machtype`bin method. For example:
old style: /mit/locker/`machtype`bin new style: `athdir /mit/locker` old style: /afs/athena/astaff/project/myproj/`machtype`bin new style: `athdir /afs/athena/astaff/project/myproj` old style: set path = ( $path /mit/locker/`machtype`bin ) new style: set path = ( $path `athdir /mit/locker/` ) old style: athena% /afs/athena/astaff/project/logos/`machtype`bin/mycommand new style: athena% `athdir /afs/athena/astaff/project/logos`/mycommand
The point is not that athdir is much shorter (it isn't). The point is that athdir intelligently looks at the specified directory and searches for new-style subdirectories, then for old-style ones. The old machtype method involves no intelligence at all but merely substitutes a string and assumes you know what you are talking about (i.e., assumes that the old conventions are followed).
The bindir variable, on older platforms, is set to the value `machtype`bin. On newer platforms, it is set to arch/platform/bin. (Note that it is defines as arch/@sys/bin, but is not set to this literal string but to arch/(the value of @sys)/bin; the literal string @sys should never be used except in convenience symlinks.)
The ATHENA_SYS environment variable is used lieu of the AFS string @sys. (ATHENA_SYS is derived in the global cshrc from the output of machtype -S.) In all cases, it should be equal to what @sys resolves to for any particular platform. So in shell scripts, makefiles, etc., one should never attempt to find one's libraries with a string such as "arch/@sys/lib" but rather "arch/$ATHENA_SYS/lib." It is usually preferable to use athdir, however.
Avoidance of the literal string "@sys" is done in order to keep the locker conventions filesystem-independent. If for some reason a locker is copied to (or is maintained in) NFS space, it will still work correctly. If AFS is translated through some other medium which does not magically know how to cope with the "@sys" string properly, it will also still work correctly. If Athena migrates to another kind of filesystem that does not support the use of "@sys", nothing will need to be done to update lockers and everything will still work.