Packaging Software For Nekoware
This is a draft. Do not follow it for advice
Package Format - tardist
IRIX packages are called tardists and, as the name implies, are essentially tar(1)'ed sets of distribution files. These files describe the location and permissions of all files related to a particular piece of software. These tardists can then be distributed and installed on similarly-configured SGI machines.
Tools Required
Compilers
For IRIX, the system compiler is known in 6.5 and 6.2 as MIPSPro, and earlier versions as IRIS Development Option. For SGI software, there are two viable options:
MIPSPro (c99/cc/CC): Caveats are that thread local storage is not supported, C11 and C++11 and up features are mostly absent, IRIX's CC (c++) compiler has some important limitations and that cross-compilation is not possible. GCC and MIPSPro are C-ABI compatible, but not C++. MIPSPro is fast at compilation, uses less memory, and can potentially improve performance
GCC: Caveats are that applications that use native C++ libs (libfam, libviewkit, libinst) etc. cannot be built under GCC. They won't link correctly. Link-time optimization and thread local storage do not work correctly on IRIX, SGUG RSE disables the latter and the former will break binaries. GCC has trouble linking under RSE to pthread and GL libs, and pthread libraries otherwise. GNU binutils has limitations, and most GCC will only build 32-bit apps correctly.
Platform
In order to build software for Nekoware, it's recommended to run IRIX 6.5.21 or newer, have at least 512M RAM, and an R10000 at 175MHz or an R5200 at 300MHz or greater. Nekoware does not support cross compilation, but these machines are fast enough to not require it.
Software
MIPSPro requires a license. Flexlm licenses exist for it, but it can also be patched to remove the check. GNU make 3.81 or later is also recommended, as IRIX pmake, smake and make are all unsuitable for modern software.
Build Environment
Be sure to set some global variables for development on IRIX. The following is a set of sensible starting point. setenv(1) is used on tcsh, export for KSH and GNU Bash:
setenv CC c99 setenv CXX CC setenv CFLAGS '-O2 -mips4 -TARG:proc=r10000' setenv CXXFLAGS '-O2 -mips4 -TARG:proc=r10000' setenv CPPFLAGS '-I/usr/nekoware/include -I/usr/include' setenv LDFLAGS '-L/usr/nekoware/lib32 -L/usr/lib32 -Wl,-rpath -Wl,/usr/nekoware/lib32' setenv PKG_CONFIG '/usr/nekoware/bin/pkgconf' setenv PKG_CONFIG_PATH '/usr/nekoware/lib32/pkgconfig' setenv ACLOCAL_PATH '/usr/nekoware/share/aclocal'
export CC=c99
export CXX=CC
export CFLAGS='-O2 -mips4 -TARG:proc=r10000'
export CXXFLAGS='-O2 -mips4 -TARG:proc=r10000'
export CPPFLAGS='-I/usr/nekoware/include -I/usr/include'
export LDFLAGS='-L/usr/nekoware/lib32 -L/usr/lib32 -Wl,-rpath -Wl,/usr/nekoware/lib32'
export PKG_CONFIG='/usr/nekoware/bin/pkgconf'
export PKG_CONFIG_PATH='/usr/nekoware/lib32/pkgconfig'
export ACLOCAL_PATH='/usr/nekoware/share/aclocal'
All Nekoware is installed relative to /usr/nekoware and should be built and linked against the libraries there.
Furthermore, there are some MIPSpro options that should be set:
Defining CC and CXX specifies the MIPSpro compiler suite. The CFLAGS option indicates building for MIPS IV, that being any MIPS processor R5000 or newer (see above).
Also present are additional optimizations being done for the R10000 platform (more specifically the SGI Origin 200 IP27), which is a common processor among the SGI O2, SGI Octane, and SGI Origin platforms. This build will work on R5000, but will see additional optimization on R10000 or better. If the software is optimization-friendly, e.g. mplayer or blender - it is possible to provide different builds, optimized for specific processors and have inst or swmgr automatically select the best build during installation.
Alternatively, the -TARG option could be disgarded entirely to build without any platform-specific optimizations whatsoever. Many Nekoware packages use only -03 -mips4.
Note: A great deal of open source software expects the compiler suite to halt processing if an "#error" preprocessor directive is encountered. GCC does this, while MIPSpro does not by default. To get this behavior with MIPSpro you can add "-diag_error 1035" to the CFLAGS definition above.
Creating a Sandbox
Typically, a location under the home directory would be used for holding source tarballs, build directories and other working files. Under this directory, creating two copies of the build directory will aid the patch creation process later on. Suppose one were building fooware, version 1.2.3, within this working directory:
(prompt) gunzip -c fooware-1.2.3.tar.gz | tar xf - (prompt) cp -r fooware-1.2.3 fooware-1.2.3-PATCHED
There are now two identical versions of the source directory. By maintaining the original source and working within the "-PATCHED" directory, creating a patch for the package will be much more simple.
Building the Software
Typically, after a sandbox has been created, the software should be configured from within the "-PATCHED" directory (following the directions above). Remember, all Nekoware packages are installed relative to /usr/nekoware, so the --prefix option should be passed to configure. libdir must be done as well, as /lib is for o32 software and /lib64 for 64-bit
% ./configure --prefix=/usr/nekoware --libdir=/usr/lib32
Be sure to specify any other configure arguments necessary to build the software. It may be necessary to write down any dependencies configure checks to create accurate rules later.
After the software has been successfully configured, compile the source as usual. As an example, something like this might be used to compile the software on a dual-processor machine:
% gmake -j3
If the software successfully compiled, it may be possible to simply run the software from within the working directory, without installing anything. If so, try that now and verify that all desired functionality is present. If not, the software should now be installed using something similar to the following:
% sudo -s % gmake install % exit
After installation, test the software as above to verify the successful build.
Installing the Software
To package, after confirming build and install correctly, reconfigure the software to install to a staging directory:
./configure --prefix=/usr/neko_staging --libdir=/usr/neko_staging/lib32
This will allow tracing by either swpkg(1) or genidb(1) to be useful.
Package Preparation
Directory Structure
Nekoware packages use a consistent directory structure for patches, sources, release notes and distribution files. Create this structure under the /tmp/build directory (created above), now:
% mkdir /usr/neko_staging/patches % mkdir /usr/neko_staging/src % mkdir /usr/neko_staging/relnotes % mkdir /usr/neko_staging/dist
Copy the original source distribution and the patch file to their respective directory:
% cp neko_fooware-1.2.3_irix.patch /usr/neko_staging/patches % cp fooware-1.2.3.tar.gz /usr/neko_staging/src
Release Notes
This section is under construction
Creating a Package
Now that the temporary directory contains the binary software, the source code, the patch and a release notes file, create a distribution directory and run Software Packager:
% mkdir /usr/neko_dist && cd /usr/neko_dist % swpkg
The Software Packager interface is designed to be intuitive. The other option will discussed soon.
Creating a Product Hierarchy using swpkg
On the first worksheet, a tree is displayed showing a default arrangement of subsystems within a package. This may, or may not, suit the software's layout, but it is provided as a starting point for creating a new package.
The first step is to give the package a name on the root node. Something like neko_fooware is recommended. Add a description to this node on the right using other packages in the Software Manager as a guide. Be sure to save any changes or they will be lost.
Create the other nodes according to the typical Nekoware hierarchy, as follows:
neko_fooware.eoe (binaries and files for execution only) neko_fooware.include (headers) neko_fooware.lib (shared libraries) neko_fooware.man.manpages (manual pages) neko_fooware.opt.src (the original source tarball, this is optional) neko_fooware.opt.patches (patches against the original source) neko_fooware.opt.relnotes (release notes for this package) neko_fooware.opt.dist (distribution files, generated by swpkg)
Not all of the tree may be required by the software. Use common sense and other Nekoware package as a guide.
Versioning
Every package will have two version numbers that should never be confused: the package version and the software version.
The package version is assigned to all children of the root node on the first worksheet of the Software Packager. This number is incremented for each build of this package independently of the software version contained within the package.
For Nekoware, start a package version at 100. This is because historical nekoware may conflict, which can break things.
Remember: a package's version is independent of the version of the software it contains.
Also, all package version numbers must be equal within a package. If, for example, the execution-only subsystem has changed in a new package, all other subsystem's must have the version numbers incremented, as well. This holds true even if no other subsystems have changed between package versions.
Dependencies
This is, perhaps, the most important portion of the packaging process and is the reason for recording the software's dependencies during the build stage above. Find the package version of each dependency using the versions(1) command. For example:
(prompt) versions -n neko_foo (prompt) versions -n neko_bar (prompt) versions -n neko_baz
Each versions(1) command will output several lines, one for each subsystem in the package. Write down the version numbers for each subsystem the software depends on. Be sure to be specific: which subsystem does the software depend on? Just the shared libraries, or the execution environment? Both?
Now, in Software Packager, specify the dependencies for each subsystem in the Rules section. The rules will look somewhat like the following:
replaces self prereq ( neko_fooware.lib 100 maxint neko_foo.lib 100 maxint neko_bar.eoe 100 maxint neko_baz.include 100 maxint )
Final Steps
There is also an "install by default" checkbox that should be checked for any subsystems that require it. Typically, the sw.eoe, sw.lib, sw.hdr and man.manpages will have this option checked.
Before continuing, be sure to save the .spec and .idb files that Software Packager creates to /usr/neko_staging/dist. This will make them available to the next step.
Headless Alternative to swpkg, genidb
Running an SGI headless? No problem! Raion and EasyMode developed genidb(1). It skips many of these steps.
Referencing the above section, start with a spec file. Here's one for fooware, using all of the above steps:
product neko_fooware
id "Your description goes here"
image eoe
id "fooware binaries"
version 100
order 9999
subsys base default
id "fooware binaries"
prereq (
neko_fooware.lib 100 maxint
neko_foo.lib 100 maxint
neko_bar.eoe 100 maxint
neko_baz.include 100 maxint
)
replaces self
exp neko_fooware.eoe.base
endsubsys
endimage
image lib
id "fooware libs"
version 100
order 9999
subsys base default
id "fooware libs"
prereq (
neko_fooware.lib 100 maxint
neko_foo.lib 100 maxint
neko_bar.eoe 100 maxint
neko_baz.include 100 maxint
)
replaces self
exp neko_fooware.lib.base
endsubsys
endimage
image include
id "fooware headers"
version 100
order 9999
subsys base default
id "fooware headers"
replaces self
exp neko_fooware.includes.base
endsubsys
endimage
image man
id "fooware manpages"
version 100
order 9999
subsys manpages default
id "fooware manpages"
replaces self
exp neko_fooware.man.manpages
endsubsys
endimage
image opt
id "optional"
version 1
order 9999
subsys patch
id "patch files"
replaces self
exp neko_fooware.opt.patch
endsubsys
subsys relnotes
id "release notes"
replaces self
exp neko_fooware.opt.relnotes
endsubsys
subsys dist
id "distribution files"
replaces self
exp neko_fooware.opt.dist
endsubsys
endimage
endproduct
If this looks complicated, just go section by section. If problems still exist, ask on the IRIXNet forums.
Once that's established, copy it to /usr/neko_staging/dist, add all other files including patches, relnotes etc.
Then:
genidb -p /usr/neko_staging -s /usr/neko_staging/dist/neko_fooware.spec -v
The output should say:
(-s) Input Spec: /usr/neko_staging/dist/neko_fooware.spec (-p) Path: /usr/neko_staging Working through specified path: /usr/neko_staging IDB output file: /usr/neko_staging/dist/neko_fooware.spec Parsed Spec File Scanning files in: /usr/neko_staging Completed with 25 entries and 0 rejections.
Or something to this effect. It'll tag files as best it can. Manually review the file, as sometimes subsystems get messed around with. Skip the next section, Tagging.
Building the package
Next, the idb must be sorted per the gendist manual:
sort +4u -6 < neko_fooware.idb > /usr/nekoware/dist/neko_fooware.idb
This will copy it to /usr/nekoware/dist. cp the spec file, patches, relnotes and src if present.
Next, fix the paths in idb: perl -pi -e 's/_staging/ware/g' neko_fooware.idb
Running Gendist
Finally, try gendist:
% cd /usr/neko_dist % gendist -sbase / -spec /usr/nekoware/dist/neko_fooware.spec -idb /usr/nekoware/dist/neko_fooware.idb -distdir .
It'll give any errors such as empty subsystems and such.
Skip to Quickstarting for relevant info.
Tagging
Every file that the software creates and installs must be tagged with a particular subsystem within the package. Above, we created a clean application tree in /tmp/build for this very purpose. Had we not, each file that the software installed would need to be found in /usr/nekoware - a very tedious process!
Import the entire /tmp/build directory and tag each file. For example, files installed in lib/ would be tagged with sw.lib, those in bin/ and share/ should be in sw.eoe and include/ is tagged with sw.hdr.
All directory entries should be removed from the distribution entirely. Leaving them is considered sloppy and clutters the package.
Set all files as owned by the root user and sys group, save the .spec and .idb files once more and exit Software Packager.
Fixing Paths
The current package now contains all files installed by the software with the correct permissions and tagged with the correct subsystem. However, due to using the sandbox, /tmp/build, all of the paths are incorrect. Thankfully, correcting them is an easy proposition:
(prompt) cat /tmp/build/usr/nekoware/dist/neko_fooware.idb | \ sed 's/\/tmp\/build//g' > tmp.idb (prompt) mv tmp.idb /tmp/build/usr/nekoware/dist/neko_fooware.idb
Alternatively, the .idb file can be opened using a text editor and the necessary replacements done using the editor's find-and-replace functionality.
Re-launch the Software Packager and open the .spec and .idb files. Verify that the paths are correct.
Quickstarting
Note that the myth that any files should be marked with norqs is false. gendist automatically marks all ELF files with needrqs, unless the file has been tagged with norqs. Any other files, such as text files, graphics etc. will not trigger an rqsall run after installation, nor will they even be read by rqs.
During most package installations using inst(1M) or swmgr(1M), certain files need to be "Re-quickstarted" using rqsall(1). When creating a new package, certain files may be marked as not needing this process using the norqs option.
Toolchest Entries
Testing and Building
Save the .spec and .idb files once more and move to the next worksheet. This worksheet provides the final steps in Software Packager. Click the Test button to test the package build process. It is likely that the process will complain about files missing from /usr/nekoware. Move to a shell and copy these files from /tmp/build into their respective directories in /usr/nekoware and run the test again.
It's worth it even if built with genidb to check, as genidb doesn't support post install commands or other features.
Once the test routine passes without complaint, click the Build button and exit Software Packager. The distribution files for the package should now be in /usr/neko_dist . Change to that directory and archive these files:
% cd /usr/neko_dist % tar -cf neko_fooware-1.2.3.tardist *
The next, and final, step is crucial. Change to the working directory, "-PATCHED" above, and uninstall the software:
% su % gmake uninstall % rm /usr/nekoware/dist/neko_fooware.* % rm /usr/nekoware/src/fooware-1.2.3.tar.gz % rm /usr/nekoware/relnotes/neko_fooware.txt % rm /usr/nekoware/patches/neko_fooware-1.2.3_irix.patch
Now, install the package as though it were a normal Nekoware tardist:
% inst -f /usr/dist/neko_fooware-1.2.3.tardist
Verify that the software installs correctly and runs as before, with no errors. If it installs and runs correctly, contact Raion on the forums for instructions.
Postprocessing
Certain installed files may require some kind of post-processing to integrate or register them with e.g. a service or database that is part of another package. The steps necessary will either be documented in the installation instructions or silently performed as part of make install. In the latter case, one will have to carefully inspect the output and determine if any special action needs to be taken. Examples of files that require post-processing are Icons or texinfo pages. While the tools to install icons are likely already installed on the system, the install-info tool would warrant an additional prerequisite in the spec file for the subsystem containing the file. When you have determined the commands required, add an appropriate exitop to the idb file. For a texinfo file, the exitop may look as follows:
f 0644 root sys usr/nekoware/info/foo.info usr/nekoware/info/foo.info neko_foo.man.info exitop('if test -x $rbase/usr/nekoware/bin/install-info; then chroot $rbase /sbin/sh -c "/usr/nekoware/bin/install-info --info-dir=/usr/nekoware/info /usr/nekoware/info/foo.info;" ; fi')
For Perl modules, perllocal.pod should not be installed, but amended as follows:
f 0555 root sys usr/nekoware/lib/pidgin/perl/auto/Pidgin/Pidgin.so install/usr/nekoware/lib/pidgin/perl/auto/Pidgin/Pidgin.so neko_pidgin.sw.lib exitop('$rbase/usr/nekoware/bin/perl -MExtUtils::Command::MM -e "perllocal_install" -- "Module" "Pidgin" "installed into" "/usr/nekoware/lib/perl5" LINKTYPE "dynamic" VERSION "0.01" EXE_FILES "" >> $rbase/usr/src/pidgin/install/usr/nekoware/lib/perl5/irix-n32/perllocal.pod')
Standardization of nekoware
The package release notes should be installed by default. Also please put man pages in /usr/nekoware/share/man rather than /usr/nekoware/man, etc. It may be necessary to have to override some default locations (e.g. "--mandir=$PREFIX/share/man" instead of "--mandir=$PREFIX/man").
Quality Standards
Updated later
