UnitedLinux media description

Motivation

UnitedLinux defines a set of base components for a Linux server distribution. These come on a single and clearly defined media, which is identical for every UnitedLinux vendor.
Applications are installed separately and the install media contents are set up by the vendor.
In order to make application installation and esp. media (content) descriptions commonly defined, this document proposes a such a description format.

Requirements

  1. Multiple products per media
    A media may contain several products, i.e. a combined patch media.
  2. Multiple architectures per product
    A media may contain products for multiple architectures.
  3. Bi-Arch support
    An architecture may be bi-arch, where mixing of 'compatible' architectures is allowed.
  4. Multiple languages per product
    A media may contain products for multiple languages.
  5. Multiple media
    a) The content may be splitted onto multiple media.
    b) It must be able to merge splitted media into a single directory.
    c) Media can be double-sided (e.g. DVD-ROM).
  6. A media must be easily recognizable by the system
    This is needed when re-inserting a cd-rom media for further package installation. The system must verify that the correct media is inserted and can be used for package installation.
  7. product name
  8. product version
  9. base distribution
    specify the distribution the product is based on.
  10. base distribution version
  11. product vendor
  12. user-readable content description
    This is needed to present media information to the user prior to installing packages. The user must be able to decide on this information to proceed or cancel the installation process.
  13. package path
    toplevel path to packages (i.e "suse/" on SuSE media, "unitedlinux/" on UL media, etc.)
  14. default language
    default language if no language is determinable.
  15. inital server for online update
  16. no wildcard search for files
    doing a wildcard search is cheap on a local media but almost impossible via HTTP. So it should be avoided if possible.

Implementation

The media and product description is splitted to a set of files, separating the media preparation data from the contents (product) and making it possible to combine several products on a single media.
At toplevel of a media, a directory
	/media.1/

should exists.

If it does not exist, the presence of the correct media can not be verified. This may be convenient for an on-line media which just provides a set of packages.
In this case, no verfication takes place.
The numeric extension depends on the media number in case of splitted medias. The first media has .1, the second .2, and so on.

This allows merging all medias of a splitted set into a single directory. YaST always knows which media to expect and can thereby directly access the correct media directory.

Files inside media.N/

Inside the media.N/ directory the files media and products must exist.
Optionally, a file named patches may exist to denote a media with patches (e.g. for YaST online update).

The media file contains the media identifier needed to determine correctness when using removable media.
In case of splitted media, it must be present on every media of the set.

The products file contains directory specifications for each product.
In case of splitted media, it must be present of the first media of the set.

The patches file contains a single directory specification for the root of the patches directory tree and an optional list of exclusive products the patches apply to.
In case of splitted media, it must be present of the first media of the set.

The /media.N/media file

This should contain a key to quickly identify correctness of the inserted media. It is not shown to the user but should contain human-readable data for debugging purposes.

Proposal: Encode Vendor and Timestamp information in a file

File:media
Location /media.N/ directory on media
Contenttwo or more lines of ASCII as follows
<vendor>
<YYYYMMDDHHMMSS>
[<media count>]
[<media flags>]
[<media names>]

The identification data is stored inside the file on purpose. It could also be encoded in the file name (older SuSE media did it). But this requires a directory scan for file access and should be avoided.

By storing the vendor name in the file, the timestamp can be kept vendor specific. (It's the name of the vendor who created the media which might be different from a product vendor! It's just used as an additional key for media identification.)

All media of a media-set must provide identical vendor and timestamp information in the media file.

The /media.1/media file

The /media.1/media file may contain additional information.
The remaining media files of a media-set may repeat this information, but it is not evaluated again.

The <media count> is needed if the installation data is split up on multiple (identical) medias.
The media count is a numerical value (all characters are digits), and defaults to 1.

<media flags> may further describe the media.
In order to distinguish the flags from the count line, the flags values start with a letter.

Currently defined flags:
flagmeaning
doublesidedmedia is double sided, YaST will ask for 'front side' for odd-numbered media and 'back side' for even-numbered media.
The default is single-sided media.

<media names> may define alternate strings to use when asking to insert a certain media.
They are defined as <key><whitespace><value> pairs, separated by \n.

keyvalue
MEDIA<N> UTF-8 encoded string to use when asking for media <N>.
MEDIA<N>.<lang> UTF-8 encoded translation of MEDIA<N> may be defined by appending the ISO <language> or <language>_<country> code.

Example

linux# mount /dev/cdrom /mnt
linux# ls /mnt/media.1
media
products
linux# cat /mnt/media.1/media
SuSE Linux AG
20020921153042
3
MEDIA1 SUSE Linux 9.1 DVD 1
MEDIA2 SUSE Linux 9.1 DVD 2
MEDIA3 SUSE Linux 9.1 DVD 3
linux#

The products file

This file contains the directory name for each product contained on the media.

If this file is not present, a single product on the media's root directory is assumed.

It is an ASCII file with one line per product.

Each line starts with the directory name (relative to the media's root dir !), followed by whitespace (space or tab) followed by the product name and version.
A leading slash in the directory name is only needed to specify the root directory of the media.

File:products
Location/media.1/ directory
Contentone or more lines of ASCII as follows
<directory> <whitespace> <product> <whitespace> <version>

Example

linux# mount /dev/cdrom /mnt
linux# cat /mnt/media.1/products
/	SuSE Linux Personal 8.1-0
sles-8	SuSE Linux Enterprise Server 8-0
openlinux	Caldera OpenLinux 4.0-0
unitedlinux-1.0	UnitedLinux 1.0-1
linux# ls -a /mnt
...
media.1/
...
sles-8/
openlinux/
unitedlinux-1.0/
...
In the above example, "SuSE Linux Personal" uses the toplevel directory, whereas the other products have their own sub-directories.

The content file

This file resides inside the product directory as specified by the products file on the media. It contains all product-specfic data to describe and identify the contents of the product.
If there was no products file, the product directory defaults to the root directory on the media.


The content file might be missing if the media does not contain a product (but just a set of packages).
In this case, the user has to 'blindly accept' the media.


Proposal:
File:content
Locationproduct-dir as specified in /media.1/products file
ContentUTF-8 encoded <key><whitespace><value> pairs, separated by \n
For allowed keys see next table.

Mandatory key strings
priokeyallowed value
MPRODUCTproduct name
MVERSIONproduct version and release as in RPM
<major>.<minor>-<release>
MDISTPRODUCTdistribution id (vendor specific)
MDISTVERSIONdistribution version (vendor specific)
OBASEPRODUCTname of base product if different from PRODUCT
OBASEVERSIONversion of BASEPRODUCT, mandatory if BASEPRODUCT is defined
<major>.<minor>-<release>
MVENDORvendor name (free form)
MARCH.<base>space separated list of allowed architectures for <base>
(see below for further explanation)
MDEFAULTBASEMinimal architecture base supported by this product.
Default <base> architecture if no matching ARCH.<base> is found.
MREQUIRESpackage name or tag which must be provided by the installed product.
OLINGUASISO <language> or <language>_<country> code
OLABELUTF-8 encoded label.
Default label if LINGUAS is omitted or no default language can be determined.
OLABEL.<lang>UTF-8 encoded label.
<lang> has the same syntax as the LINGUAS values.
For each language in LINGUAS a matching LABEL.<lang> is expected.
MDESCRDIRpackage description directory
(relative to product directory)
MDATADIRpackage data directory
(relative to product directory)
OLANGUAGEdefault language code
OTIMEZONEdefault timezone
ORELNOTESURLurl of release notes, see below
OFLAGSproduct specific capabilities, see below
OYOUURLURL of text file containing list of online update servers
OYOUTYPEcomma-separated list of YOU flags. business for SuSE business products, noyou for products to be ignored by YOU
OYOUPATHpath to patches on the server
(triggers YOU to look at the maintenance web server for updates.
prio 'M' == mandatory, 'O' == optional
Either LABEL or LABEL<lang> must be present.

The ARCH.<base> line needs a bit more explanation:
Supporting bi-arch architectures and optimized packages requires multiple (architecture optimized) occurances of a package (under the same name and version).
In order to determine which the 'right' occurance is, YaST needs a fallback strategy (a search order).
The <base> tag is used as a starting point and determines 'the' architecture of the system. This is determined by running uname -m (and looking at /proc/cpuinfo).
For this <base>, a matching ARCH.<base> must exist. If it isn't found, <base> gets the value of DEFAULTBASE for which a matching ARCH.<base> is guaranteed.
The value of the ARCH.<base> line is the search order for packages. The first package matching the architecture is taken as the preferred candidate.
Packages with architectures not in this list are discarded and do not appear for selection. Packages with 'lower' architectures are still available for selection, so the user can manually choose such a package.

The RELNOTESURL line is used to define the location of the release notes rpm. Currently only http is supported. The rpm is installed with rpm -U --replacepkgs 'basename of url'.

The FLAGS line is used to describe the (installation) capabilities of the product. It's value is a list of space separated keywords.
The following keywords are currently defined:
update  Offer update button in YaST
usercwd  Add '.' (current working dir) to PATH environment.
Normally, this is a risk but some users can't live without it.
no_you  Do not offer online update during installation
SP  This source is a service pack
cache  Try to cache packages on disk (in order to reduce CD shuffling)
Only the first source (ordered by priority) will really be cached.

The location of the rpm file for a package on the installation media is determined by the =Loc: line in the
packages file.

Example

linux# mount /dev/cdrom /mnt
linux# cat /mnt/sles/content
PRODUCT SuSE Linux Enterprise Sever
VERSION 8.0-0
BASEPRODUCT UnitedLinux
BASEVERSION 1.0-0
VENDOR SuSE Linux AG
ARCH.x86_64 x86-64 i686 i586 i486 i386 noarch
ARCH.i686 i686 i586 i486 i386 noarch
ARCH.i586 i586 i486 i386 noarch
DEFAULTBASE i386
LINGUAS en de_DE
LABEL.en SuSE Linux Enterprise Sever based on UnitedLinux
LABEL.de_DE SuSE Linux Enterprise Sever basierend auf UnitedLinux
DESCRDIR suse/setup/descr
DATADIR suse
LANGUAGE de_DE
TIMEZONE Europe/Berlin
RELNOTESURL http://www.suse.com/relnotes/sl_82/release-notes.rpm
FLAGS update
YOUURL http://www.suse.de/cgi-bin/suseservers.cgi
YOUTYPE business
YOUPATH i386/update/SuSE-SLES/8
linux#

The patches file

This file contains a single directory name for the root of the patches directory contained on the media and an optional list of exclusive products the patches apply to.
If this file is not present, the media's root directory is assumed and no restrictions about the products are made.

The first line of the file starts with the directory name (relative to the media's root dir!), followed by whitespace (space or tab) followed by an optional comment with the description/patchlevel of the Patch CD.
A leading slash in the directory name is only needed to specify the root directory of the media.

Additional lines are used to specify exclusive products. If no additional lines are present, no exclusive products are assumed. If there are exclusive products the online or patch CD update ignores all products on the installed system which are not contained in the list of exclusive products. If there are no exclusive products all products on the installed system are used for the update.

The exclusive products are listed as one product per line in the form of the product name and the product version separated by a dash.

Proposal:
File:patches
Location/media.1/ directory
Contentone line of ASCII as follows
<directory> <whitespace> <optional comment>
zero or more lines specifying exclusive products: <productname>-<productversion>

Example

linux# cat /mnt/media.1/patches
/ Patch-CD SuSE-SLES 7 (i386) Patchlevel 4
linux# cat /mnt2/media.1/patches
patches Patch-CD SuSE-SLOX 4 (i386) Patchlevel 1
SUSE CORE-9

The package description directory

Each such directory contains a file listing the selections (selections) and a file listing the packages (packages).

Example

linux# grep Enterprise /mnt/media.1/products
sles	SuSE Linux Enterprise Server
linux# grep DESCRDIR /mnt/sles/content
DESCRDIR suse/setup/descr
linux# ls -R /mnt/sles/suse/setup/descr
...
packages
selections
...
linux#

The packages file

This is basically a cache file for package data needed for package selection and dependency resolution.
See current
here for the syntax.

The packages file must exist in order to enable a media.

The package file contains pure package / rpm data and no user readable string (i.e. translations).

The packages.LANG file

For each language defined in content:LINGUAS a packages.LANG file exists in the content:DESCRDIR directory.
This file contains translated strings (i.e. package summary, description, etc.) to be viewed by the user.

The packages.LANG file might be ommitted. The package summary defaults to name-version-release.architecture in this case.

See here for a format definition of such a file.

The packages.DU file

This file contains disk usage information for each package and directory used by the package.
It it used to approximate file system requirements esp. when multiple partitions (i.e. for /usr, /var, /opt, ...) are used.
Exact usage information can't be computed by YaST since it heavily depends on hard or symbolic links (either in the package or in the file system) and the filesystem in use (i.e. ext2 vs. reiserfs).

See here for a format definition of this file.

(N.B. The .DU suffix does not interfere with the .LANG suffix since the language suffix starts with lower case letters.)

The packages.DU file might be ommitted. No size estimations can be given in this case.

Example

linux# grep Enterprise /mnt/media.1/products
sles	SuSE Linux Enterprise Server
linux# grep DATADIR /mnt/sles/content
DATADIR suse
linux# grep ^ARCH /mnt/sles/content
ARCH.x86_64 x86-64 i686 i586 i486 noarch
ARCH.i586 i586 i486 noarch
linux# ls -R /mnt/sles/suse
...
x86-64/
i686/
i586/
i486/
noarch/
...
linux#

The selections file

This file contains a list of selections file names (one per line, possibly with directory pathes, relative to the DESCRDIR directory).
See current *.sel for the syntax. (To be defined).

The selections file might be ommitted. YaST tries to access *.sel files in the DESCRDIR directory in this case.

Example

linux# cat /mnt/suse/setup/descr/selections
Games.sel
Gnome.sel
Kde-Desktop.sel
Kde.sel
LAMP.sel
Linux-Devel.sel
Minimal.sel
Multimedia.sel
Network.sel
Office.sel
Standard.sel
SuSE-Documentation.sel
X11.sel
default.sel
linux#

Reading directories

When YaST reads a directory by first listing its content, it looks for a file directory.yast and takes its content as directory listing, if it exists. If it doesn't exist, it tries to list the directory by using the media backend. Listing directories isn't supported by all backends, e.g. the HTTP backend doesn't support directory listings. So directories which are required to be listable by YaST for all media have to include the directory.yast file.

The file can be created by ls >directory.yast. YaST ignores the directory.yast entry itself and the . and .. entries.