Teki install file format (*.tek files)

Introduction
    This file describes the format of TEKI install files (.tek files).  It
    is probably only of interest to Tcl extension writers.

    A .tek file is read by The Tek Installer (TEKI) to announce the
    availability of one or more extensions to TEKI.  The .tek file
    communicates this information to TEKI by setting elements in
    the newPackage array.  To read the .tek file , TEKI executes
    the Tcl command "source filename.tek" in the directory where the file
    is located. Thus, the file can execute arbitrary Tcl commands (note 1)
    in setting the values in newPackage.  This also means that the
    easiest way to find errors in the file is to try source-ing it
    in a Tcl interpreter.

    The easiest way to make a .tek file is to start with one of the
    example .tek files and modify it.  I'll walk through a sample
    file below.

    The first line of a .tek file looks like this:

    # TekiFile 1.0

    It identifies this file as a .tek file, and gives the version number
    of this .tek file (note 2).  This information is hidden inside a Tcl
    comment.

Concepts
    The rest of the file contains commands to set elements in the
    newPackage array.  Before describing these elements in detail, I
    will describe a few concepts used in the file.

    o Each extension has a unique identifier that is used internally to
      identify a single extension.  The id must be one word (i.e., it
      must contain no white-space).  For example, "dp4.0" is the id of
      the Tcl-dp extension, version 4.0.

    o Each extension consists of seven types of files, all of which are
      optional:
        - an informational file (e.g. a readme) that describes the extension
        - a copyright file
        - a set of architecture independent Tcl files that are source'd when the
          extension is loaded
        - a set of architecture dependent Tcl and object files that are loaded
          (using the Tcl "load" command) when the extension is loaded.
        - a set documentation files, in HTML format
        - a set of data files (e.g., images or sample data) used by the extension
        - a set of examples

    o To support architecture dependent files (such as shared libraries), Teki
      predefines a set of "system names."  Examples are names like
      "solaris," "win95," and "windows."  System names are matched with
      names and versions of the operating system,  For example, the "solaris"
      system is matched with SunOS version 5.*

    The .tek file has three parts.  The first part defines attributes that
    are global to the .tek file.  The second part defines attributes for 
    each package in this .tek file.  The third part defines default
    installation parameters.  These parts are described in detail below.
    In each part, a sample from a .tek file is given follow by an
    explanation.  Keep in mind that the newPackage array is empty
    and the current directory is the directory where the .tek file
    is located when the file is sourced.

File-wide variables

    lappend newPackage(available) dp4.0
    lappend tekiInfo(systemNames)   {
            {OSF Ultrix 2*}
    }

    This section defines two attributes: available and systemNames

    "available" is a list giving the identifiers of the packages
    defined in this .tek file.  You should use lappend, rather than
    set, to define the avalable attribute so that your extension can
    be easily bundled (bundling is discussed below).

    "systemNames" is a list of system definitions.  These names are
    used to identify architectural dependencies at runtime (note 3).
    Each definition is a three-tuple consisting of the system name,
    the operating system (OS), and the OS version number.  The OS and
    OS version are determined at runtime by matching (glob-style)
    the OS and OS version in the system definition to the
    tcl_platform(os) and tcl_platform(osVersion) variables.  These
    systemNames are stored in the tekiInfo global variable, which
    is set by teki to the following value:

        set tekiInfo(systemNames)   {
                {Solaris SunOS 5*}
                {SunOS SunOS 4*}
                {HPUX HP* 9*}
                {Linux Linux* 2*}
                {Win95/NT Win* *}
        }

    For example, some of the systemNames defined above are Solaris, SunOS,
    and Win95/NT.  Solaris corresponds to tcl_platform(os) ~= SunOS and
    tcl_platform(osVersion) ~= 5* (where ~= denotes glob-style string
    matching).  Similarly, the Win95/NT matches any version of the
    operating system where tcl_platform(os) ~= Win*.

    Multiple definitions of a single system name are allowed.  Thus,
    the following defines a system named "Windows" that matches Win95
    (any version) and Windows NT version 4, and a system name Solaris2
    that matches SunOS version 5.2 and later:

    lappend tekiInfo(systemNames)   {
            {Solaris2 SunOS 5.[2-9]*}
            {Win95/NT Win95 *}
            {Win95/NT {Windows NT} 4*}
    }


Package attributes:
    For each package in newPackage(available), several attributes are
    defined.  An excerpt of these definitions from the dp4.0 .tek file
    are shown below:

        set newPackage(dp4.0,name)         dp
        set newPackage(dp4.0,version)      4.0

        set newPackage(dp4.0,description)  "Tcl Distributed Programming"
        set newPackage(dp4.0,requires)     {{Tcl 7.6}}
        set newPackage(dp4.0,updateURL)    http://www2.cs.cornell.edu/zeno/tcl-dp/teki/toc.tkr
        set newPackage(dp4.0,registerURL)  http://www2.cs.cornell.edu/zeno/tcl-dp/register.html
        set newPackage(dp4.0,srcURL)       http://www2.cs.cornell.edu/zeno/tcl-dp/teki/
        set newPackage(dp4.0,srcDir)       .
        set newPackage(dp4.0,destDir)      dp4.0
        set newPackage(dp4.0,copyright)    license.terms
        set newPackage(dp4.0,infoFile)     readme
        set newPackage(dp4.0,tclFiles)     {acl.tcl distribObj.tcl dp_atclose.tcl dp_atexit.tcl ldelete.tcl oo.tcl rpc.tcl}
        set newPackage(dp4.0,exampleFiles) [glob examples/*/*]
        set newPackage(dp4.0,dataFiles)    [glob data/*]
        set newPackage(dp4.0,objFiles)  {
                {Win95/NT dp40.dll}
                {Solaris solaris/libdp.so}
                {SunOS sun4/libdp.so}
        }
        set newPackage(dp4.0,objFiles)  {
                {Win95/NT win/dp40.dll win/gui.tcl}
                {Solaris unix/gui.tcl unix/solaris/libdp.so}
                {SunOS unix/gui.tcl unix/sun4/libdp.so}
        }
        set newPackage(dp4.0,docDir)       doc
        cd doc
        set newPackage(dp4.0,docFiles)     [glob *.html]
        cd ..

    Let's go through this, line by line:

      set newPackage(dp4.0,name)         dp
      set newPackage(dp4.0,version)      4.0

    "name" and "version" give the name and version number of the
    extension.  These correspond to the package require command used
    to load the extension.  In the example above, the extension would
    be loaded using "package require dp 4.0".

      set newPackage(dp4.0,description)  "Tcl Distributed Programming"

    "description" is a short string describing the package.  It appears
    in the TEKI user interface under the manifest of installed packages.

      set newPackage(dp4.0,requires)     {{Tcl 7.6}}

    "requires" is a list of {package version} pairs indicating which
    packages are required.  I don't currently use it, but it could be
    used later to auto-load the required packages (once Web-based loading
    is working)

      set newPackage(dp4.0,updateURL)    http://www2.cs.cornell.edu/zeno/tcl-dp/teki/

    "updateURL" is a URL where package updates can be found.  It will
    eventually be used by the "update" command to automatically install
    new versions of a package.  It's currently not used.

      set newPackage(dp4.0,registerURL)  http://www2.cs.cornell.edu/zeno/tcl-dp/register.html

    "registerURL" is a URL where users can go to fill out a regiistration
    form.  It's currently not used.

        set newPackage(dp4.0,srcURL)       http://www2.cs.cornell.edu/zeno/tcl-dp/teki/
      set newPackage(dp4.0,srcDir)       .

    "srcURL" and "srcDir" are the base directories for find the
    files to copy.  The srcURL is the base URL where all the documentation,
    tcl, binary, data, and example files can be found -- it is preprended
    to every file in the package for web based installs.
    The srcDir is relative to the directory where the .tek file is
    located, and contains all files relevant to this package.  It's
    useful when bundling many packages, since each can have its
    own subdirectory.  If you're bundling just one package, you can
    set it to "."

      set newPackage(dp4.0,destDir)      dp4.0

    "destDir" is the name of the directory where the package will
    be installed.  Typically, this is the name of a sibling directory
    to [info library] that will be created when the package is installed.
    On my system, [info library] returns /usr/local/lib/tcl/tcl7.6, so
    the example above would install Tcl-DP in /usr/local/lib/tcl/dp4.0
    (although the prefix, /usr/local/lib/tcl, is determined by the user
    at run-time).  We call this the "code target directory" in what
    follows.

      set newPackage(dp4.0,copyright)    license.terms

    "copyright" is a name of a file giving the copyright notice and
    license terms.  It's currently not used, but will eventually be
    thrown up in a dialog (with "accept" and "do not accept" buttons)
    when the pakcage is about to be installed.

      set newPackage(dp4.0,infoFile)     readme

    "infoFile" is a file describing the package. Its contents appear
    in the upper portion of the TEKI window when the user selects on
    this package.  It's typically a README file.

      set newPackage(dp4.0,tclFiles)     {acl.tcl distribObj.tcl dp_atclose.tcl dp_atexit.tcl ldelete.tcl oo.tcl rpc.tcl}

    "tclFiles" is a list of Tcl files that are source'd when the
    extension is loaded via the "package require" command.  All
    file names are relative to the .tek file's directory.  The are
    installed in the code target directory.  The directory structure
    is preserved in the target directory.

      set newPackage(dp4.0,exampleFiles) [glob examples/*/*]

    "exampleFiles" is a list of example or demo files.  These are
    (optionally) copied to the code target directory.  The directory
    structure is preserved in the target directory.  The files in
    the "demos" subdirectory of the tk directory is an example of
    a set of exampleFiles.

      set newPackage(dp4.0,dataFiles)    [glob data/*]

    "dataFiles" is a list of data files that are optionally installed
    in the code target directory.  These might include sample data sets
    and the like.  The directory structure is preserved in the target
    directory.  The files in "$tk_library/_demos/images" is an example
    of dataFiles

      set newPackage(dp4.0,objFiles)  {
              {Win95/NT dp40.dll}
              {Solaris solaris/libdp.so}
              {SunOS sun4/libdp.so}
      }

    "objFiles" is a list of architecture dependent files associated with
    a system.  The format of each list element is a system name (which
    must be defined in the newPackage(systemNames)) followed by a list
    of object and tcl files.  

    At install time, the user selects which system names to install.
    The associated files are copied to the code target directory,
    preserving any directory structure that is present.  At run-time,
    when the extension is loaded through a "package require" command,
    the run-time system-name is found as described above (see
    "systemNames"), and the associated files are loaded or source'd.
    Files matching *.tcl will be source'd, other files will be load'ed

    In the excerpt above, object files are defined for three
    architectures, SunOS, Solaris, and Wind95/NT.

    As a more complex example, suppose my architecture dependent files
    consist of win/dp40.dll (for windows) and unix/solaris/libdp.so
    (solaris), and unix/sun4/libdp.so (SunOSv4). Furthermore, under
    unix systems, the file unix/gui.tcl has bindings appropriate for
    x-windows, and win/gui.tcl has bindings for Windows 95 and NT.
    Assuming Solaris, SunOS, and Wind95/NT are defined, I would set
    objFiles like this:

      set newPackage(dp4.0,objFiles)  {
              {Win95/NT win/dp40.dll win/gui.tcl}
              {Solaris unix/gui.tcl unix/solaris/libdp.so}
              {SunOS unix/gui.tcl unix/sun4/libdp.so}
      }

    As you can see, the mechanism is fairly powerful and flexible.

      set newPackage(dp4.0,docDir)       doc
      cd doc
      set newPackage(dp4.0,docFiles)     [glob *.html]
      cd ..

    "docDir"  and "docFiles" are used for documentation.  "docDir" is
    a directory where documentation files are stored, and "docFiles"
    is a list of HTML files, whose names are given relative to
    "docDir."

    Just as code file are installed in the "code target directory,",
    documentation is installed in a the "documentation target
    directory."  The name of this directory is obtained from the
    user (e.g., /usr/local/doc/tcl) with the "destDir" attribute
    appended.  For example, on my windows system, I install Tcl_DP
    in C:\Program Files\Tcl\doc\dp4.0.  All the html files in "docDir"
    will be copied to the documentation target directory, and the
    directory structure beneath "docDir" will be preserved.

Defaults

    The third section of the .tek file defines the default paramters
    for installation.  This is what happens when the user selects the
    "typical" install option (as opposed to "custom" installation).
    This section sets five variables:

    set newPackage(defaultPackages)        dp4.0
    set newPackage(defaultArch)            all
    set newPackage(defaultInstallDoc)      1
    set newPackage(defaultInstallExamples) 0
    set newPackage(defaultInstallData)     0

    "defaultPackages" is a list of package identifiers installed
    by default.

    "defaultArch" is a list of systems names to install (i.e.,
    defined in the "systemNames" attribute), or the string "all"

    "defaultInstallDoc" is a boolean indicating whether or not to
    install documentation

    "defaultInstallExamples" is a boolean indicating whether or not to
    install example files

    "$defaultInstallData" is a boolean indicating whether or not to
    install data files


-----------------
Notes:

1. This is a potential security hole, but I'm not sure what to do about it.
   After all, the user is installing an extension, which could contain
   arbitrarily bad security breeches.  In the end, I think it all comes
   down to trust.

2. The function TekiReadFile looks at this line first, to be sure that
   the file is a teki install file.  It only source's the file if it
   looks like a TEKI install file.  This will prevent accidentlly
   sourcing a non-.tek file

3. I thought of more complex scenarios, where an extension may only want
   to load some files when a certain condition is true (e.g., when the
   g++ compiler is installed), but decided that such conditions would
   unnecessairly complicate TEKI.  They can always be handled by the
   extension author at load time.

----------------------
Appendix: .tek file for Tcl-DP and the example extension (in one bundle)

# TekiFile 1.0
#
# TEKI install file for Tcl-DP and the "example" extensions
#
set newPackage(available) {dp4.0 example1.0 sample1.0}
set newPackage(systemNames)   {
        {Solaris SunOS 5*}
        {SunOS SunOS 4*}
        {HPUX HP* 9*}
        {Linux Linux* 2*}
        {Win95/NT Win* *}
}

set newPackage(dp4.0,name)         dp
set newPackage(dp4.0,version)      4.0
set newPackage(dp4.0,description)  "Tcl Distributed Programming"
set newPackage(dp4.0,requires)     {{Tcl 7.6}}
set newPackage(dp4.0,updateURL)    http://www2.cs.cornell.edu/zeno/tcl-dp/teki/
set newPackage(dp4.0,registerURL)  http://www2.cs.cornell.edu/zeno/tcl-dp/register.html
set newPackage(dp4.0,srcDir)       dp
set newPackage(dp4.0,destDir)      dp4.0
set newPackage(dp4.0,copyright)    license.terms
set newPackage(dp4.0,infoFile)     readme
set newPackage(dp4.0,tclFiles)     {acl.tcl distribObj.tcl dp_atclose.tcl dp_atexit.tcl ldelete.tcl oo.tcl rpc.tcl}
cd dp
set newPackage(dp4.0,dataFiles)    [glob data/*]
set newPackage(dp4.0,docDir)       doc
cd doc
set newPackage(dp4.0,docFiles)     [glob *.html]
cd ..
set newPackage(dp4.0,exampleFiles) [glob examples/*/*]
cd ..
set newPackage(dp4.0,objFiles)  {
        {Win95/NT dp40.dll}
        {Solaris solaris25/libdp40.so}
        {SunOS sun41/libdp40.so}
        {Linux linux2/libdp40.so}
        {HPUX hpux9/libdp40.sl}
}

set newPackage(example1.0,name)         Example
set newPackage(example1.0,version)      1.0
set newPackage(example1.0,description)  "Example C Extension to Tcl"
set newPackage(example1.0,requires)     {{Tcl 7.6}}
set newPackage(example1.0,updateURL)    {}
set newPackage(example1.0,registerURL)  {}
set newPackage(example1.0,srcDir)       example
set newPackage(example1.0,destDir)      example1.0
set newPackage(example1.0,copyright)    {}
set newPackage(example1.0,infoFile)     {}
set newPackage(example1.0,tclFiles)     {}
set newPackage(example1.0,dataFiles)    {}
set newPackage(example1.0,docDir)       {}
set newPackage(example1.0,docFiles)     {}
set newPackage(example1.0,exampleFiles) {}
set newPackage(example1.0,objFiles)  {
        {Win95/NT example.dll}
}

set newPackage(sample1.0,name)         Sample
set newPackage(sample1.0,version)      1.0
set newPackage(sample1.0,description)  "Example Tcl-only Extension"
set newPackage(sample1.0,requires)     {{Tcl 7.6}}
set newPackage(sample1.0,updateURL)    {}
set newPackage(sample1.0,registerURL)  {}
set newPackage(sample1.0,srcDir)       sample
set newPackage(sample1.0,destDir)      sample1.0
set newPackage(sample1.0,copyright)    {}
set newPackage(sample1.0,infoFile)     {}
set newPackage(sample1.0,tclFiles)     {sample.tcl}
set newPackage(sample1.0,dataFiles)    {}
set newPackage(sample1.0,docDir)       {}
set newPackage(sample1.0,docFiles)     {}
set newPackage(sample1.0,exampleFiles) {}
set newPackage(sample1.0,objFiles)  { }

#
# Defaults for "typical" install
#
set newPackage(defaultPackages)        {dp4.0 example1.0}
set newPackage(defaultArch)            all
set newPackage(defaultInstallDoc)      1
set newPackage(defaultInstallExamples) 0
set newPackage(defaultInstallData)     0
