NQSRN(1)

NAME
NQSRN - Network Queueing System Release 3.36 Release Notes.

DESCRIPTION
The Network Queueing System (NQS) provides the means to submit batch jobs to local and remote Unix and VMS machines. This is the release notes for Release 3.36 of NQS at Monsanto, and includes information on all changes made to NQS since the release of Version 3.35. The release notes for earlier versions of NQS at Monsanto are appended to this document.

Version 3.36 consists of several small fixes and enhancements to release 3.35. Here are the major changes since release 3.35:

This version of NQS now supports Solaris. Thanks for the help from Don Hjort (dhjort@fhcrc.org).

There is now some support for compute servers of varying compute power. Previously the load balancing code assumed that the servers all had the same compute power. Now it is possible through the "help set server" command in qmgr to define the relative performance of servers to the scheduler. The scheduler will then take that information into account when assigning jobs to servers. See the qmgr man pages for more information.

I received more fixes to the DEC OSF/1 code from Bruno Wolff III (bruno@alpha2.csd.uwm.edu) which I have added to the code.

A fix was made to correct problems relating to interpreting times and the Daylight time / Standard time switch. It was discovered in the fall of 1993 when the date of the switch did not match when NQS thought it should be. This release will be out in time to see how it works in 1994.

This version now supports IRIX 5.2 and this is the default version.

The code for determining if an inbound load balanced queue should reject or permit a request has been improved.

The qalter utility has been added. See the man page for more information.

When the nqsdaemon sends a broadcast message, it now sets up the broadcast child as its own session and does not wait for it.

The -S switch has been added to qacct to create summary reports. See the man page for more information.

The printing of debug messages has been reviewed so that no messages are printed if the debug level is set to 0. Of course, the cure for exessive log messages is to set /dev/null as the log file.

A file descriptor leak was fixed by changing establish.c so that the socket is closed if there are any errors.

The QSUB_QUEUE environment variable has been added which provides the name of the execution queue to the running request.

  NQS Version 3.35 Release Notes

The main enhancement in this release is the ability to stage a release of NQS. A common problem is that long running NQS jobs prevent upgrading to a newer version easily. This feature allows one to place a new release of NQS in a staging area. NQS checks when a job completes to determine if there is no other activity, and if not, will shut itself down, install the new version and restart itself. This is useful when it is not urgent to install a new release of NQS, and one does not want to disable the queues.

Here is a summary of other changes since Version 3.34:

The makefiles were reorganized, with the site-specific defines at the top and the architecture-specific defines further down.

Qmgr set pipe_client now checks to see if the pipeclient file exists and is a regular file and will not allow you to change it if it isn't.

Fixed a small, but embarassing, security hole in qdel.

The $(NQS_SPOOL)/private directory has been made mode 700 rather than 755. This increases the security of the NQS spool area. For existing sites you should manually do a chmod on this directory.

Added the -u flag for qcat to indicate the owner of the request.

The snap file is now written to the current working directory unless an absolute path is given.

The entry for the qmgr command "delete request" was removed, since it didn't work anyway.

Determined by trial and error the maximum CPU time allowed in the setrlimit on SGI machines: 21474837 (Irix 4.0.5). If attempt to set a limit higher than this, the message "Overflow or semantic error in cpu time limit." will be returned.

Remote request completion message to the NQS scheduler are now not checked for a valid username, as not all users will have accounts on the scheduler machine.

Fixed bugs with processing load information and setting scheduler for mids greater than 0x7fffffff. Thanks to Ryoichi Shibano (shibano@bsd2.kbnes.nec.co.jp).

  NQS Version 3.34 Release Notes

The main changes in this release is the addition of two enhancements supplied by Chuck Keagle at Boeing, The first is the ability to assign Machine IDs implicitly based on the IP address of the hosts. This makes it easier to do distributed management of NQS. The second change makes it easier to install the parts of NQS into locations other than the standard (/usr/lib/nqs, /usr/spool/nqs, etc.). Thanks to Chuck and Boeing for making these enhancements available.

Other changes include:

Added support for DEC's OSF for DECstation AXP machines courtesy of Claus Kalle, University of Cologne (Kalle@rrz.Uni-Koeln.DE).

The problem with IRIX and NIS has been fixed, so that one no longer has to link against old versions of the IRIX libraries.

Added the ability to indicate a token in the sdout and/or stderr name of the request which would be replaced by the request number. This token is defined as "#".

Fixed the problem which occured that when the request name started with a digit qsub would drop the last character of the request name.

Added the "unimplemented feature" of qresume/qsuspend -a.

Changed the utilities to print out the NQS version, rather than a version of their own.

Documented nmapmgr by creating nmapmgr.1m.

Documented qmsg by creating qmsg.1.

Changed nmapmgr to run the CREATE command implicitly if the database has not already been created.

Fixed bug in loading queue complex configuration courtesy of Tom Schwab of Gesellschaft fuer Schwerionenforschung mbH (schwab@rzri6b.gsi.de).

Qstat now returns 0 if all queues are found, and 1 if there is any error.

Qstat @local-machine gets the status of the local machine instead of returning the error "Queue: @local-machine does not exist"

Accounting file now records the request name.

There have been several changes to qacct: enabled the -a (after), -b (before), -i (input file), and -o (output file) switches. Added request name to the Request Start message and the report listing. Added the -s (summary) and -r (rollover) processing.

NQS now does not count queued requests when deciding whether to allow a request to be transferred when a queue is load balanced inbound.

Fixed bug where excess TCP/IP ports would be kept open after sending messages to the NQS Scheduler.

Time zones with long names were causing problems at certain locations, so the default time zone buffer in nqs_boot.c was increased in size.

  NQS Version 3.31 Release Notes

  Changes to User Commands

New utility -- Qcat
    The Qcat utility has been provided which allows one to view the spooled NQS log file, error file, or input script file.

New utility -- Qacct
    The Qacct utility prints out the NQS accounting information in a summary format.

Qsub changes
    Requests now default to not restartable. The -rs switch has been added to indicate that a request is restartable.

Qstat changes
    Qstat -l now indicates whether a job is restartable or recoverable.
    Each user can specify a list of systems to check when running the "qstat -d" command. Qstat now checks for a file in the user's home directory called .qstat with the same format as the /usr/lib/nqs/nqs-domain file. If present, this file will supercede the system-wide file.

Qlimit changes
    Qlimit now can print limit information for remote systems and the -v switch has been added to print out the version number.

Broadcast messages
    A user can override the system default list of systems to which the broadcast message is sent. The request termination processing code now first checks for a file in the user's home directory called .nqs-domain with the same format as the /usr/lib/nqs/nqs-domain file. If this file is present in the user's home directory on the execution machine, broadcast messages will be sent to this list of systems rather than the default list.

  Changes to System Manager Commands

Accounting changes
    NQS now writes startup and shutdown messages to the accounting log. The request completion message now includes the completion time.

Logfile handling changes
    The log file /usr/lib/nqs/logfile is no longer automatically created when NQS starts up. The logfile is not truncated when opened, so that successive restarts will append to the logfile. If this is not the desired behavior, you must manually delete the logfile before starting up NQS. A sample script is provided in support/nqs_log_rollover to roll over the NQS log files daily and delete the files older than 7 days.

Load Balancing Changes
    Load Balanced outbound processing was fixed to round robin by position in destination list, not by machine ID. Load Balance inbound processing was modified to consider complex user limits and complex run limits.

Scheduling Changes
    The NQS scheduling algorithms have been changed. The scheduler now checks the list of waiting jobs before any jobs are scheduled. Previously, waiting jobs had to be triggered by a timer, which meant that the system may be idle waiting for the timer to expire. Since NQS now distinquishes between jobs actually waiting for a certain time and jobs waiting for resources, it can schedule jobs more promptly with less waste of system resources.

    There is a new concept of an NQS scheduler. This is the node that does the scheduling for all the nodes in the NQS "cluster". This node should do the load balancing for the other nodes. It is defined by setting the scheduler using the new Qmgr command "set scheduler". The system will start up a new NQS daemon called the loaddaemon which will send load information to the scheduler node at intervals. The interval defaults to 3 minutes, but can be modified using the "set default load_interval" Qmgr command.

    There has been the extension of the NQS network protocol to add two new message types. The first is the remote request completion message and the second is the load message. These are directed to the scheduler, so as long as a scheduler is not defined, the protocol is the same as earlier versions.

Delivery of Jobs to Pipe Queue Destinations
    The way jobs are delivered to pipe queue destinations may be confusing. Here is a short explanation:

    If a pipe queue has multiple destinations but is not load balanced outbound, the same destination will be chosen every time. This destination will be the queue on the system with the lowest mid. This may or may not be the first destination in the list when you do a qstat -x.

    If a pipe queue has multiple destinations and is load balanced outbound but is not on the scheduler machine, then the destinations will be chosen using a round robin algorithm. The sequence number of the request will be divided by the number of destinations, and the remainder is the index into the list of the destinations.

    If the pipe queue has multiple destinations, is load balanced outbound, and is on the scheduler machine, then the load information is taken into account when the job is attempted to be delivered to a machine. Assuming that load information is available for all of the machines, the destinations are ordered by load (number of jobs and secondarily the 5 minute load average) and the job is attempted to be delivered to the "lightest loaded" machine.

    The load balancing code is admittedly elementary. It does not distinguish processor speed, memory, or any other characteristic of machines. In addition, the scalability of the implementation has not been tested. The compute cluster used here is on the order of 4-10 machines. The overhead on a scheduler machine in this environment is probably not great. I would hesitate to guess for performance for clusters of 20 or more machines served by a single scheduler.

Check for a default path
    Under some circumstances the default path set up by NQS for a request may not be appropriate. The NQS Shepard processing has been changed to check for a file called /etc/environment to set up the default path. If there is a file /etc/environment (present in AIX 3.2), and it has a line indicating the path with the format:
    PATH=/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin
    then this path will be used for the requests.

Platforms supported
    HP 9000/700 series machines running HPUX systems are supported in this release. Sun support seems to be improved, but I don't have reliable access to a Sun machine, so I cannot be sure.

New qmgr commands
    Here are the new qmgr commands added in 3.31:

    set scheduler <node>
        This command sets the scheduler to the indicated node. This means that request completion messages and load messages (if the load daemon is running) will be sent to this system.

    set no_scheduler
        This command indicates that there is no scheduler.

    set default load_interval
        This command indicates the number of minutes between load messages sent to the scheduler.

    set load daemon=(file-spec)
        This command indicates the full file specification for the load daemon.

    set no_load_daemon
        This command indicates that there is no load daemon specified.

    start load_dameon
        This command indicates that the load daemon is to be started, and is to send load messages to the scheduler indicated.

    stop load_daemon
        This command terminates the load daemon.

    set echo
        This command causes qmgr to echo every line of input. Its primary use is to echo command lines in regression testing scrips.

Bug Fixes

Fixed problem with incorrect start times with qstat -m.

Fixed problem on SGI where qstat local and remote showed different start times.

Fixed bugs in Qsuspend / Qresume processing

Fixed the queue ordering for waiting jobs to strictly FIFO.

When running the test version, give the processes the appropriate names (This does not work on SGI platforms).

An NQS manager can now suspend or resume other's jobs.

NQS now tolerates whitespace between the hostname and username in the .rhosts file.

  NQS Version 3.20 Release Notes

An additional concept has been added to this version of NQS. This is concept of an NQS "domain". An NQS domain is all of the machines which submit NQS jobs among each other. It includes the compute servers and the personal workstations of a group. This concept has been utilized in the use of broadcast messages, as indicated below.

Here is a list of the changes and enhancements:

Queue complexes
    NQS managers can now associate several queues and create a user run limit and a total run limit for the queues as a group.

Move queue and requests
    NQS managers can move one or more queued requests from one queue to another. In the same way they can move all queued requests in a queue to another queue.

Hold and Release requests
    Queued requests can be held, which means that they will not be scheduled to run until manually released. Qhold and Qrls are the new commands which implement these functions.

Suspend and Resume requests
    On SGI computers one can suspend running requests and resume suspended requests manually. Qsuspend and Qresume are the new commands which implement these functions.

Modify requests
    NQS managers can modify the time limit, nice value, and request memory limit of queued requests using Qmgr.

Load balancing
    Simple load balancing has been implemented by allowing pipe queues to be designated load-balanced inbound or outbound. Load-balanced inbound means that the pipe queue will not accept requests unless it can immediately run in one of its destination queues. Load-balanced outbound means that the pipe queue will round-robin on its destinations rather than always selecting the first destination in the list.

Log file return
    NQS will make every effort to return the log file. If it cannot return the file to the designated location, NQS will try to deliver the log file to the user's home directory on the execution machine. If NQS cannot deliver the file there, it will move it to the directory /usr/spool/nqs/dump/<username> and send a mail message to that effect. The user will have to move the log file out manually.

Qstat changes
    There is a new Qstat switch, -c. This prints out information on queue complexes. In addition, the sense of the -d switch is changed slightly. It now checks all of the queues in the NQS domain rather than all systems which are destinations for the pipe queues.

Qsub changes
    There are two new Qsub commands, -bb and -be, which indicate that a broadcast message is to be written to the user's terminal(s) when the job begins and ends, respectively. The message indicates the time, the request, and the system on which it is running. The message is delivered to all terminals within the NQS domain. If one does not want the message displayed on a particular window, use the mesg -n command in that window to turn the messages off.

Qmgr changes
    Qmgr now accepts single commands from the command line and returns to the shell. Qmgr has numerous command additions. Notable is the addition of the snap function to print out the current NQS configuration, and start nqs which starts up NQS. The Qmgr help screen is now sized to your terminal window.

New Qmgr commands are

    Create complex
    Delete complex
    Set complex run_limit
    Set complex user_limit
    Move request
    Move queue
    Modify request
    Set global_pipe_limit
    Set queue lb_in
    Set queue lb_out
    Set queue pipeonly
    Set queue nolb_in
    Set queue nolb_out
    Set queue nopipeonly
    Snap
    Start nqs

  NQS Version 3.0 Release Notes

There are several changes which are general and apply to all parts of NQS. These are:

1.  Queue names can be a maximum of 31 characters in size.

2.  Request names can be a maximum of 63 characters in size.

3.  Several new limits have been added to queues. For further information, see the Qmgr and Qsub descriptions below. The new limits are:

    Corefile limit
    Data limit
    Nice value limit
    Per process cpu limit
    Per process memory limit
    Per process permfile limit
    Stack limit
    Working set limit

4.  The -v switch has been added to utilities to print their current version.

Information on changes to specific programs follow. The man pages for these uilities have been updated to indicate the changes.

  Qdel
The main change to Qdel is the addition of the -r and -c switches. The -r switch allows you to indicate a request name pattern to delete. The -c switch, which is only valid with -r, indicates that the user is to be prompted to confirm each deletion. Qdel will print the name of the request matching the pattern and query the user whether the request is to be deleted or not, or to quit.

  Qsub
Qsub has been enhanced with the -d switch, which indicates that the script file is to be deleted after spooling. This is valuable for temporary scripts that can easily be recreated.

In addition, the switches associated with the new limits above are no effective. The Qsub switches and their corresponding limits are as follows:

        -lc    Corefile limit
        -ld    Data limit
        -ln    Nice value limit
        -lt    Per process cpu limit
        -lm    Per process memory limit
        -ls    Per process permfile limit
        -lw    Working set limit

  Qstat
Qstat has been extensively changed from the original Cosmic release. A new summary format has been made the default, and several additional switches have been added to indicate which requests are to be displayed. See the man pages on Qstat for further information.

  Qmgr
The Qmgr utility has been modified in several ways. Here are the changes, and the commands that are affected:

    1.  Set a user limit for a queue (create batch_queue, set user_limit). The allows the system manager to set a limit on the number of jobs that an individual user can run in a queue at the same time.

    2.  Set a non-degrading priority on a queue (set ndp). This sets a non-degrading priority on a queue to determine the effective priority of requests in that queue.

    3.  Set a global batch limit (set global_batch_limit). This determines the maximum number of jobs that can run at any one time.

    4.  The commands which modify the above limits on requests in queues
        are now effective. The commands are:

        set corefile_limit=(limit) queue
        set data_limit=(limit) queue
        set nice_value_limit=(nice-value) queue
        set per_process cpu_limit=(limit) queue
        set per_process memory_limit=(limit) queue
        set per_process permfile_limit=(limit) queue
        set working_set_limit=(limit) queue

SEE ALSO
nqs(1), nqsconfig(1), nqsgs(1), qacct(1), qcat(1), qdel(1), qdev(1), qhold(1), qlimit(1), qmgr(1), qpr(1), qresume(1), qrls(1), qstat(1), qsub(1), and qsuspend(1) in the NPSN UNIX System Administrator Reference Manual.

NQS HISTORY
Origin: Sterling Software Incorporated

May 1986 - Brent Kingsbury, Sterling Software
Original release.

August, 1994 - John Roman, Monsanto Company
Release 3.36.