Trf: Seeking on transforms
Author: Andreas Kupries,,,

# Seeking

[ Commands | Sitemap ]

The way transforms are handling seek requests was completely revamped in version 2.0. This page now seeks to explain the new behaviour to anyone trying to port an application using old-style seeks.

A short recapitulation of how it was before 2.0: Any seek request was passed unchanged to the channel (or transformation) below it. Therefore seeking was always relative to the notion of the base channel about its access point.

Before actually stating the possible new behaviours of a transformation with regard to a seek request some notes on how it is selected.

The code now places each transformation in one of two groups:

1. Either input and out are in a linear relationship,
2. or they are not.

The name of attribute used to describe this is the 'seek policy'. In the case of members of the second group its value is 'unseekable'. For all other transformations it is a list of the two numbers describing the ratio between input and output.

Examples from the first group are almost all conversions provided by Trf, like 'hex', 'base64', etc. 'hex' for example converts every byte into 2 ASCII characters during encoding, thereas 'base64' encodes every triple of bytes into 4 ASCII characters.

Members of the other group are the (de)compression algorithms and the 'ascii85' conversion.

What I described above is actually the 'natural seek policy' of the transformation. The moment some transformations are stacked upon each other the new code will consider the properties of the base channel and the policies of all the transformations below it while selected a 'chosen seek policy'. Under most circumstances this is the 'used seek policy', but sometimes the user is allowed to overide the choice with one of his own, so a difference is possible.

The rules for the selection of the 'chosen seek policy' are thus:

- An 'unseekable' base channel (= no SeekProcedure in his driver, examples: socket, pipe) implies an 'unseekable' transformation, regardless of the natural policy. The user cannot overide this choice.
- An 'unseekable' transformation below (used! policy) implies an 'unseekable' transformation, regardless of the natural policy. The user cannot overide this choice.
- The natural policy is chosen. The user is allowed to overide the choice.

The order of the rules above implies the order of their application and dominance. I.e. the first applicable rule determines the chosen policy, the rules behind it are not considered anymore and have no effects.

Remark: Whereas Trf is able to peek into the structures of the transformations provided by itself it has no way of doing so with transformations coming from different packages. Because of this it goes the safe and easy way, i.e such transformations are considered as 'unseekable'. Even if they are not, Trf is unable to ascertain this.

In the case of an overidable chosen policy the option -seekpolicy can be used during attachment of a transformation, or later via 'fconfigure' to enforce the choice of the user. The available values and their meanings are:

unseekable
  The transformation is forced to use 'unseekable' as its seek policy.
identity
  The transformation is forced to be seekable, with a ratio of 1:1. This effectively restores the old behaviour of transformations. Beware that this is possible only if neither the base channel nor all transformations below are unseekable !
{}
  This value restores the chosen seek policy as the used one.

Now we can talk about the behaviour of a transformation under the various seek policies.

Unseekable

- An unseekable transformation keeps a running count of the number of bytes written to or read from it. This number is returned upon request, i.e. a call to 'tell'.

Seekable with ratio n:m

- n:m means that n bytes in the input are transformed into m bytes of output (for writing to the channel).
- The transformation channel is restricted to seeking to locations which are multiples of n. Reading and writing is not restricted in this way.
- Seeking relative to the end of the stream is not permitted. (I currently don't quite know how to deal with incomplete data at the end of the channel downstream. Some transformations are able to handle such, some are not; and the effects on where the end of the stream actually is or should be are not clear).
- If a non-multiple of n number of bytes is written into the channel the incomplete tuple is stored in the transformation for the next write. A read or seek will discard this partial information.
- Each seek discards the information in the buffers of the transformation, both the read-ahead buffer of complete tuples and internal buffers containing partial information. Actually a little optimization is made. The read-ahead buffer is not discarded if the new location of the access point is still inside its boundaries.
- Seeking a channel X and then stacking a transformation upon it will cause the transformation to use the current location in X as its base location, i.e. 0. Seeking behind that point is not allowed.
- Forcing a transformation into 'identity', seeking it and then restoring the chosen policy will cause the transformation to use the new current location in the channel downstream as its new base location.
- Seeking an unseekable transformation by forcing it into identity will discard any state it has.

Two read-only options to 'fconfigure' are available to get at the current configuration and state of the seek system for a particular transformation:

-seekcfg
  The value is a list containing keywords and values (notation suitable for 'array set') describing the current configuration of the seek (natural and chosen policy, ability to overide them).
-seekstate
  The value is a list containing keywords and values (notation suitable for 'array set') describing the current state of the seek system (used policy, various locations).

-seekcfg
Key | Value
ratioNatural | The value is a list containing two integer numbers describing the natural ratio of input to output for the transformation. If either number is 0 the transformation is unseekable.
ratioChosen | The value is a list as described above, but this time containing the chosen ratio, after considering the base channel and all transformations below.
overideAllowed | A boolean value. Set if the user is allowed to overide 'ratioChosen' with his own preferences.
identityForced | A boolean value. Set if the user forced the transformation via -seekpolicy to use 'identity' while doing seeks.

-seekstate
Key | Value
ratio | See 'ratioNatural' and 'ratioChosen above. Describes the ratio actually used.
seekable | A boolean value, derived from 'ratio'. Set if the transformation is 'unseekable'.
up | An integer number. The location of the access point in the stream before/above the transformation, as the transformation sees it. This value can differ from the one returned by [tell], the difference is the number of bytes buffered in the generic I/O layer of Tcl for this channel.
upBufStart upBufEnd | See above, describing the range in the transformed stream covered by the read-buffer of the transformation maintained by Trf itself.
downBase | The location in the channel downstream which is equivalent to 'up == 0'. See the description some paragraphs above about a 'base location'.
down | The current location of the access point in the channel downstream. Not equivalent to 'up' due to the read-ahead done by Trf, but to 'upBufEnd'. Actually not quite excatly equivalent as Trf may have read a non-multiple of m number of bytes.
downAhead | 'down - downAhead' is the real location in downstream equivalent to 'upBufEnd'. It basically is the number of bytes in the last and incomplete tuple read from below.
changed | A boolean value. Set if the transformation was forced into identity (see above) and the user actually seeked after forcing this.

Last update at Mon Aug 20 22:52:27 PDT 2001