# The XSLT C library for Gnome

# libxslt

Libxslt is the XSLTC librarydeveloped for the Gnome project. XSLT itself is a an XML language to definetransformation for XML. Libxslt is based on libxml2the XML C library developed for theGnome project. It also implements most of the EXSLTset of processor-portable extensionsfunctions and some of Saxon's evaluate and expressions extensions.

People can either embed the library in their application or use xsltprocthe command line processing tool. This library is free software and can bereused in commercial applications (see the intro)

External documents:

- John Fleck wrote a tutorial forlibxslt
- xsltproc user manual
- the libxml documentation

Logo designed by Marc Liyanage.

## Introduction

This document describes libxslt,the XSLTC library developed for theGnomeproject.

Here are some key points about libxslt:

- Libxslt is a C implementation
- Libxslt is based on libxml for XML parsing, tree manipulation and XPathsupport
- It is written in plain C, making as few assumptions as possible, andsticking closely to ANSI C/POSIX for easy embedding. Should works onLinux/Unix/Windows.
- This library is released under the MITLicence
- Though not designed primarily with performances in mind, libxslt seemsto be a relatively fast processor.

## Documentation

There are some on-line resources about using libxslt:

1. Check the APIdocumentationautomatically extracted from code comments (using theprogram apibuild.py, developed for libxml, together with the xsl script'newapi.xsl' and the libxslt xsltproc program).
2. Look at the mailing-listarchive.
3. Of course since libxslt is based on libxml, it's a good idea to atleast read libxml description

## Reporting bugs and getting help

If you need help with the XSLT language itself, here are a number ofuseful resources:

- I strongly suggest to subscribe to XSL-list, check the XSL-listarchives
- The XSL FAQ.
- The tutorialwritten by Paul Grosso and Norman Walsh is a very good on-lineintrodution to the language.
- The onlyZvon XSLT tutorialdetails a lot of constructs with examples.
- Jeni Tennison'sXSLTpages provide links to a lot of answers
- the Gallery ofXSLT Tricksprovides non-standard use case of XSLT
- And I suggest to buy Michael Kay "XSLT Programmer's Reference" bookpublished by Wroxif you plan to workseriously with XSLT in the future.

Well, bugs or missing features are always possible, and I will make apoint of fixing them in a timely fashion. The best way to report a bug is touse the Gnome bugtracking database(make sure to use the "libxslt" module name). Beforefiling a bug, check the list of existinglibxslt bugsto make sure it hasn't already been filed. I look at reportsthere regularly and it's good to have a reminder when a bug is still open. Besure to specify that the bug is for the package libxslt.

For small problems you can try to get help on IRC, the #xml channel onirc.gnome.org (port 6667) usually have a few person subscribed which may help(but there is no garantee and if a real issue is raised it should go on themailing-list for archival).

There is also a mailing-list xslt@gnome.orgfor libxslt, with an on-line archive. To subscribeto this list, please visit the associated Webpageand follow the instructions.

Alternatively, you can just send the bug to the xslt@gnome.orglist, if it's really libxsltrelated I will approve it.. Please do not send me mail directly especiallyfor portability problem, it makes things really harder to track and in somecases I'm not the best person to answer a given question, ask the listinstead. Do not send code, I won't debug it(but patches arereally appreciated!).

Please note that with the current amount of virus and SPAM, sending mailto the list without being subscribed won't work. There is *far too manybounces* (in the order of a thousand a day !) I cannot approve them manuallyanymore. If your mail to the list bounced waiting for administrator approval,it is LOST ! Repost it and fix the problem triggering the error. Also pleasenote that emails witha legal warning asking to not copy or redistribute freely the informationsthey containare NOTacceptable for the mailing-list,such mail will as much as possible be discarded automatically, and are lesslikely to be answered if they made it to the list, DO NOTpost to the list from an email address where such legal requirements areautomatically added, get private paying support if you can't shareinformations.

Check the following too beforeposting:

- use the search engineto get informationsrelated to your problem.
- make sure you are using a recentversion, and that the problem still shows up in those
- check the listarchivesto see if the problem was reported already, in this casethere is probably a fix available, similarly check the registeredopen bugs
- make sure you can reproduce the bug with xsltproc, a very useful thingto do is run the transformation with -v argument and redirect thestandard error to a file, then search in this file for the transformationlogs just preceding the possible problem
- Please send the command showing the error as well as the input andstylesheet (as an attachment)

Then send the bug with associated informations to reproduce it to the xslt@gnome.orglist; if it's really libxsltrelated I will approve it. Please do not send mail to me directly, it makesthings really hard to track and in some cases I am not the best person toanswer a given question, ask on the list.

To be really clear about support:

- Support or help request MUST be sent tothe list or on bugzillain case of problems, so that the Questionand Answers can be shared publicly. Failing to do so carries the implicitmessage "I want free support but I don't want to share the benefits withothers" and is not welcome. I will automatically Carbon-Copy thexslt@gnome.org mailing list for any technical reply made about libxml2 orlibxslt.
- There is no garantee for support,if your question remains unanswered after a week, repost it, making sureyou gave all the detail needed and the informations requested.
- Failing to provide informations as requested or double checking firstfor prior feedback also carries the implicit message "the time of thelibrary maintainers is less valuable than my time" and might not bewelcome.

Of course, bugs reports with a suggested patch for fixing them willprobably be processed faster.

If you're looking for help, a quick look at the list archivemay actuallyprovide the answer, I usually send source samples when answering libxsltusage questions. The auto-generated documentationisnot as polished as I would like (I need to learn more about Docbook), butit's a good starting point.

## How to help

You can help the project in various ways, the best thing to do first is tosubscribe to the mailing-list as explained before, check the archives and the Gnome bugdatabase::

1. provide patches when you find problems
2. provide the diffs when you port libxslt to a new platform. They may notbe integrated in all cases but help pinpointing portability problemsand
3. provide documentation fixes (either as patches to the code comments oras HTML diffs).
4. provide new documentations pieces (translations, examples, etc ...)
5. Check the TODO file and try to close one of the items
6. take one of the points raised in the archive or the bug database andprovide a fix. Get in touch with mebefore to avoid synchronization problems and check that the suggestedfix will fit in nicely :-)

## Downloads

The latest versions of libxslt can be found on the xmlsoft.orgserver and on mirrors (France) or on the Gnome FTP serveras asourcearchive, Antonin Sprinzl also provides a mirror in Austria. (NOTE thatyou need the libxml2,libxml2-devel,libxsltand libxslt-develpackages installed to compile applications using libxslt.) Igor Zlatkovicis now the maintainer ofthe Windows port, he providesbinaries. Gary Penningtonprovides Solaris binaries.Steve Ballprovides Mac Os Xbinaries.

Contribs:

I do accept external contributions, especially if compiling on anotherplatform, get in touch with me to upload the package. I will keep them in thecontrib directory

Libxslt is also available from CVS:

- The Gnome CVSbase. Check the Gnome CVS Toolspage; the CVS module is libxslt.

- snapshots fromCVSupdated every hour are also provided

## FAQ

1. Troubles compiling or linking programs using libxslt

   Usually the problem comes from the fact that the compiler doesn't getthe right compilation or linking flags. There is a small shell script`xslt-config`which is installed as part of libxslt usualinstall process which provides those flags. Use

   `xslt-config --cflags`

   to get the compilation flags and

   `xslt-config --libs`

   to get the linker flags. Usually this is done directly from theMakefile as:

   `CFLAGS=`xslt-config --cflags``

   `LIBS=`xslt-config --libs``

   Note also that if you use the EXSLT extensions from the program thenyou should prepend `-lexslt`to the LIBS options

2. passing parameters on the xsltproc command line doesn't work

   xsltproc --param test alpha foo.xsl foo.xml

   the param does not get passed and ends up as ""

   In a nutshell do a double escaping at the shell prompt:

   xsltproc --param test "'alpha'" foo.xsl foo.xml

   i.e. the string value is surrounded by " and ' then terminated by 'and ". Libxslt interpret the parameter values as XPath expressions, sothe string ->`alpha`<- is intepreted as the node setmatching this string. You really want ->`'alpha'`<- tobe passed to the processor. And to allow this you need to escape thequotes at the shell level using ->`"'alpha'"`<- .

   or use

   xsltproc --stringparam test alpha foo.xsl foo.xml

3. Is there C++ bindings ?

   Yes for example xmlwrapp, see the related pages about bindings

## News

The change logdescribes the recents commitsto the CVScode base.

Those are the public releases made:

### 1.1.17: Jun 6 2006

- portability fixes: python detection
- bug fixes: some regression tests, attribute/namespaces output (Kasimier Buchcik), problem in mixed xsl:value-of and xsl:text uses (Kasimier)
- improvements: internal refactoring (Kasimier Buchcik), use of the XPath object cache in libxml2-2.6.25 (Kasimier)

### 1.1.16: May 01 2006

- portability fixes: EXSLT date/time on Solaris and IRIX (Albert Chin),HP-UX build (Albert Chin),
- build fixes: Python detection(Joseph Sacco), plugin configurei(Joel Reed)
- bug fixes: pattern compilation fix(William Brack), EXSLT date/timefix (Thomas Broyer), EXSLT function bug, potential loop on variableeval, startup race (Christopher Palmer), debug statement left in python(Nic Ferrier), various cleanup based on Coverity reports), error onOut of memory condition (Charles Hardin), various namespace prefixesfixes (Kasimier Buchcik),
- improvement: speed up sortingi, start of internals refactoring (KasimierBuchcik)
- documentation: man page fixes and updates (Daniel Leidert)

### 1.1.15: Sep 04 2005

- build fixes: Windows build cleanups and updates (Igor Zlatkovic),remove jhbuild warnings
- bug fixes: negative number formatting (William Brack), numberformatting per mille definition (William Brack), XInclude default values(William), text copy bugs (William), bug related to xmlXPathContext size,reuse libxml2 memory management for text nodes, dictionnary text bug,forbid variables in match (needs libxml2-2.6.21)
- improvements: EXSLT dyn:map (Mark Vakoc),
- documentation: EXSLT date and time functions namespace in man (JonathanWakely)

### 1.1.14: Apr 02 2005

- bug fixes: text node on stylesheet document without a dictionary(William Brack), more checking of XSLT syntax, calling xsltInit()multiple times, mode values interning raised by Mark Vakoc, bug inpattern matching with ancestors, bug in patterna matching with cascadingselect, xinclude and document() problem, build outside of source tree(Mike Castle)
- improvement: added a --nodict mode to xsltproc to check problems fordocuemtns without dictionnaries

### 1.1.13: Mar 13 2005

- build fixes: 64bits cleanup (William Brack), python 2.4 test (William),LIBXSLT_VERSION_EXTRA on Windows (William), Windows makefiles fixes (JoelReed), libgcrypt-devel requires for RPM spec.
- bug fixes: exslt day-of-week-in-month (Sal Paradise), xsl:call-templateshould not change the current template rule (William Brack), evaluationof global variables (William Brack), RVT's in XPath predicates (William),namespace URI on template names (Mark Vakoc), stat() for Windows patch(Aleksey Gurtovoy), pattern expression fixes (William Brack), out ofmemory detection misses (William), parserOptions propagation (William),exclude-result-prefixes fix (William), // patten fix (William).
- extensions: module support (Joel Reed), dictionnary based speedupstrying to get rid of xmlStrEqual as much as possible.
- documentation: added Wiki (Joel Reed)

### 1.1.12: Oct 29 2004

- build fixes: warnings removal (William).
- bug fixes: attribute document pointer fix (Mark Vakoc), exslt datenegative periods (William Brack), generated tree structure fixes,namespace lookup fix, use reentrant gmtime_r (William Brack),exslt:funtion namespace fix (William), potential NULL pointer reference(Dennis Dams, William), force string interning on generateddocuments.
- documentation: update of the second tutorial (Panagiotis Louridas), addexslt doc in rpm packages, fix the xsltproc man page.

### 1.1.11: Sep 29 2004

- bug fixes: xsl:include problems (William Brack), UTF8 number pattern(William), date-time validation (William), namespace fix (William),various Exslt date fixes (William), error callback fixes, leak withnamespaced global variable, attempt to fix a weird problem #153137
- improvements: exslt:date-sum tests (Derek Poon)
- documentation: second tutorial by Panagiotis Lourida

### 1.1.10: Aug 31 2004

- build fix: NUL in c file blocking compilation on Solaris, Windows build(Igor Zlatkovic)
- fix: key initialization problem (William Brack)
- documentation: fixed missing man page description for --path

### 1.1.9: Aug 22 2004

- build fixes: missing tests (William Brack), Python dependancies, Pythonon 64bits boxes, --with-crypto flag (Rob Richards),
- fixes: RVT key handling (William), Python binding (William and SitsofeWheeler), key and XPath troubles (William), template priority on imports(William), str:tokenize with empty strings (William), #default namespacealias behaviour (William), doc ordering missing for main document(William), 64bit bug (Andreas Schwab)
- improvements: EXSLT date:sum added (Joel Reed), hook for documentloading for David Hyatt, xsltproc --nodtdattr to avoid defaulting DTDattributes, extend xsltproc --version with CVS stamp (William).
- Documentation: web page problem reported by Oliver Stoeneberg

### 1.1.8: July 5 2004

- build fixes: Windows runtime options (Oliver Stoeneberg), Windowsbinary package layout (Igor Zlatkovic), libgcrypt version test and link(William)
- documentation: fix libxslt namespace name in doc (William)
- bug fixes: undefined namespace message (William Brack), search engine(William), multiple namespace fixups (William), namespace fix for keyevaluation (William), Python memory debug bindings,
- improvements: crypto extensions for exslt (Joel Reed, William)

### 1.1.7: May 17 2004

- build fix: warning about localtime_r on Solaris
- bug fix: UTF8 string tokenize (William Brack), subtle memorycorruption, linefeed after comment at document level (William),disable-output-escaping problem (William), pattern compilation in deepimported stylesheets (William), namespace extension prefix bug,libxslt.m4 bug (Edward Rudd), namespace lookup for attribute, namespacedDOCTYPE name

### 1.1.6: Apr 18 2004

- 2 bug fixes about keys fixed one by Mark Vakoc

### 1.1.5: Mar 23 2004

- performance: use dictionnary lookup for variables
- remove use of _private from source documents
- cleanup of "make tests" output
- bugfixes: AVT in local variables, use localtime_r to avoid threadtroubles (William), dictionary handling bug (William), limited number ofstubstitutions in AVT (William), tokenize fix for UTF-8 (William),superfluous namespace (William), xsltproc error code on<xsl:message> halt, OpenVMS fix, dictionnary reference countingchange.

### 1.1.4: Feb 23 2004

- bugfixes: attributes without doc (Mariano Suárez-Alvarez), problem withYelp, extension problem
- display extension modules (Steve Little)
- Windows compilation patch (Mark Vadoc), Mingw (Mikhail Grushinskiy)

### 1.1.3: Feb 16 2004

- Rewrote the Attribute Value Template code, new XPath compilationinterfaces, dictionnary reuses for XSLT with potential for seriousperformance improvements.
- bug fixes: portability (William Brack), key() in node-set() results(William), comment before doctype (William), math and node-set() problems(William), cdata element and default namespace (William), behaviour onunknown XSLT elements (Stefan Kost), priority of "//foo" patterns(William), xsl:element and xsl:attribute QName check (William), commentswith -- (William), attribute namespace (William), check for ?> in PI(William)
- Documentations: cleanup (John Fleck and William)
- Python: patch for OS-X (Gianni Ceccarelli), enums export (Stephanebidoul)

### 1.1.2: Dec 24 2003

- Documentation fixes (John Fleck, William Brack), EXSLT documentation(William Brack)
- Windows compilation fixes for MSVC and Mingw (Igor Zlatkovic)
- Bug fixes: exslt:date returning NULL strings (William Brack),namespaces output (William Brack), key and namespace definition problem,passing options down to the document() parser, xsl:number fixes (WilliamBrack)

### 1.1.1: Dec 10 2003

- code cleanup (William Brack)
- Windows: Makefile improvements (Igor Zlatkovic)
- documentation improvements: William Brack, libexslt man page (JonathanWakely)
- param in EXSLT functions (Shaun McCance)
- XSLT debugging improvements (Mark Vakoc)
- bug fixes: number formatting (Bjorn Reese), exslt:tokenize (WilliamBrack), key selector parsing with | reported by Oleg Paraschenko,xsl:element with computed namespaces (William Brack), xslt:import/includerecursion detection (William Brack), exslt:function used in keys (WilliamBrack), bug when CDATA_SECTION are foun in the tree (William Brack),entities handling when using XInclude.

### 1.1.0: Nov 4 2003

- Removed DocBook SGML broken support
- fix xsl:key to work with PIs
- Makefile and build improvement (Graham Wilson), build cleanup (WilliamBrack), macro fix (Justin Fletcher), build outside of source tree (RoumenPetrov)
- xsltproc option display fix (Alexey Efimov), --load-trace (CrutcherDunnavant)
- Python: never use stdout for error
- extension memory error fix (Karl Eichwalder)
- header path fixes (Steve Ball)
- added saxon:line-number() to libexslt (Brett Kail)
- Fix some tortuous template problems when using predicates (WilliamBrack)
- Debugger status patch (Kasimier Buchcik)
- Use new libxml2-2.6.x APIs for faster processing
- Make sure xsl:sort is empty
- Fixed a bug in default processing of attributes
- Removes the deprecated breakpoint library
- detect invalid names on templates (William Brack)
- fix exslt:document (and similar) base handling problem

### 1.0.33: Sep 12 2003

This is a bugfix only release

- error message missing argument (William Brack)
- mode not cascaded in template fallbacks (William Brack)
- catch redefinition of parameter/variables (William Brack)
- multiple keys with same namespace name (William Brack)
- patch for compilation using MingW on Windows (Mikhail Grushinskiy)
- header export macros for Windows (Igor Zlatkovic)
- cdata-section-elements handling of namespaced names
- compilation without libxml2 XPointer support (Mark Vadoc)
- apply-templates crash (William Brack)
- bug with imported templates (William Brack)
- imported attribute-sets merging bug (DocBook) (William Brack)

### 1.0.32: Aug 9 2003

- bugfixes: xsltSaveResultToFile() python binding (Chris Jaeger), EXSLTfunction (William Brack), RVT for globals (William Brack), EXSLT date(William Brack),

  speed of large text output, xsl:copy with attributes, strip-space andnamespaces prefix, fix for --path xsltproc option, EXST:tokenize (ShaunMcCance), EXSLT:seconds (William Brack), sort with multiple keys (WilliamBrack), checking of { and } for attribute value templates (WilliamBrack)

- Python bindings for extension elements (Sean Treadway)
- EXSLT:split added (Shaun McCance)
- portability fixes for HP-UX/Solaris/IRIX (William Brack)
- doc cleanup

### 1.0.31: Jul 6 2003

- bugfixes: xsl:copy on namespace nodes, AVT for xsl:sort order, fix forthe debugger (Keith Isdale), output filename limitation, trio.h andtriodef.h added (Albert Chin), EXSLT node-set (Peter Breitenlohner),xsltChoose and whitespace (Igor Zlatkovic),

  stylesheet compilation (Igor Zlatkovic), NaN and sort (William Brack),RVT bug introduced in 1.0.30

- avoid generating &quot; (fix in libxml2-2.5.8)
- fix 64bit cleaness problem and compilation troubles introduced in1.0.30
- Windows makefile generation (Igor Zlatkovic)
- HP-UX portability fix

### 1.0.30: May 4 2003

- Fixes and new APIs to handle Result Value Trees and avoid leaks
- Fixes for: EXSLT math pow() function (Charles Bozeman), globalparameter and global variables mismatch, a segfault on patterncompilation errors, namespace copy in xsl:copy-of, python generatorproblem, OpenVMS trio update, premature call to xsltFreeStackElem (Igor),current node when templates applies to attributes

### 1.0.29: Apr 1 2003

- performance improvements especially for large flat documents
- bug fixes: Result Value Tree handling, XML IDs, keys(), extra namespacedeclarations with xsl:elements.
- portability: python and trio fixes (Albert Chin), python on Solaris(Ben Phillips)

### 1.0.28: Mar 24 2003

- fixed node() in patterns semantic.
- fixed a memory access problem in format-number()
- fixed stack overflow in recursive global variable or params
- cleaned up Result Value Tree handling, and fixed a couple of old bugsin the process

### 1.0.27: Feb 24 2003

- bug fixes: spurious xmlns:nsX="" generation, serialization bug (inlibxml2), a namespace copy problem, errors in the RPM spec prereqs
- Windows path canonicalization and document cache fix (Igor)

### 1.0.26: Feb 10 2003

- Fixed 3 serious bugs in document() and stylesheet compilation whichcould lead to a crash

### 1.0.25: Feb 5 2003

- Bug fix: double-free for standalone stylesheets introduced in 1.0.24, Csyntax pbm, 3 bugs reported by Eric van der Vlist
- Some XPath and XInclude related problems were actually fixed inlibxml2-2.5.2
- Documentation: emphasize taht --docbook is not for XML docs.

### 1.0.24: Jan 14 2003

- bug fixes: imported global varables, python bindings (Stéphane Bidoul),EXSLT memory leak (Charles Bozeman), namespace generation onxsl:attribute, space handling with imports (Daniel Stodden),extension-element-prefixes (Josh Parsons), comments within xsl:text (MattSergeant), superfluous xmlns generation, XInclude related bug fornumbering, EXSLT strings (Alexey Efimov), attribute-sets computation onimports, extension module init and shutdown callbacks not called
- HP-UX portability (Alexey Efimov), Windows makefiles (Igor and StephaneBidoul), VMS makefile updates (Craig A. Berry)
- adds xsltGetProfileInformation() (Michael Rothwell)
- fix the API generation scripts
- API to provide the sorting routines (Richard Jinks)
- added XML description of the EXSLT API
- added ESXLT URI (un)escaping (Jörg Walter)
- Some memory leaks have been found and fixed
- document() now support fragment identifiers in URIs

### 1.0.23: Nov 17 2002

- Windows build cleanup (Igor)
- Unix build and RPM packaging cleanup
- Improvement of the python bindings: extension functions and activatingEXSLT
- various bug fixes: number formatting, portability for bounded stringfunctions, CData nodes, key(), @*[...] patterns
- Documentation improvements (John Fleck)
- added libxslt.m4 (Thomas Schraitle)

### 1.0.22: Oct 18 2002

- Updates on the Windows Makefiles
- Added a security module, and a related set of new options toxsltproc
- Allowed per transformation error handler.
- Fixed a few bugs: node() semantic, URI escaping, media-type, attributelists

### 1.0.21: Sep 26 2002

- Bug fixes: match="node()", date:difference() (Igor and CharlieBozeman), disable-output-escaping
- Python bindings: style.saveResultToString() from Ralf Mattes
- Logos from Marc Liyanage
- Mem leak fix from Nathan Myers
- Makefile: DESTDIR fix from Christophe Merlet, AMD x86_64 (Mandrake),Windows (Igor), Python detection
- Documentation improvements: John Fleck

### 1.0.20: Aug 23 2002

- Windows makefile updates (Igor) and x86-64 (Frederic Crozat)
- fixed HTML meta tag saving for Mac/IE users
- possible leak patches from Nathan Myers
- try to handle document('') as best as possible depending in thecases
- Fixed the DocBook stylesheets handling problem
- Fixed a few XSLT reported errors

### 1.0.19: July 6 2002

- EXSLT: dynamic functions and date support bug fixes (Mark Vakoc)
- xsl:number fix: Richard Jinks
- xsl:format-numbers fix: Ken Neighbors
- document('') fix: bug pointed by Eric van der Vlist
- xsl:message with terminate="yes" fixes: William Brack
- xsl:sort order support added: Ken Neighbors
- a few other bug fixes, some of them requiring the latest version oflibxml2

### 1.0.18: May 27 2002

- a number of bug fixes: attributes, extra namespace declarations(DocBook), xsl:include crash (Igor), documentation (Christian Cornelssen,Charles Bozeman and Geert Kloosterman), element-available (RichardJinks)
- xsltproc can now list teh registered extensions thanks to MarkVakoc
- there is a new API to save directly to a stringxsltSaveResultToString() by Morus Walter
- specific error registration function for the python API

### 1.0.17: April 29 2002

- cleanup in code, XSLT debugger support and Makefiles for Windows byIgor
- a C++ portability fix by Mark Vakoc
- EXSLT date improvement and regression tests by Charles Bozeman
- attempt to fix a bug in xsltProcessUserParamInternal

### 1.0.16: April 15 2002

- Bug fixes: strip-space, URL in HTML output, error when xsltproc can'tsave
- portability fixes: OSF/1, IEEE on alphas, Windows, Python bindings

### 1.0.15: Mar 25 2002

- Bugfixes: XPath, python Makefile, recursive attribute sets, @foo[..]templates
- Debug of memory alocation with valgind
- serious profiling leading to significant improvement for DocBookprocessing
- revamp of the Windows build

### 1.0.14: Mar 18 2002

- Improvement in the XPath engine (libxml2-2.4.18)
- Nasty bug fix related to exslt:node-set
- Fixed the python Makefiles, cleanup of doc comments, Windowsportability fixes

### 1.0.13: Mar 8 2002

- a number of bug fixes including "namespace node have no parents"
- Improvement of the Python bindings
- Charles Bozeman provided fixes and regression tests for exslt datefunctions.

### 1.0.12: Feb 11 2002

- Fixed the makefiles especially the python module ones
- half a dozen bugs fixes including 2 old ones

### 1.0.11: Feb 8 2002

- Change of Licence to the MITLicence
- Added a beta version of the Python bindings, including support toextend the engine with functions written in Python
- A number of bug fixes
- Charlie Bozeman provided more EXSLT functions
- Portability fixes

### 1.0.10: Jan 14 2002

- Windows fixes for Win32 from Igor
- Fixed the Solaris compilation trouble (Albert)
- Documentation changes and updates: John Fleck
- Added a stringparam option to avoid escaping hell at the shelllevel
- A few bug fixes

### 1.0.9: Dec 7 2001

- Makefile patches from Peter Williams
- attempt to fix the compilation problem associated to prelinking
- obsoleted libxsltbreakpoint now deprecated and frozen to 1.0.8 API
- xsltproc return codes are now significant, John Fleck updated thedocumentation
- patch to allow as much as 40 steps in patterns (Marc Tardif), should bemade dynamic really
- fixed a bug raised by Nik Clayton when using doctypes with HTMLoutput
- patches from Keith Isdale to interface with xsltdebugger

### 1.0.8: Nov 26 2001

- fixed an annoying header problem, removed a few bugs and some codecleanup
- patches for Windows and update of Windows Makefiles by Igor
- OpenVMS port instructions from John A Fotheringham
- fixed some Makefiles annoyance and libraries prelinkinginformations

### 1.0.7: Nov 10 2001

- remove a compilation problem with LIBXSLT_PUBLIC
- Finishing the integration steps for Keith Isdale debugger
- fixes the handling of indent="no" on HTML output
- fixes on the configure script and RPM spec file

### 1.0.6: Oct 30 2001

- bug fixes on number formatting (Thomas), date/time functions (BruceMiller)
- update of the Windows Makefiles (Igor)
- fixed DOCTYPE generation rules for HTML output (me)

### 1.0.5: Oct 10 2001

- some portability fixes, including Windows makefile updates fromIgor
- fixed a dozen bugs on XSLT and EXSLT (me and Thomas Broyer)
- support for Saxon's evaluate and expressions extensions added (initialcontribution from Darren Graves)
- better handling of XPath evaluation errors

### 1.0.4: Sep 12 2001

- Documentation updates from John fleck
- bug fixes (DocBook FO generation should be fixed) and portabilityimprovements
- Thomas Broyer improved the existing EXSLT support and added String,Time and Date core functions support

### 1.0.3: Aug 23 2001

- XML Catalog support see the doc
- New NaN/Infinity floating point code
- A few bug fixes

### 1.0.2: Aug 15 2001

- lot of bug fixes, increased the testsuite
- a large chunk of EXSLT is implemented
- improvements on the extension framework
- documentation improvements
- Windows MSC projects files should be up-to-date
- handle attributes inherited from the DTD by default

### 1.0.1: July 24 2001

- initial EXSLT framework
- better error reporting
- fixed the profiler on Windows
- bug fixes

### 1.0.0: July 10 2001

- a lot of cleanup, a lot of regression tests added or fixed
- added a documentation for writingextensions
- fixed some variable evaluation problems (with William)
- added profiling of stylesheet execution accessible as the xsltproc--profile option
- fixed element-available() and the implementation of the variouschunking methods present, Norm Walsh provided a lot of feedback
- exclude-result-prefixes and namespaces output should now work asexpected
- added support of embedded stylesheet as described in section 2.7 of thespec

### 0.14.0: July 5 2001

- lot of bug fixes, and code cleanup
- completion of the little XSLT-1.0 features left unimplemented
- Added and implemented the extension API suggested by Thomas Broyer
- the Windows MSC environment should be complete
- tested and optimized with a really large document (DocBook DefinitiveGuide) libxml/libxslt should really be faster on serious workloads

### 0.13.0: June 26 2001

- lots of cleanups
- fixed a C++ compilation problem
- couple of fixes to xsltSaveTo()
- try to fix Docbook-xslt-1.4 and chunking, updated the regression testwith them
- fixed pattern compilation and priorities problems
- Patches for Windows and MSC project mostly contributed by Yon Derek
- update to the Tutorial by John Fleck
- William fixed bugs in templates and for-each functions
- added a new interface xsltRunStylesheet() for a more flexible output(incomplete), added -o option to xsltproc

### 0.12.0: June 18 2001

- fixed a dozen of bugs reported
- HTML generation should be quite better (requires libxml-2.3.11 upgradetoo)
- William fixed some problems with document()
- Fix namespace nodes selection and copy (requires libxml-2.3.11 upgradetoo)
- John Fleck added atutorial
- Fixes for namespace handling when evaluating variables
- XInclude global flag added to process XInclude on document() ifrequested
- made xsltproc --version more detailed

### 0.11.0: June 1 2001

Mostly a bug fix release.

- integration of catalogs from xsltproc
- added --version to xsltproc for bug reporting
- fixed errors when handling ID in external parsed entities
- document() should hopefully work correctly but ...
- fixed bug with PI and comments processing
- William fixed the XPath string functions when using unicode

### 0.10.0: May 19 2001

- cleanups to make stylesheet read-only (not 100% complete)
- fixed URI resolution in document()
- force all XPath expression to be compiled at stylesheet parsing time,even if unused ...
- Fixed HTML default output detection
- Fixed double attribute generation #54446
- Fixed {{ handling in attributes #54451
- More tests and speedups for DocBook document transformations
- Fixed a really bad race like bug in xsltCopyTreeList()
- added a documentation on the libxslt internals
- William Brack and Bjorn Reese improved format-number()
- Fixed multiple sort, it should really work now
- added a --docbook option for SGML DocBook input (hackish)
- a number of other bug fixes and regression test added as people weresubmitting them

### 0.9.0: May 3 2001

- lot of various bugfixes, extended the regression suite
- xsltproc should work with multiple params
- added an option to use xsltproc with HTML input
- improved the stylesheet compilation, processing of complex stylesheetsshould be faster
- using the same stylesheet for concurrent processing on multithreadedprograms should work now
- fixed another batch of namespace handling problems
- Implemented multiple level of sorting

### 0.8.0: Apr 22 2001

- fixed ansidecl.h problem
- fixed unparsed-entity-uri() and generate-id()
- sort semantic fixes and priority prob from William M. Brack
- fixed namespace handling problems in XPath expression computations(requires libxml-2.3.7)
- fixes to current() and key()
- other, smaller fixes, lots of testing with N Walsh DocBook HTMLstylesheets

### 0.7.0: Apr 10 2001

- cleanup using stricter compiler flags
- command line parameter passing
- fix to xsltApplyTemplates from William M. Brack
- added the XSLTMark in the regression tests as well as document()

### 0.6.0: Mar 22 2001

- another beta
- requires 2.3.5, which provide XPath expression compilation support
- document() extension should function properly
- fixed a number or reported bugs

### 0.5.0: Mar 10 2001

- fifth beta
- some optimization work, for the moment 2 XSLT transform cannot use thesame stylesheet at the same time (to be fixed)
- fixed problems with handling of tree results
- fixed a reported strip-spaces problem
- added more reported/fixed bugs to the test suite
- incorporated William M. Brack fix for imports and global variables aswell as patch for with-param support in apply-templates
- a bug fix on for-each

### 0.4.0: Mar 1 2001

- fourth beta test, released at the same time of libxml2-2.3.3
- bug fixes
- some optimization
- started implement extension support, not finished
- implemented but not tested multiple file output

### 0.3.0: Feb 24 2001

- third beta test, released at the same time of libxml2-2.3.2
- lot of bug fixes
- some optimization
- added DocBook XSL based testsuite

### 0.2.0: Feb 15 2001

- second beta version, released at the same time as libxml2-2.3.1
- getting close to feature completion, lot of bug fixes, some in the HTMLand XPath support of libxml
- start becoming usable for real work. This version can now regeneratethe XML 2e HTML from the original XML sources and the associatedstylesheets (in section I of the XMLREC)
- Still misses extension element/function/prefixes support. Support ofkey() and document() is not complete

### 0.1.0: Feb 8 2001

- first beta version, released at the same time as libxml2-2.3.0
- lots of bug fixes, first "testing" version, but incomplete

### 0.0.1: Jan 25 2001

- first alpha version released at the same time as libxml2-2.2.12
- Framework in place, should work on simple examples, but far from beingfeature complete

## The xsltproc tool

This program is the simplest way to use libxslt: from the command line. Itis also used for doing the regression tests of the library.

It takes as first argument the path or URL to an XSLT stylesheet, the nextarguments are filenames or URIs of the inputs to be processed. The output ofthe processing is redirected on the standard output. There is actually a fewmore options available:

    orchis:~ -> xsltproc
    Usage: xsltproc [options] stylesheet file [file ...]
       Options:
          --version or -V: show the version of libxml and libxslt used
          --verbose or -v: show logs of what's happening
          --output file or -o file: save to a given file
          --timing: display the time used
          --repeat: run the transformation 20 times
          --debug: dump the tree of the result instead
          --novalid: skip the Dtd loading phase
          --noout: do not dump the result
          --maxdepth val : increase the maximum depth
          --html: the input document is(are) an HTML file(s)
          --docbook: the input document is SGML docbook
          --param name value : pass a (parameter,value) pair
          --nonet refuse to fetch DTDs or entities over network
          --warnnet warn against fetching over the network
          --catalogs : use the catalogs from $SGML_CATALOG_FILES
          --xinclude : do XInclude processing on document intput
          --profile or --norman : dump profiling informations
    orchis:~ ->

## DocBook

DocBookis anXML/SGML vocabulary particularly well suited to books and papers aboutcomputer hardware and software.

xsltproc and libxslt are not specifically dependant on DocBook, but sincea lot of people use xsltproc and libxml2 for DocBook formatting, here are afew pointers and informations which may be helpful:

- The DocBookhomepage at Oasisyou should find pointers there on all the lastestversions of the DTDs and XSLT stylesheets
- DocBook: The Definitive Guideisthe official reference documentation for DocBook.
- DocBookOpen Repositorycontains a lot of informations about DocBook
- Bob Stayton provides a lot ofresourcesand consulting services around DocBook.
- Here is a shell scriptto generateXML Catalogs for DocBook 4.1.2 . If it can write to the /etc/xml/directory, it will set-up /etc/xml/catalog and /etc/xml/docbook based onthe resources found on the system. Otherwise it will just create~/xmlcatalog and ~/dbkxmlcatalog and doing:

  `export XMLCATALOG=$HOME/xmlcatalog`

  should allow to process DocBook documentations without requiringnetwork accesses for the DTd or stylesheets

- I have uploaded asmall tarballcontaining XML Catalogs for DocBook 4.1.2 which seemsto work fine for me too
- Informations on installing a WindowsDocBook processing setupbased on Cygwin (using the binaries from theofficial Windows port should be possible too)
- Alexander Kirillov's page on Using DocBook XML4.1.2(RPM packages)
- Tim Waugh's xmlto front-endconversion script
- Linux Documentation Project DocBook-Install-mini-HOWTO
- ScrollKeeper the open documentation cataloging project has a DocBooksection
- Dan York presentation on Publishingusing DocBook XML

Do not use the --docbook option of xsltproc to process XML DocBookdocuments, this option is only intended to provide some (limited) support ofthe SGML version of DocBook.

Points which are not DocBook specific but still worth mentionningagain:

- if you think DocBook processing time is too slow, make sure you haveXML Catalogs pointing to a local installation of the DTD of DocBook.Check the XML Catalog pageto understand more on this subject.
- before processing a new document, use the command

  `xmllint --valid --noout path_to_document`

  to make sure that your input is valid DocBook. And fixes the errorsbefore processing further. Note that XSLT processing may work correctlywith some forms of validity errors left, but in general it can givetroubles on output.

## The programming API

Okay this section is clearly incomplete. But integrating libxslt into yourapplication should be relatively easy. First check the few steps describedbelow, then for more detailed informations, look at thegenerated pagesfor the API and the sourceof libxslt/xsltproc.c and the tutorial.

Basically doing an XSLT transformation can be done in a few steps:

1. configure the parser for XSLT:

   xmlSubstituteEntitiesDefault(1);

   xmlLoadExtDtdDefaultValue = 1;

2. parse the stylesheet with xsltParseStylesheetFile()
3. parse the document with xmlParseFile()
4. apply the stylesheet using xsltApplyStylesheet()
5. save the result using xsltSaveResultToFile() if needed setxmlIndentTreeOutput to 1

Steps 2,3, and 5 will probably need to be changed depending on youprocessing needs and environment for example if reading/saving from/tomemory, or if you want to apply XInclude processing to the stylesheet orinput documents.

## Python and bindings

There is a number of language bindings and wrappers available for libxml2,the list below is not exhaustive. Please contact the xml-bindings@gnome.org(archives) inorder to get updates to this list or to discuss the specific topic of libxml2or libxslt wrappers or bindings:

- MattSergeantdevelopped XML::LibXMLand XML::LibXSLT, Perl wrappers for libxml2/libxslt as part of the AxKit XML application server
- Dave Kuhlmanprovides andearlier version of the libxml/libxslt wrappers for Python
- Petr Kozelka provides Pascal units to gluelibxml2with Kylix, Delphi and other Pascal compilers
- Wai-Sun "Squidster" Chia provides bindings for Rubyandlibxml2 bindings are also available in Ruby through the libgdome-rubymodulemaintained by Tobias Peters.
- Steve Ball and contributors maintains libxml2 and libxslt bindings forTcl
- Peter Jonesmaintains C++bindings for libxslt within xmlwrapp
- Mike Phillipsprovides a moduleusing libxsltfor PHP.
- LibxmlJisan effort to create a 100% JAXP-compatible Java wrapper for libxml2 andlibxslt as part of GNU ClasspathX project.
- Patrick McPhee provides Rexx bindings fof libxml2 and libxslt, look forRexxXML.
- Satimageprovides XMLLibosax. This is an osax for Mac OS X with a set of commands toimplement in AppleScript the XML DOM, XPATH and XSLT.

The libxslt Python module depends on the libxml2 Pythonmodule.

The distribution includes a set of Python bindings, which are garanteed tobe maintained as part of the library in the future, though the Pythoninterface have not yet reached the completeness of the C API.

Stéphane Bidoulmaintains a Windows portof the Python bindings.

Note to people interested in building bindings, the API is formalized asan XML API description filewhich allows toautomate a large part of the Python bindings, this includes functiondescriptions, enums, structures, typedefs, etc... The Python script used tobuild the bindings is python/generator.py in the source distribution.

To install the Python bindings there are 2 options:

- If you use an RPM based distribution, simply install the libxml2-pythonRPMand the libxslt-pythonRPM.
- Otherwise use the libxml2-pythonmodule distributioncorresponding to your installed version oflibxml2 and libxslt. Note that to install it you will need both libxml2and libxslt installed and run "python setup.py build install" in themodule tree.

The distribution includes a set of examples and regression tests for thepython bindings in the `python/tests`directory. Here are someexcepts from those tests:

### basic.py:

This is a basic test of XSLT interfaces: loading a stylesheet and adocument, transforming the document and saving the result.

    import libxml2
    import libxslt

    styledoc = libxml2.parseFile("test.xsl")
    style = libxslt.parseStylesheetDoc(styledoc)
    doc = libxml2.parseFile("test.xml")
    result = style.applyStylesheet(doc, None)
    style.saveResultToFilename("foo", result, 0)
    style.freeStylesheet()
    doc.freeDoc()
    result.freeDoc()

The Python module is called libxslt, you will also need the libxml2 modulefor the operations on XML trees. Let's have a look at the objects manipulatedin that example and how is the processing done:

- `styledoc`: is a libxml2 document tree. It is obtained byparsing the XML file "test.xsl" containing the stylesheet.
- `style`: this is a precompiled stylesheet ready to be usedby the following transformations (note the plural form, multipletransformations can resuse the same stylesheet).
- `doc`: this is the document to apply the transformation to.In this case it is simply generated by parsing it from a file but anyother processing is possible as long as one get a libxml2 Doc. Note thatHTML tree are suitable for XSLT processing in libxslt. This is actuallyhow this page is generated !
- `result`: this is a document generated by applying thestylesheet to the document. Note that some of the stylesheet informationsmay be related to the serialization of that document and as in thisexample a specific saveResultToFilename() method of the stylesheet shouldbe used to save it to a file (in that case to "foo").

Also note the need to explicitely deallocate documents with freeDoc()except for the stylesheet document which is freed when its compiled form isgarbage collected.

### extfunc.py:

This one is a far more complex test. It shows how to modify the behaviourof an XSLT transformation by passing parameters and how to extend the XSLTengine with functions defined in python:

    import libxml2
    import libxslt
    import string

    nodeName = None
    def f(ctx, str):
        global nodeName

        #
        # Small check to verify the context is correcly accessed
        #
        try:
            pctxt = libxslt.xpathParserContext(_obj=ctx)
            ctxt = pctxt.context()
            tctxt = ctxt.transformContext()
            nodeName = tctxt.insertNode().name
        except:
            pass

        return string.upper(str)

    libxslt.registerExtModuleFunction("foo", "http://example.com/foo", f)

This code defines and register an extension function. Note that thefunction can be bound to any name (foo) and how the binding is alsoassociated to a namespace name "http://example.com/foo". From an XSLT pointof view the function just returns an upper case version of the string passedas a parameter. But the first part of the function also read some contextualinformation from the current XSLT processing environement, in that case itlooks for the current insertion node in the resulting output (either theresulting document or the Result Value Tree being generated), and saves it toa global variable for checking that the access actually worked.

For more informations on the xpathParserContext and transformContextobjects check the libray internals description.The pctxt is actually an object from a class derived from thelibxml2.xpathParserContext() with just a couple more properties including thepossibility to look up the XSLT transformation context from the XPathcontext.

    styledoc = libxml2.parseDoc("""
    <xsl:stylesheet version='1.0'
      xmlns:xsl='http://www.w3.org/1999/XSL/Transform'
      xmlns:foo='http://example.com/foo'
      xsl:exclude-result-prefixes='foo'>

      <xsl:param name='bar'>failure</xsl:param>
      <xsl:template match='/'>
        <article><xsl:value-of select='foo:foo($bar)'/></article>
      </xsl:template>
    </xsl:stylesheet>
    """)

Here is a simple example of how to read an XML document from a pythonstring with libxml2. Note how this stylesheet:

- Uses a global parameter `bar`
- Reference the extension function f
- how the Namespace name "http://example.com/foo" has to be bound to aprefix
- how that prefix is excluded from the output
- how the function is called from the select

    style = libxslt.parseStylesheetDoc(styledoc)
    doc = libxml2.parseDoc("<doc/>")
    result = style.applyStylesheet(doc, { "bar": "'success'" })
    style.freeStylesheet()
    doc.freeDoc()

that part is identical, to the basic example except that thetransformation is passed a dictionnary of parameters. Note that the stringpassed "success" had to be quoted, otherwise it is interpreted as an XPathquery for the childs of root named "success".

    root = result.children
    if root.name != "article":
        print "Unexpected root node name"
        sys.exit(1)
    if root.content != "SUCCESS":
        print "Unexpected root node content, extension function failed"
        sys.exit(1)
    if nodeName != 'article':
        print "The function callback failed to access its context"
        sys.exit(1)

    result.freeDoc()

That part just verifies that the transformation worked, that the parametergot properly passed to the engine, that the function f() got called and thatit properly accessed the context to find the name of the insertion node.

### pyxsltproc.py:

this module is a bit too long to be described there but it is basically arewrite of the xsltproc command line interface of libxslt in Python. Itprovides nearly all the functionalities of xsltproc and can be used as a basemodule to write Python customized XSLT processors. One of the thing to noticeare:

    libxml2.lineNumbersDefault(1)
    libxml2.substituteEntitiesDefault(1)

those two calls in the main() function are needed to force the libxml2processor to generate DOM trees compliant with the XPath data model.

## Library internals

### Table of contents

- Introduction
- Basics
- Keep it simple stupid
- The libxml nodes
- The XSLT processing steps
- The XSLT stylesheet compilation
- The XSLT template compilation
- The processing itself
- XPath expressions compilation
- XPath interpretation
- Description of XPathObjects
- XPath functions
- The variables stack frame
- Extension support
- Further reading
- TODOs

### Introduction

This document describes the processing of libxslt, the XSLTC library developed for the Gnomeproject.

Note: this documentation is by definition incomplete and I am not good atspelling, grammar, so patches and suggestions are really welcome.

### Basics

XSLT is a transformation language. It takes an input document and astylesheet document and generates an output document:

Libxslt is written in C. It relies on libxml, the XML C library for Gnome, forthe following operations:

- parsing files
- building the in-memory DOM structure associated with the documentshandled
- the XPath implementation
- serializing back the result document to XML and HTML. (Text is handleddirectly.)

### Keep it simple stupid

Libxslt is not very specialized. It is built under the assumption that allnodes from the source and output document can fit in the virtual memory ofthe system. There is a big trade-off there. It is fine for reasonably sizeddocuments but may not be suitable for large sets of data. The gain is that itcan be used in a relatively versatile way. The input or output may never beserialized, but the size of documents it can handle are limited by the sizeof the memory available.

More specialized memory handling approaches are possible, like buildingthe input tree from a serialization progressively as it is consumed,factoring repetitive patterns, or even on-the-fly generation of the output asthe input is parsed but it is possible only for a limited subset of thestylesheets. In general the implementation of libxslt follows the followingpattern:

- KISS (keep it simple stupid)
- when there is a clear bottleneck optimize on top of this simpleframework and refine only as much as is needed to reach the expectedresult

The result is not that bad, clearly one can do a better job but morespecialized too. Most optimization like building the tree on-demand wouldneed serious changes to the libxml XPath framework. An easy step would be toserialize the output directly (or call a set of SAX-like output handler tokeep this a flexible interface) and hence avoid the memory consumption of theresult.

### The libxml nodes

DOM-like trees, as used and generated by libxml and libxslt, arerelatively complex. Most node types follow the given structure except a fewvariations depending on the node type:

Nodes carry a nameand the node typeindicates the kind of node it represents, the most common ones are:

- document nodes
- element nodes
- text nodes

For the XSLT processing, entity nodes should not be generated (i.e. theyshould be replaced by their content). Most nodes also contains the following"navigation" informations:

- the containing document
- the parentnode
- the first childrennode
- the lastchildren node
- the previous sibling
- the following sibling (next)

Elements nodes carries the list of attributes in the properties, anattribute itself holds the navigation pointers and the children list (theattribute value is not represented as a simple string to allow usage ofentities references).

The nspoints to the namespace declaration for thenamespace associated to the node, nsDefis the linked listof namespace declaration present on element nodes.

Most nodes also carry an _privatepointer which can beused by the application to hold specific data on this node.

### The XSLT processing steps

There are a few steps which are clearly decoupled at the interfacelevel:

1. parse the stylesheet and generate a DOM tree
2. take the stylesheet tree and build a compiled version of it (thecompilation phase)
3. take the input and generate a DOM tree
4. process the stylesheet against the input tree and generate an outputtree
5. serialize the output tree

A few things should be noted here:

- the steps 1/ 3/ and 5/ are optional
- the stylesheet obtained at 2/ can be reused by multiple processing 4/(and this should also work in threaded programs)
- the tree provided in 2/ should never be freed using xmlFreeDoc, but byfreeing the stylesheet.
- the input tree 4/ is not modified except the _private field which maybe used for labelling keys if used by the stylesheet

### The XSLT stylesheet compilation

This is the second step described. It takes a stylesheet tree, and"compiles" it. This associates to each node a structure stored in the_private field and containing information computed in the stylesheet:

One xsltStylesheet structure is generated per document parsed for thestylesheet. XSLT documents allow includes and imports of other documents,imports are stored in the importslist (hence keeping thetree hierarchy of includes which is very important for a proper XSLTprocessing model) and includes are stored in the doclistlist. An imported stylesheet has a parent link to allow browsing of thetree.

The DOM tree associated to the document is stored in doc.It is preprocessed to remove ignorable empty nodes and all the nodes in theXSLT namespace are subject to precomputing. This usually consist ofextracting all the context information from the context tree (attributes,namespaces, XPath expressions), and storing them in an xsltStylePreCompstructure associated to the _privatefield of the node.

A couple of notable exceptions to this are XSLT template nodes (more onthis later) and attribute value templates. If they are actually templates,the value cannot be computed at compilation time. (Some preprocessing couldbe done like isolation and preparsing of the XPath subexpressions but it'snot done, yet.)

The xsltStylePreComp structure also allows storing of the precompiled formof an XPath expression that can be associated to an XSLT element (more onthis later).

### The XSLT template compilation

A proper handling of templates lookup is one of the keys of fast XSLTprocessing. (Given a node in the source document this is the process offinding which templates should be applied to this node.) Libxslt follows thehint suggested in the 5.2Patternssection of the XSLT Recommendation, i.e. it doesn't evaluate itas an XPath expression but tokenizes it and compiles it as a set of rules tobe evaluated on a candidate node. There usually is an indication of the nodename in the last step of this evaluation and this is used as a key check forthe match. As a result libxslt builds a relatively more complex set ofstructures for the templates:

Let's describe a bit more closely what is built. First the xsltStylesheetstructure holds a pointer to the template hash table. All the XSLT patternscompiled in this stylesheet are indexed by the value of the the targetelement (or attribute, pi ...) name, so when a element or an attribute "foo"needs to be processed the lookup is done using the name as a key.

Each of the patterns is compiled into an xsltCompMatch structure. It holdsthe set of rules based on the tokenization of the pattern stored in reverseorder (matching is easier this way). It also holds some information about theprevious matches used to speed up the process when one iterates over a set ofsiblings. (This optimization may be defeated by trashing when runningthreaded computation, it's unclear that this is a big deal in practice.)Predicate expressions are not compiled at this stage, they may be at run-timeif needed, but in this case they are compiled as full XPath expressions (theuse of some fixed predicate can probably be optimized, they are not yet).

The xsltCompMatch are then stored in the hash table, the clash list isitself sorted by priority of the template to implement "naturally" the XSLTpriority rules.

Associated to the compiled pattern is the xsltTemplate itself containingthe information required for the processing of the pattern including, ofcourse, a pointer to the list of elements used for building the patternresult.

Last but not least a number of patterns do not fit in the hash tablebecause they are not associated to a name, this is the case for patternsapplying to the root, any element, any attributes, text nodes, pi nodes, keysetc. Those are stored independently in the stylesheet structure as separatelinked lists of xsltCompMatch.

### The processing itself

The processing is defined by the XSLT specification (the basis of thealgorithm is explained in the Introductionsection). Basically it works by taking the root of the input document andapplying the following algorithm:

1. Finding the template applying to it. This is a lookup in the templatehash table, walking the hash list until the node satisfies all the stepsof the pattern, then checking the appropriate(s) global templates to seeif there isn't a higher priority rule to apply
2. If there is no template, apply the default rule (recurse on thechildren)
3. else walk the content list of the selected templates, for each of them:
   - if the node is in the XSLT namespace then the node has a _privatefield pointing to the preprocessed values, jump to the specificcode
   - if the node is in an extension namespace, look up the associatedbehavior
   - otherwise copy the node.

   The closure is usually done through the XSLTapply-templatesconstruct recursing by applying theadequate template on the input node children or on the result of anassociated XPath selection lookup.

Note that large parts of the input tree may not be processed by a givenstylesheet and that on the opposite some may be processed multiple times.(This often is the case when a Table of Contents is built).

The module `transform.c`is the one implementing most of thislogic. xsltApplyStylesheet()is the entry point, itallocates an xsltTransformContext containing the following:

- a pointer to the stylesheet being processed
- a stack of templates
- a stack of variables and parameters
- an XPath context
- the template mode
- current document
- current input node
- current selected node list
- the current insertion points in the output document
- a couple of hash tables for extension elements and functions

Then a new document gets allocated (HTML or XML depending on the type ofoutput), the user parameters and global variables and parameters areevaluated. Then xsltProcessOneNode()which implements the1-2-3 algorithm is called on the root element of the input. Step 1/ isimplemented by calling xsltGetTemplate(), step 2/ isimplemented by xsltDefaultProcessOneNode()and step 3/ isimplemented by xsltApplyOneTemplate().

### XPath expression compilation

The XPath support is actually implemented in the libxml module (where itis reused by the XPointer implementation). XPath is a relatively classicexpression language. The only uncommon feature is that it is working on XMLtrees and hence has specific syntax and types to handle them.

XPath expressions are compiled using xmlXPathCompile().It will take an expression string in input and generate a structurecontaining the parsed expression tree, for example the expression:

    /doc/chapter[title='Introduction']

will be compiled as

    Compiled Expression : 10 elements
      SORT
        COLLECT  'child' 'name' 'node' chapter
          COLLECT  'child' 'name' 'node' doc
            ROOT
          PREDICATE
            SORT
              EQUAL =
                COLLECT  'child' 'name' 'node' title
                  NODE
                ELEM Object is a string : Introduction
                  COLLECT  'child' 'name' 'node' title
                    NODE

This can be tested using the `testXPath`command (in thelibxml codebase) using the `--tree`option.

Again, the KISS approach is used. No optimization is done. This could bean interesting thing to add. MichaelKay describesa lot of possible and interesting optimizations done inSaxon which would be possible at this level. I'm unsure they would providemuch gain since the expressions tends to be relatively simple in general andstylesheets are still hand generated. Optimizations at the interpretationsounds likely to be more efficient.

### XPath interpretation

The interpreter is implemented by xmlXPathCompiledEval()which is the front-end to xmlXPathCompOpEval()the functionimplementing the evaluation of the expression tree. This evaluation followsthe KISS approach again. It's recursive and callsxmlXPathNodeCollectAndTest()to collect nodes set whenevaluating a `COLLECT`node.

An evaluation is done within the framework of an XPath context stored inan xmlXPathContextstructure, in the framework of atransformation the context is maintained within the XSLT context. Its contentfollows the requirements from the XPath specification:

- the current document
- the current node
- a hash table of defined variables (but not used by XSLT)
- a hash table of defined functions
- the proximity position (the place of the node in the current nodelist)
- the context size (the size of the current node list)
- the array of namespace declarations in scope (there also is a namespacehash table but it is not used in the XSLT transformation).

For the purpose of XSLT an extrapointer has been addedallowing to retrieve the XSLT transformation context. When an XPathevaluation is about to be performed, an XPath parser context is allocatedcontaining and XPath object stack (this is actually an XPath evaluationcontext, this is a remain of the time where there was no separate parsing andevaluation phase in the XPath implementation). Here is an overview of the setof contexts associated to an XPath evaluation within an XSLTtransformation:

Clearly this is a bit too complex and confusing and should be refactoredat the next set of binary incompatible releases of libxml. For example thexmlXPathCtxt has a lot of unused parts and should probably be merged withxmlXPathParserCtxt.

### Description of XPath Objects

An XPath expression manipulates XPath objects. XPath defines the defaulttypes boolean, numbers, strings and node sets. XSLT adds the result treefragment type which is basically an unmodifiable node set.

Implementation-wise, libxml follows again a KISS approach, thexmlXPathObject is a structure containing a type description and the variouspossibilities. (Using an enum could have gained some bytes.) In the case ofnode sets (or result tree fragments), it points to a separate xmlNodeSetobject which contains the list of pointers to the document nodes:

The XPath API(andits 'internal'part) includes a number of functions to create, copy, compare, convert orfree XPath objects.

### XPath functions

All the XPath functions available to the interpreter are registered in thefunction hash table linked from the XPath context. They all share the samesignature:

    void xmlXPathFunc (xmlXPathParserContextPtr ctxt, int nargs);

The first argument is the XPath interpretation context, holding theinterpretation stack. The second argument defines the number of objectspassed on the stack for the function to consume (last argument is on top ofthe stack).

Basically an XPath function does the following:

- check `nargs`for proper handling of errors or functionswith variable numbers of parameters
- pop the parameters from the stack using `obj =valuePop(ctxt);`
- do the function specific computation
- push the result parameter on the stack using `valuePush(ctxt,res);`
- free up the input parameters with`xmlXPathFreeObject(obj);`
- return

Sometime the work can be done directly by modifying in-situ the top objecton the stack `ctxt->value`.

### The XSLT variables stack frame

Not to be confused with XPath object stack, this stack holds the XSLTvariables and parameters as they are defined through the recursive calls ofcall-template, apply-templates and default templates. This is used to definethe scope of variables being called.

This part seems to be the most urgent attention right now, first it isdone in a very inefficient way since the location of the variables andparameters within the stylesheet tree is still done at run time (it reallyshould be done statically at compile time), and I am still unsure that myunderstanding of the template variables and parameter scope is actuallyright.

This part of the documentation is still to be written once this part ofthe code will be stable. TODO

### Extension support

There is a separate document explaining how theextension support works.

### Further reading

Michael Kay wrote areally interesting article on Saxon internalsand the work he did onperformance issues. I wishes I had read it before starting libxslt design (Iwould probably have avoided a few mistakes and progressed faster). A lot ofthe ideas in his papers should be implemented or at least tried inlibxslt.

The libxml documentation, especially the I/O interfacesand the memory management.

### TODOs

redesign the XSLT stack frame handling. Far too much work is done atexecution time. Similarly for the attribute value templates handling, atleast the embedded subexpressions ought to be precompiled.

Allow output to be saved to a SAX like output (this notion of SAX like APIfor output should be added directly to libxml).

Implement and test some of the optimization explained by Michael Kayespecially:

- static slot allocation on the stack frame
- specific boolean interpretation of an XPath expression
- some of the sorting optimization
- Lazy evaluation of location path. (this may require more changes butsounds really interesting. XT does this too.)
- Optimization of an expression tree (This could be done as a completelyindependent module.)

Error reporting, there is a lot of case where the XSLT specificationspecify that a given construct is an error are not checked adequately bylibxslt. Basically one should do a complete pass on the XSLT spec again andadd all tests to the stylesheet compilation. Using the DTD provided in theappendix and making direct checks using the libxml validation API sounds agood idea too (though one should take care of not raising errors forelements/attributes in different namespaces).

Double check all the places where the stylesheet compiled form might bemodified at run time (extra removal of blanks nodes, hint on thexsltCompMatch).

## Writing extensions

### Table of content

- Introduction
- Basics
- Extension modules
- Registering a module
- Loading a module
- Registering an extensionfunction
- Implementing an extensionfunction
- Examples for extensionfunctions
- Registering an extensionelement
- Implementing an extensionelement
- Example for extensionelements
- The shutdown of a module
- Future work

### Introduction

This document describes the work needed to write extensions to thestandard XSLT library for use with libxslt, the XSLTC library developed for the Gnomeproject.

Before starting reading this document it is highly recommended to getfamiliar with the libxslt internals.

Note: this documentation is by definition incomplete and I am not good atspelling, grammar, so patches and suggestions are really welcome.

### Basics

The XSLT specificationprovidestwo ways to extend an XSLT engine:

- providing new extensionfunctionswhich can be called from XPath expressions
- providing new extensionelementswhich can be inserted in stylesheets

In both cases the extensions need to be associated to a new namespace,i.e. an URI used as the name for the extension's namespace (there is no needto have a resource there for this to work).

libxslt provides a few extensions itself, either in the libxslt namespace"http://xmlsoft.org/XSLT/namespace" or in namespaces for other well knownextensions provided by other XSLT processors like Saxon, Xalan or XT.

### Extension modules

Since extensions are bound to a namespace name, usually sets of extensionscoming from a given source are using the same namespace name defining inpractice a group of extensions providing elements, functions or both. Fromthe libxslt point of view those are considered as an "extension module", andmost of the APIs work at a module point of view.

Registration of new functions or elements are bound to the activation ofthe module. This is currently done by declaring the namespace as an extensionby using the attribute `extension-element-prefixes`on the`xsl:stylesheet`element.

An extension module is defined by 3 objects:

- the namespace name associated
- an initialization function
- a shutdown function

### Registering a module

Currently a libxslt module has to be compiled within the application usinglibxslt. There is no code to load dynamically shared libraries associated toa namespace (this may be added but is likely to become a portabilitynightmare).

The current way to register a module is to link the code implementing itwith the application and to call a registration function:

    int xsltRegisterExtModule(const xmlChar *URI,
                              xsltExtInitFunction initFunc,
                              xsltExtShutdownFunction shutdownFunc);

The associated header is read by:

    #include<libxslt/extensions.h>

which also defines the type for the initialization and shutdownfunctions

### Loading a module

Once the module URI has been registered and if the XSLT processor detectsthat a given stylesheet needs the functionalities of an extended module, thisone is initialized.

The xsltExtInitFunction type defines the interface for an initializationfunction:

    /**
     * xsltExtInitFunction:
     * @ctxt:  an XSLT transformation context
     * @URI:  the namespace URI for the extension
     *
     * A function called at initialization time of an XSLT
     * extension module
     *
     * Returns a pointer to the module specific data for this
     * transformation
     */
    typedef void *(*xsltExtInitFunction)(xsltTransformContextPtr ctxt,
                                         const xmlChar *URI);

There are 3 things to notice:

- The function gets passed the namespace name URI as an argument. Thisallows a single function to provide the initialization for multiplelogical modules.
- It also gets passed a transformation context. The initialization isdone at run time before any processing occurs on the stylesheet but itwill be invoked separately each time for each transformation.
- It returns a pointer. This can be used to store module specificinformation which can be retrieved later when a function or an elementfrom the extension is used. An obvious example is a connection to adatabase which should be kept and reused along with the transformation.NULL is a perfectly valid return; there is no way to indicate a failureat this level

What this function is expected to do is:

- prepare the context for this module (like opening the databaseconnection)
- register the extensions specific to this module

### Registering an extension function

There is a single call to do this registration:

    int xsltRegisterExtFunction(xsltTransformContextPtr ctxt,
                                const xmlChar *name,
                                const xmlChar *URI,
                                xmlXPathEvalFunc function);

The registration is bound to a single transformation instance referred byctxt, name is the UTF8 encoded name for the NCName of the function, and URIis the namespace name for the extension (no checking is done, a module couldregister functions or elements from a different namespace, but it is notrecommended).

### Implementing an extension function

The implementation of the function must have the signature of a libxmlXPath function:

    /**
     * xmlXPathEvalFunc:
     * @ctxt: an XPath parser context
     * @nargs: the number of arguments passed to the function
     *
     * an XPath evaluation function, the parameters are on the
     * XPath context stack
     */

    typedef void (*xmlXPathEvalFunc)(xmlXPathParserContextPtr ctxt,
                                     int nargs);

The context passed to an XPath function is not an XSLT context but an XPath context. However it is possible tofind one from the other:

- The function xsltXPathGetTransformContext provides this lookup facility:

      xsltTransformContextPtr
               xsltXPathGetTransformContext
                                (xmlXPathParserContextPtr ctxt);

- The `xmlXPathContextPtr`associated to an`xsltTransformContext`is stored in the `xpathCtxt`field.

The first thing an extension function may want to do is to check thearguments passed on the stack, the `nargs`parameter will tell howmany of them were provided on the XPath expression. The macro valuePop willextract them from the XPath stack:

    #include <libxml/xpath.h>
    #include <libxml/xpathInternals.h>

    xmlXPathObjectPtr obj = valuePop(ctxt);

Note that `ctxt`is the XPath context not the XSLT one. It isthen possible to examine the content of the value. Check the description of XPath objectsifnecessary. The following is a common sequence checking whether the argumentpassed is a string and converting it using the built-in XPath`string()`function if this is not the case:

    if (obj->type != XPATH_STRING) {
        valuePush(ctxt, obj);
        xmlXPathStringFunction(ctxt, 1);
        obj = valuePop(ctxt);
    }

Most common XPath functions are available directly at the C level and areexported either in `<libxml/xpath.h>`or in`<libxml/xpathInternals.h>`.

The extension function may also need to retrieve the data associated tothis module instance (the database connection in the previous example) thiscan be done using the xsltGetExtData:

    void * xsltGetExtData(xsltTransformContextPtr ctxt,
                          const xmlChar *URI);

Again the URI to be provided is the one which was used when registeringthe module.

Once the function finishes, don't forget to:

- push the return value on the stack using `valuePush(ctxt,obj)`
- deallocate the parameters passed to the function using`xmlXPathFreeObject(obj)`

### Examples for extension functions

The module libxslt/functions.c contains the sources of the XSLT built-infunctions, including document(), key(), generate-id(), etc. as well as a fullexample module at the end. Here is the test function implementation for thelibxslt:test function:

    /**
     * xsltExtFunctionTest:
     * @ctxt:  the XPath Parser context
     * @nargs:  the number of arguments
     *
     * function libxslt:test() for testing the extensions support.
     */
    static void
    xsltExtFunctionTest(xmlXPathParserContextPtr ctxt, int nargs)
    {
        xsltTransformContextPtr tctxt;
        void *data;

        tctxt = xsltXPathGetTransformContext(ctxt);
        if (tctxt == NULL) {
            xsltGenericError(xsltGenericErrorContext,
                "xsltExtFunctionTest: failed to get the transformation context\n");
            return;
        }
        data = xsltGetExtData(tctxt, (const xmlChar *) XSLT_DEFAULT_URL);
        if (data == NULL) {
            xsltGenericError(xsltGenericErrorContext,
                "xsltExtFunctionTest: failed to get module data\n");
            return;
        }
    #ifdef WITH_XSLT_DEBUG_FUNCTION
        xsltGenericDebug(xsltGenericDebugContext,
                         "libxslt:test() called with %d args\n", nargs);
    #endif
    }

### Registering an extension element

There is a single call to do this registration:

    int xsltRegisterExtElement(xsltTransformContextPtr ctxt,
                               const xmlChar *name,
                               const xmlChar *URI,
                               xsltTransformFunction function);

It is similar to the mechanism used to register an extension function,except that the signature of an extension element implementation isdifferent.

The registration is bound to a single transformation instance referred toby ctxt, name is the UTF8 encoded name for the NCName of the element, and URIis the namespace name for the extension (no checking is done, a module couldregister elements for a different namespace, but it is not recommended).

### Implementing an extension element

The implementation of the element must have the signature of an XSLTtransformation function:

    /**
     * xsltTransformFunction:
     * @ctxt: the XSLT transformation context
     * @node: the input node
     * @inst: the stylesheet node
     * @comp: the compiled information from the stylesheet
     *
     * signature of the function associated to elements part of the
     * stylesheet language like xsl:if or xsl:apply-templates.
     */
    typedef void (*xsltTransformFunction)
                              (xsltTransformContextPtr ctxt,
                               xmlNodePtr node,
                               xmlNodePtr inst,
                               xsltStylePreCompPtr comp);

The first argument is the XSLT transformation context. The second andthird arguments are xmlNodePtr i.e. internal memory representation of XML nodes. They arerespectively `node`from the the input document being transformedby the stylesheet and `inst`the extension element in thestylesheet. The last argument is `comp`a pointer to a precompiledrepresentation of `inst`but usually for an extension functionthis value is `NULL`by default (it could be added and associatedto the instruction in `inst->_private`).

The same functions are available from a function implementing an extensionelement as in an extension function, including`xsltGetExtData()`.

The goal of an extension element being usually to enrich the generatedoutput, it is expected that they will grow the currently generated outputtree. This can be done by grabbing ctxt->insert which is the currentlibxml node being generated (Note this can also be the intermediate valuetree being built for example to initialize a variable, the processing shouldbe similar). The functions for libxml tree manipulation from <libxml/tree.h>canbe employed to extend or modify the tree, but it is required to preserve theinsertion node and its ancestors since there are existing pointers to thoseelements still in use in the XSLT template execution stack.

### Example for extension elements

The module libxslt/transform.c contains the sources of the XSLT built-inelements, including xsl:element, xsl:attribute, xsl:if, etc. There is a smallbut full example in functions.c providing the implementation for thelibxslt:test element, it will output a comment in the result tree:

    /**
     * xsltExtElementTest:
     * @ctxt:  an XSLT processing context
     * @node:  The current node
     * @inst:  the instruction in the stylesheet
     * @comp:  precomputed informations
     *
     * Process a libxslt:test node
     */
    static void
    xsltExtElementTest(xsltTransformContextPtr ctxt, xmlNodePtr node,
                       xmlNodePtr inst,
                       xsltStylePreCompPtr comp)
    {
        xmlNodePtr comment;

        if (ctxt == NULL) {
            xsltGenericError(xsltGenericErrorContext,
                             "xsltExtElementTest: no transformation context\n");
            return;
        }
        if (node == NULL) {
            xsltGenericError(xsltGenericErrorContext,
                             "xsltExtElementTest: no current node\n");
            return;
        }
        if (inst == NULL) {
            xsltGenericError(xsltGenericErrorContext,
                             "xsltExtElementTest: no instruction\n");
            return;
        }
        if (ctxt->insert == NULL) {
            xsltGenericError(xsltGenericErrorContext,
                             "xsltExtElementTest: no insertion point\n");
            return;
        }
        comment =
            xmlNewComment((const xmlChar *)
                          "libxslt:test element test worked");
        xmlAddChild(ctxt->insert, comment);
    }

### The shutdown of a module

When the XSLT processor ends a transformation, the shutdown function (ifit exists) for each of the modules initialized is called. ThexsltExtShutdownFunction type defines the interface for a shutdownfunction:

    /**
     * xsltExtShutdownFunction:
     * @ctxt:  an XSLT transformation context
     * @URI:  the namespace URI for the extension
     * @data:  the data associated to this module
     *
     * A function called at shutdown time of an XSLT extension module
     */
    typedef void (*xsltExtShutdownFunction) (xsltTransformContextPtr ctxt,
                                             const xmlChar *URI,
                                             void *data);

This is really similar to a module initialization function except a thirdargument is passed, it's the value that was returned by the initializationfunction. This allows the routine to deallocate resources from the module forexample close the connection to the database to keep the same example.

### Future work

Well, some of the pieces missing:

- a way to load shared libraries to instantiate new modules
- a better detection of extension functions usage and their registrationwithout having to use the extension prefix which ought to be reserved toelement extensions.
- more examples
- implementations of the EXSLTcommonextension libraries, Thomas Broyer nearly finished implementing them.

## Contributions

- Bjorn Reese is the author of the number support and worked on theXSLTMark support
- William Brack was an early adopted, contributed a number of patches andspent quite some time debugging non-trivial problems in early versions oflibxslt
- Igor Zlatkovicis now themaintainer of the Windows port, he providesbinaries
- Thomas Broyer provided a lot of suggestions, and drafted most of theextension API
- John Fleck maintains a tutorialfor libxslt
- MattSergeantdeveloped XML::LibXSLT, a perl wrapper forlibxml2/libxslt as part of the AxKit XMLapplication server
- there is a module for libxml/libxslt supportin OpenNSD/AOLServer
- Dave Kuhlmanprovideslibxml/libxslt wrappers forPython
- Steve Ball, andcontributors maintain tclbindings for libxml2 and libxslt, as well as tkxmllinta GUI forxmllint and tkxsltproca GUI for xsltproc.
- If you want to use libxslt in a Mac OS X/Cocoa or Objective-Cframework, Marc Liyanage provides an applicationTestXSLT for XSLT and XML editingincluding wrapper classes for theXML parser and XSLT processor.

Daniel Veillard