xmlroff-testing Package

xmlroff testing uses:

The test cases have not yet been set up on the SourceForge system.

The XSL Working Group (WG) had to demonstrate that every feature of XSL had been implemented before XSL 1.0 advanced from Candidate Recommendation (CR) to Proposed Recommendation (PR).

In 2001, the WG and NIST (National Institute of Science and Technology in the U.S.) developed a DTD for both test suite descriptions and test suite results, and NIST and the XSL formatter implementors contributed both test suites and their descriptions.

When working on the XSL 1.0 CR test suite results with Max Froumentin of the W3C, we developed a simple, well-formed document type for referring to multiple test suite descriptions.

The XSL 1.0 CR test suite was put on the web. NIST also continued to develop tests.

The current test cases used for testing xmlroff are the XSL 1.0 CR test cases, an interim release of NIST's later test cases, and some test cases developed at Sun.

This is run manually whenever the test cases are modified.

Usage:

   xsltproc --stringparam testsuites \
   ../../TestSuite/coverage/testsuites.xml test2sh.xsl test2sh.xsl

The testsuites parameter is the filename of the XML document referring to multiple test suite descriptions.

The stylesheet expects that the current directory will have a subdirectory with the same name as the subdirectory of each of the test suite description files referred to by the file referred to by the 'testsuites' parameter. I.e., if testsuites.xml refers to ../NIST/testsuite.xml, then test2sh.xsl expects to be able to write to a NIST subdirectory of the current directory.

In each subdirectory corresponding to the subdirectories of the referred-to test suite descriptions, test2sh.xsl writes a test.sh file containing the shell commands to run xmlroff on each test case described in the test suite description.

For each test case, test2sh.xsl writes shell commands to:

The test.sh file also runs a simple command-line Perl script on the error files to remove process IDs from the error messages in the .err files so the files can be meaningfully diffed.

This Perl script is run nightly as part of a cron job.

It executes the shell scripts created by test2sh.xsl.

This script is run from the directory containing the subdirectories with the test.sh shell scripts.

The Perl script scans the subdirectories of the current directory for test.sh files.

In each subdirectory containing a test.sh file, the Perl script:

This Perl script is run after dotests.pl.

It generates an XML file listing file names and file sizes of test result files that is used as input to the XSLT stylesheet that generates the HTML reports. Using this script saves having to worry about trying to find and use XSLT processor-specific extensions for accessing file details.

testsucess.pl scans the subdirectories containing the test results and outputs a well-formed XML document listing the file names and file sizes for all the files relating to each PDF file.

This is the record of xmlroff's results for each test. It conforms to the DTD from the XSL WG.

This file should have a new entry added to it for every new test case added to the test suite.

This is run after testsuccess.pl has generated the XML file listing the test result file names and file sizes.

It generates HTML reports: one large report summarizing all the test cases and their results, plus an individual report for each test case.

Usage:

   xsltproc -o testsuccess.html --novalid \
   --stringparam testsuites /usr/local/src/TestSuite/coverage/testsuites.xml \
   --stringparam success testsuccess.xml \
   testsuccess.xsl testresults.xml

The summary report and the individual reports include information from both the test case description and the xmlroff test results.

The test case description information is obtained by following file references in the file referred to by the testsuites parameter to individual test suite decription files.

The xmlroff test results information is obtained from both testresults.xml and testsuccess.xsl.

The summary report is a large table with one row for each test case. Each row contains:

The individual reports contain:

Selecting the 'Update' button in the 'Result' form both updates the test result information in testresults.xml and copies the current PNG files to the ref subdirectory. Unfortunately, it does not update the form because the current XSLT stylesheet generates both the summary report and all the individual reports, all of which takes a long time. Ideally the summary report and the individual reports should be XHTML that can be simply modified, not regenerated, whenever a test's results are updated.

The summary report should also be broken into multiple HTML files, one or more for each test suite, but that also hasn't been done yet.

update-results.pl is the CGI script called when the 'Update' button in an individual report is selected. It modifies testresults.xml using the update-result.xsl stylesheet, copies the current test result's PNG files to its ref subdirectory, zeroes the corresponding files in the diff subdirectory, and removes the corresponding files in the stereo subdirectory.