In my <element/>

I was recently invited to Helsinki by Varieng to teach a workshop on TEI XML, and specifically on TEI XML concentrating on transcription. The workshop slides and materials are at http://tei.oucs.ox.ac.uk/Oxford/2010-10-helsinki/. Though these were largely based on the TEI Summer School 2010 that we taught earlier in the year. We may hopefully be partnering with Varieng to convert the Helsinki Corpus to TEI P5 XML.

I often transform from TEI XML to XHTML as part of projects, but in some instances it is more difficult to manage using things like the eXist XML Database or Apache Cocoon, or even AxKit. This is because the hosting arrangement means that only a limited number of technologies are available.

In most cases these days a linux-based server will have Apache’s http server installed, and hopefully the Apache ReWrite module installed. In addition most hosting, even shared hosting, has PHP installed with libxml for XSL processing. Sadly, this only copes with XSLT1 not XSLT2.

However, one way to use this is to have one’s .htaccess file rewrite incoming URLs to run an xml2html.php conversion.

Basic preceding stuff:

#Turn on Rewriting
RewriteEngine On
RewriteBase /
# Redirect any svn requests
RewriteRule ^.svn/(.*)$ http://subversion.tigris.org [R]
# utf-8 please
AddDefaultCharset UTF-8
# change directory index to index.xml as default
DirectoryIndex index.xml index.php index.html index.shtml
#ErrorDocuments
ErrorDocument 404 /unavailable.html
ErrorDocument 403 /forbidden.html

Here we start by turning the RewriteEngine on and setting the RewriteBase to the root of the domain. I’ve also got a RewriteRule that takes any requests for stuff in subversion directories and redirects it to the subversion site instead. (Though actually I’m thinking of having that just 404 or 403 instead.) After that we set the default character set to UTF-8 and change the default directory index file names. and specify some error documents for 404s and 403s. (These are of course actually unavailable.xml and forbidden.xml, and are transformed by the rule further down.)

After this comes the bit where the rewriting of requests for HTML files get turned into parameters on a PHP script:

# If I ask for .xhtml then give me xml2html
RewriteRule ^(.*).xhtml$ /scripts/xml2html.php?xml=../$1.xml&xsl=site.xsl&%{QUERY_STRING} [L]
# If I have asked for .html then if the .html file exists, then give it.
RewriteRule   ^(.*)\.html$              $1      [C,E=WasHTML:yes]
RewriteCond   %{REQUEST_FILENAME}.html -f
RewriteRule   ^(.*)$ $1.html [L]
# else provide XML dynamically with xml2html.php
RewriteCond   %{ENV:WasHTML}            ^yes$
RewriteCond   %{REQUEST_FILENAME}.xml -f
RewriteRule ^(.*)$ /scripts/xml2html.php?xml=../$1.xml&xsl=site.xsl&%{QUERY_STRING} [L]

The first of these says that when I ask for any url on the site ended in .xhtml then take an XML file named the same thing and transform it using the xml2html.php script and the site.xsl stylesheet both in the /scripts directory. This is just for me, so that I can force it to run the transformation if a foo.xml and foo.html exist in the same directory.

After this the next RewriteRule matches anything on the site that is asked for that ends in .html and takes the first bit of this (the path and filename). Simultaneously it uses ‘C’ to chain this with the next rule and ‘E’ to set an environmental variable ‘WasHTML’ to be ‘yes’. Then there is a Rewrite Condition testing if this filename with a .html extension exists. If so, it rewrites this to be that filename.html and ends. If not, it tests whether the environmental variable WasHTML is set to yes (because remember we’ve taken off the extension), and whether the filename we’ve asked for ending in .xml exists. If so, then it runs the script giving the filename with .xml as the xml parameter and in this case site.xsl (in the same scripts directory) as the xsl.

That .htaccess file as a whole looks like:

#Turn on Rewriting
RewriteEngine On
RewriteBase /
# Redirect any svn requests
RewriteRule ^.svn/(.*)$ http://subversion.tigris.org [R]
# utf-8 please
AddDefaultCharset UTF-8
# change directory index to index.xml as default
DirectoryIndex index.xml index.php index.html index.shtml
#ErrorDocuments
ErrorDocument 404 /unavailable.html
ErrorDocument 403 /forbidden.html
# If I ask for .xhtml then give me xml2html
RewriteRule ^(.*).xhtml$ /scripts/xml2html.php?xml=../$1.xml&xsl=site.xsl&%{QUERY_STRING} [L]
# If I have asked for .html then if the .html file exists, then give it.
RewriteRule   ^(.*)\.html$              $1      [C,E=WasHTML:yes]
RewriteCond   %{REQUEST_FILENAME}.html -f
RewriteRule   ^(.*)$ $1.html [L]
# else provide XML dynamically with xml2html.php
RewriteCond   %{ENV:WasHTML}            ^yes$
RewriteCond   %{REQUEST_FILENAME}.xml -f
RewriteRule ^(.*)$ /scripts/xml2html.php?xml=../$1.xml&xsl=site.xsl&%{QUERY_STRING} [L]

The PHP script this is using (which I borrowed from a colleague) uses the http://www.php.net/manual/en/book.xsl.php libxml based XSLT processing in PHP. It is fairly short and consists of:

<script language="php">
#Basic check for directory/site traversal
if(preg_match('/\.\.\/\.\./',$_REQUEST['xml'])) { die("invalid input"); }
if(preg_match('/http/',$_REQUEST['xml'])) { die("invalid input"); }
if(preg_match('/http/',$_REQUEST['xsl'])) { die("invalid input"); }
if(preg_match('/\.\.\//',$_REQUEST['xsl'])) { die("invalid input"); }
#load xsl document into XsltProcessor
  $xp = new XsltProcessor();
  $xsl = new DomDocument;
  $xsl->load($_REQUEST['xsl']);
  $xp->importStylesheet($xsl);
#load xml document
  $xp->setParameter( null, 'xml', $_REQUEST['xml']);
  $xml_doc = new DomDocument;
  $xml_doc->load($_REQUEST['xml']);
#Process any xincludes
  $xml_doc->xinclude();
#Transform the XML with the XSL or put out error
  if ($html = $xp->transformToXML($xml_doc)) {
      echo $html;
  } else {
      trigger_error('XSL transformation failed.', E_USER_ERROR);
  }
</script>

The first bit of this is just a security precaution against directory (or site) traversal which rejects anything that has ‘../..’ in it or ‘http’. I’m sure there are a lot better ways to do this, but just checking the xml and xsl parameters seemed the easiest. I could have made a function and then passed it to each of them, or had the regex look for either of these two things, but I think it all works out the same and doesn’t seem to have much of a speed implication. Then we start a new XsltProcessor(), and a new xsl DomDocument, we load in the xsl file given in the xsl parameter, and also pass to this the parameter ‘xml’ so that we can use this in our XSLT if we want. Then we start a new xml_doc DomDocument and load in the requested XML file, and we do any XIncludes in that XML file. We then transform the XML doc to HTML with transformToXML otherwise trigger and error and put that out.

This is a fairly lightweight way to transform XML to HTML on the fly using the technologies (PHP and .htaccess) that most hosting solutions provide. I’m using something like this on one of my personal sites and it is in use in a slightly different form in a number of work sites.

Hope it is useful to someone.

There are plenty of ways to sync one’s work (nexus, Oxford’s version of Exchange) calendar with google if you are using Windows and Outlook. However, I’m using Ubuntu Linux. The solution I’ve chosen for getting mail and shared calendaring is Thunderbird + Lightning + Davmail. This works, but had idiosyncrises such as not allowing you to share calendars (but use calendars you have shared through another method such as Outlook2007 or OWA-Messageware).

Let’s be clear here, I do not need full synchronisation. What I want to do is:

• when looking at my google apps calendars (which I intentionally separate from my work ones) I want to be able to have at least read-only view of my work calendars. Basically I want to just see them so I know that work activities are not overlapping with personal ones.
• make my calendars available read-only to specific other people who either are not inside *.ox.ac.uk or whose departments do not use calendaring aspects of nexus

The solution I’ve come up with is an ad-hoc one involving a mozilla thunderbird extension called automatic export. Once installed and the icon is added to toolbar you can select from a dropdown menu on this icon a cyclical export. I have this set to export my calendar every 10 minutes. As long as you export this to a web accessible location then google calendar can subscribe to this. In addition, I store mine on a remote server, so have a shell script that scp’s it to the correct location every 10 minutes…so at very worst it is 20 minutes out of date. On google you just subscribe to the remote .ics file… though it sometimes takes awhile for google to finally realise it is there.

Drawbacks

• The export only works when you have a copy of thunderbird that is set to do this is currently running. So, for example, TB on my laptop is not set to do this, or if I add an appointment with OWA-lite it doesn’t end up in my google calendar until I load up TB at work on Monday.
• It is fairly insecure. The entire calendar is exported as an .ics file that is world readable. While it is in a place that is fairly obscure, security by obscurity isn’t really security.
• I tried having it on a passworded WebDAV storage, but even giving google the username/password in the url, it had problems finding it.
• Private events are shared with those with whom you share the calendar… so they basically see anything you see.
• You need to have a constantly web-accessible location in which to put the calendar, exporting it to your desktop machine isn’t sufficient since google will think it has disappeared when the machine is off. (And we all hibernate our desktops and use OUCS’s Wake-On-Lan service to wake them up when needed… don’t we?)

I don’t know if this will be useful to anyone else… but that is how I export my Thunderbird+lightning+davmail Nexus Calendar to my Google Apps Calendar.

-James