What's New in XPath 2.0
This article provides a brief tour through some of the new features in XPath 2.0. It assumes that you already have a basic understanding of XPath 1.0, and that you've most likely used it in the context of XSLT. It is by no means an exhaustive overview but merely points out some of the most noteworthy features.
Relationship between XPath 1.0 and XPath 2.0
Both the XPath 1.0 recommendation and the latest XPath 2.0 working draft say that "XPath is a language for addressing parts of an XML document". This was a fairly appropriate characterization of XPath 1.0. (Of course, it doesn't mention that you can have arithmetic expressions and string, number, and boolean expressions, but those features were kept to a minimum.) On the other hand, as a characterization of XPath 2.0, it leaves a lot to be desired. XPath 2.0 is a much more powerful language that operates on a much larger domain of data types. A better way of describing XPath 2.0 is as an expression language for processing sequences, with built-in support for querying XML documents. Querying? Isn't that XQuery's job?
Relationship between XPath 2.0 and XQuery 1.0
For over a year now, the W3C XSL and XML Query Working Groups have been working closely together. The goal has been to share as much between XSLT 2.0 and XQuery 1.0 as is technically and politically feasible and to give that common subset the name "XPath 2.0". This effectively means that the driving forces behind XPath 2.0 include not only the XPath 2.0 Requirements document but also many of the XML Query language requirements.
XPath 2.0 is a strict syntactic subset of XQuery 1.0. In fact, both working drafts and language grammars were automatically generated from a common source (using XML and XSLT, of course). While it is not strictly true that one working draft is a subset of the other (because some paragraphs were devoted exclusively to XPath 2.0), it is nearly true. In any case, about 80% of the text in the XQuery draft is common to both drafts. As subsets go, the XPath 2.0 subset of XQuery 1.0 is a rather large one. The optimistic upshot is that, once you've gone through the trouble of learning XPath 2.0, you'll be pleased to discover that you're almost done learning XQuery.
In fact, the XQuery-specific features consist mostly of top-level query wrapper mechanisms, such as function definitions, namespace declarations, and schema imports, as well as element constructors. XSLT 2.0, the other primary context in which XPath 2.0 is meant to be used, doesn't need these mechanisms, since it generally provides its own versions, so they are not included in the common subset.
XML Schema support
If you recall, XPath 1.0 supported only four expression types:
- node-set
- boolean
- number (floating-point)
- string
This had the value of being simple but the disadvantage of being limited when it came to processing typed values, such as dates. XPath 2.0, on the other hand, introduces support for the XML Schema primitive types, which immediately gives the user access to 19 simple types, including dates, years, months, URIs, etc. In addition, a number of functions and operators are provided for processing and constructing these different data types. These are found in the "XQuery 1.0 and XPath 2.0 Functions and Operators" document.
For exhaustive coverage of what kinds of values XPath 2.0 expressions can return, see the "XQuery 1.0 and XPath 2.0 Data Model" document. For our purposes, suffice it to say that expressions can return simple-typed values, nodes, or sequences of nodes or simple-typed values. Actually, every expression returns a sequence, as we will see.
Nodes in XPath 2.0 have the same basic definition as in XPath 1.0, except that certain kinds (elements and attributes) can now be associated with XML Schema types and processed as such. As in XPath 1.0, there are seven node types: document nodes, elements, attributes, namespace nodes, processing instructions, comments, and text nodes. The one difference in terminology here is that "root nodes" are now called, perhaps more appropriately, "document nodes".
Sequences, sequences, sequences
As a language for processing sequences, it makes sense to talk about what XPath 2.0 thinks a sequence is and how it behaves. What follows is a set of rules that you should keep in mind. These cardinal truths about sequences are fundamental to the way that XPath 2.0 works. An understanding of them is a prerequisite to a deeper understanding and appreciation of the ways in which XPath 2.0 can be used.
Cardinal rule #1: Everything is a sequence.
If you want to impress your friends immediately, point to any given XPath 2.0 expression (or XQuery expression for that matter) and casually observe that the expression clearly returns a sequence. You don't have to spoil the fun by letting them in on the secret that all expressions in fact return sequences.
If you think about the fact that everything is a sequence, you'll realize that there is no way to make a distinction between a simple-typed value (or node) and a sequence of one simple-typed value (or node). For that reason, the XPath 2.0 Working Draft and colloquial usage in general often speak of an expression as returning a "decimal" or a "string", when in fact what is meant is "a sequence of one decimal value" or a "sequence of one string". Since there is no distinction between these two, both usages are acceptable. Just remember that it's still true that everything is a sequence.
Cardinal rule #2: Sequences are shallow.
You cannot have a sequence of sequences. If you try to nest a sequence within a sequence, which is quite possible syntactically, you'll get a "flattened" sequence with the members of the sub-sequence included alongside the members of the containing sequence.
For example, this expression,
(2, 4, (1,2,3), 6)
evaluates to exactly the same sequence as this expression,
(2, 4, 1, 2, 3, 6)
or this expression, for that matter,
( (((2))), (4,1,2,3,(6)) )
Cardinal rule #3: Sequences are ordered.
Unlike node-sets in XPath 1.0, sequences are ordered. Consider the following expression.
(/foo/bar, /foo)
As you may have gathered, the comma (,) is an operator for constructing (concatenating) sequences. By putting /foo after /foo/bar, I construct a sequence in which the bar elements come before the foo elements, regardless of the order in which they occur in the source document. Later,
we will see how XPath 2.0 sequences are able to replace XPath 1.0 node-sets, without
loss of functionality or compatibility.
Cardinal rule #4: Sequences may contain duplicates.
Also unlike XPath 1.0 node-sets (and sets in general), sequences may contain duplicates. For example, we can modify our expression above slightly:
(/foo/bar, /foo, /foo/bar)
This sequence consists of the bar element(s), followed by the foo element(s), followed again by the same bar element(s). In XPath 1.0, it was impossible to construct such a collection, because,
by definition, node-sets may not contain the same node more than once.
The rise and fall of the node-set
In XPath 1.0, if you wanted to process a collection of nodes, you had to deal with node-sets. In XPath 2.0, the concept of the node-set has been generalized and extended. As we've seen, sequences may contain simple-typed values as well as nodes. We've also seen that sequences differ from node-sets in that they are ordered and may contain duplicates. The question naturally arises: how can you do away with node-sets without breaking XPath?
How to emulate sets in a sequence-only world
Indeed, XPath 1.0 node-sets were unordered. However, in XPath's most common context, XSLT, the nodes within the node-set are always processed in some order. The default order used to process node-sets was document order (since there is a document order that is always defined for all nodes). In XSLT 2.0, the default order used to process a node collection (i.e. sequence) is not necessarily document order but, rather, the order of the sequence. To maintain backward compatibility with XPath 1.0, path expressions (and other 1.0 expressions such as union expressions) are defined to always return in document order. Specifically, whenever "/" is used in the immediate expression, you can expect the result to be in document order. In addition, duplicates are automatically removed from the result. XPath 2.0 is thus able to emulate node-sets in a sequence-only world.
If you didn't follow all of that, don't worry. You may not have even realized before now that XPath 1.0 node-sets were unordered. It's mostly for the benefit of specification writers who like to reassure ourselves that everything is consistent and well-defined. Just rest assured that sequences are in fact ordered and path expressions pretty much behave the way they used to.
Some good keywords to learn
In addition to introducing many new datatypes and functions, XPath 2.0 introduces a number of new keyword-based operators, some of which we'll look at below.
Operations on sequences
Perhaps the most powerful new operator in XPath 2.0 for processing sequences is the
for expression. It enables iteration over sequences, returning a new value for each member
in the argument sequence. This is similar to what can be done with xsl:for-each, but it is different in that it is an actual expression that returns a sequence which
can, in turn, be processed as such.
Consider the following example, which returns a sequence of simple-typed values, each consisting of the total cost of each item in a purchase order.
for $x in /order/item return $x/price * $x/quantity
We could then get the total cost of the order by using the sum() function.
sum(for $x in /order/item return $x/price * $x/quantity)
Cases such as these are much easier to solve using sequences in XPath 2.0 than they
were in XSLT/XPath 1.0. Without sequences, this problem is much harder to solve and
usually involves constructing a temporary "result tree fragment" and then using the
node-set() extension function.
Conditional expressions
Among the more powerful (and oft-requested) constructs added to XPath 2.0 is the conditional expression. Here's an example that's included in the XPath 2.0 working draft.
if ($widget1/unit-cost < $widget2/unit-cost) then $widget1 else $widget2
Quantifiers
The XPath 1.0 equals operator (=) was one of the more powerful aspects of the language. It was powerful because it
could compare node-sets. Consider the following expression.
/students/student/name = "Fred"
In XPath 1.0, this expression returns true if any student name is equal to "Fred". This might be called existential quantification because it tests for the existence of a member satisfying some condition. XPath 2.0 preserves this functionality but also provides a more explicit way of testing:
some $x in /students/student/name satisfies $x = "Fred"
This formulation is more powerful because you can replace the $x = "Fred" with any comparison you want, not just equality comparisons. Also, XPath 1.0 does
not provide a way for testing to see if every student is named "Fred". XPath 2.0 introduces this ability to do universal quantification, using a similar syntax to the above.
every $x in /students/student/name satisfies $x = "Fred"
Intersections, differences, unions
In XPath 1.0, the only real set operator was the union operator (|). This meant that it was very awkward to determine whether a given node was in a
given node-set. For example, to determine whether the node $x is included in the /foo/bar node-set, we'd have to write something like
/foo/bar[generate-id(.)=generate-id($x)]
or like
count(/foo/bar)=count(/foo/bar | $x)
XPath 2.0's introduction of the intersect operator alleviates some of the pain. Instead of going through the above gyrations,
we can simply write
$x intersect /foo/bar
XPath 2.0 also introduces the except operator, which can be very handy when we need to select all of a given node-set,
except for certain nodes. In XPath 1.0, if we wanted to, for example, select all attributes
except for the one with a given namespace-qualified name, we'd have to write
@*[not(namespace-uri()='http://example.com' and local-name()='foo')]
or
@*[not(generate-id(.)=generate-id(../@exc:foo)]
Once again, XPath 2.0 comes to our rescue with the following pleasant alternative:
@* except @exc:foo
Worrying about data types
If you take a peek at the XPath 2.0 spec, you'll see that I've left out a lot of keywords,
including things like cast, treat, assert, and instance of. These are important parts of the language, but their importance partially depends
on which context you're using XPath 2.0 in. If you will be using XPath in the context
of XSLT 2.0, you may not need to use these every day. You certainly will want to use
them in certain cases (for example, when casting a string to a date), but you won't
be required to use them. In the context of XQuery 1.0, however, you may need to become
intimately familiar with them.
The reason is that XQuery 1.0 is designed to be a statically typed language. Query analysis and optimization are aided by knowledge about what datatypes query expressions will be returning before the query is ever executed. This is only possible if the user explicitly specifies what type each of her expressions are to return. The other advantage of this approach is that errors can be caught early, thereby helping to enforce the correctness of queries.
There is certainly a tradeoff between usability and type safety. To serve the needs of both communities (sometimes artificially divided into the document-oriented and data-oriented worlds), XPath 2.0 provides a means by which the context can decide where it would like to stand in this tradeoff. Effectively, XPath 2.0 can be parameterized by its context. This may sound like a recipe for non-interoperability. However, it is important to identify the guiding principle behind the approach that has been taken. The principle is that any XPath 2.0 expression that does not first return an error will always return the same result as in another context. Thus, while an expression in one context may produce an error and not in another, it will never produce two different expression results. In other words, you always get either a right answer or an error. There is never more than one right answer.
The intended upshot for XSLT users is that they won't have to worry about most of this stuff, most of the time. A given XPath 2.0 expression may throw an "exception" in the XQuery context, but the same expression results in a silently invoked fallback conversion when in the context of XSLT.
Conclusion
It will likely become clear that XPath 2.0 represents a very significant upgrade to XPath 1.0. Its growth has been driven both by the demands of the XPath 1.0 user community, as well as the requirements for XQuery 1.0. Even if you don't agree with the entire outcome, it's hard to deny that it represents a remarkable collaboration. With any luck, it will also represent a very powerful, standard tool for several user communities.
See also: What's New in XSLT 2.0