Pivot Collections


Part 1: Introduction

Part 2: SparqlCxml

Part 3: SparqlCxml Deep Zoom Collections

Part 4: HtmlPivotViewer

Part 5: Importing CXML

Part 6: Facet Pivot Bridge

Part 7: DETs : Persisting SPARQL Query Results to DAV

Part 8: Frequently Asked Questions (FAQs)

Part 9: Glossary


The HtmlPivotViewer is a jQuery plugin that is a cross-browser, cross-platform version of Microsoft's Silverlight PivotViewer control. The control provides an intuitive interface that makes it easy to view, sort and filter huge datasets on the web. HtmlPivotViewer lets you view thousands of items at once and visualize them in ways that expose relationships, groupings or trends in the data. HtmlPivotViewer's powerful sorting and filtering make it simple to drill down into a large dataset to an item of interest with specific features. This functionality makes it a powerful tool in application domains such as business intelligence or deep comparison shopping. The HtmlPivotViewer sources can be downloaded from github or a built version, including a VAD package that can be installed in Virtuoso, is available for download as either a zip or a compressed tar archive.

Central to HtmlPivotViewer is the notion of a Collection. Collections combine large groups of similar items on the internet, allowing the web to be viewed as a web of data rather than a series of isolated pages. Collections are composed of two parts:

Conceptually, a collection is just like any other web content. There?s a set of files on a server, and a local client that knows how to display them. In the current web, the files are traditionally HTML and images. In the collection case, the files are CXML and Deep Zoom-formatted (DZC) images. When the user views the collection from a web browser, a web page hosting the HtmlPivotViewer Control is used to display the files.

Creating Collections

Creating a collection has four distinct steps:

  1. Pick your data - First, pick a set of data to turn into a collection.
  2. Create XML and images - Describe the data in Collection XML (CXML) and transform the images to the Deep Zoom format.
  3. Host the collection - To share your collection with others, host it on a web server.
  4. View it - View the collection in a browser by downloading a web page which hosts the HtmlPivotViewer control.

Virtuoso Support for Pivot Collections

Virtuoso Universal Application Server is an ideal platform for creating and hosting Pivot collections, combining amongst other things a powerful hybrid RDF/SQL/XML data store and HTTP server, which together support each of the collection creation steps outlined above. Virtuoso provides the ability to generate and view dynamic collections created directly from RDF data, with automatic Deep Zoom image generation, using custom or default images, and the option of including QR codes (matrix bar codes). The source RDF data may be hosted directly in Virtuoso, or extracted from external RDF and non-RDF data sources using the Virtuoso Sponger, or from SQL data using Linked Data Views or SPASQL.

Though designed from the outset as a viewer for web data, the reach and utility of HtmlPivotViewer are enhanced greatly by driving it using RDF as the "data substrate", the same standards-based data format used by the Data Web and the basis of numerous massive open datasets hosted in the Linked Open Data cloud.

To provide this RDF underpinning, the main components of Virtuoso's CXML offering comprise:


Combining A Human UI with Machine Readable Data

Part of what makes HtmlPivotViewer compelling as a viewer for the Data Web is its use of images in place of raw URIs. The RDF browsers available to date typically present little more than lists of entity URIs. In contrast, HtmlPivotViewer uses an image, rather than an entity URI, as the primary UI representation of the entity, making for a much more intuitive user experience. While the image tiles provide visual cues for human interaction, the underlying machine readable RDF data remains accessible via hyperlinks.

To further support this interaction paradigm, Virtuoso's CXML support includes other refinements not evident in typical Pivot collections. These include:

The screenshot below shows a Virtuoso generated collection displayed by the Virtuoso HtmlPivotViewer, a small VSP-based (Virtuoso Server Pages) application hosting the HtmlPivotViewer control. The screenshot illustrates the use of a silhouette icon for members of the Frankfurt Semantic Web Meetup group who do not have a foaf:image.

The source data for the above collection was supplied by a SPARQL DESCRIBE query:

describe ?s from <http://www.meetup.com/Frankfurt-Semantic-Web-Meetup/members/> where {?s a foaf:Person} limit 2000

submitted to the SPARQL endpoint at http://linkeddata.uriburner.com/sparql via the HtmlPivotViewer at http://linkeddata.uriburner.com/HtmlPivotViewer/. The full request URI being:


A Viewer for Hypermedia-based Structured Data

While at first glance HtmlPivotViewer might be seen as a viewer for RDF data, the utility of HtmlPivotViewer and Virtuoso's SparqlCxml subsystem extends beyond RDF. While intended to be a general method of representing information, RDF is nonetheless a form of EAV (Entity-Attribute-Value) model where data is overtly defined and constrained by a conceptual schema that's based on first-order logic. Thus, HtmlPivotViewer should be seen in the broader context as a viewer for EAV model data at intranet or interweb scales. The EAV model coupled with the use of dereferenceable identifiers in the entity, attribute or value slots enables the use of hypermedia resources for structured data representation and interchange, more commonly known as Linked Data. It's extremely important to note that Linked Data is not (in any way) inextricably bound to RDF.

In this broader (format agnostic) definition of Linked Data, data definition and description is expressed using EAV tuples where each of the following are uniquely discernible:

Each tuple is a proposition (due to its first-order logic grounding) as well as a database record. Each entity (object) has a dereferenceable ID, a hypermedia link, that resolves to the address of its representation. This representation is negotiable, courtesy of HTTP content-negotiation.

It is important to recognize that RDF defines a data model and Subject-Predicate-Object (or Entity-Attribute-Value) 3-tuples (triples) statements (proposition oriented records) that can be expressed in a wide variety of syntaxes that includes: RDF/XML, Turtle, N3, N-Triples, HTML+RDFa etc. At the model level its only tangible distinction from EAV is its overtness with regards to first-order logic foundation. At the data expression level SPO or EAV triples are indistinguishable bar RDF's emphasis on URI/IRI style identifiers in the Subject, Predicate, and Object (optionally) slots.

In the context of HtmlPivotViewer as a Linked Data explorer, the CXML data format provides a generalized means of describing Linked Data/EAV data for display, while SPARQL provides a query language for specifying the Linked Data to be contained in the dynamic collection being generated. The Linked Data itself can comes from many sources. Virtuoso can ingest data from a plethora of data sources associated with a wide variety of formats that includes RDF, XML (including Web Services), SQL, various binary formats (iCalendar, vCard, etc.).

HtmlPivotviewer, together with SparqlCxml, provides:

Configuring Virtuoso's CXML Support

Support for CXML is available in the Virtuoso Commercial Edition Release 6.02.3129 or above. It is not included in Virtuoso Open-Source Edition. To enable the CXML functionality, the necessary VADs (Virtuoso Application Distribution) must be installed. These are:

and optionally:

Any VAD listed under 'Requires' must be installed before the VAD under which it appears.

In addition, to support Deep Zoom image and QRcode generation, the Virtuoso ImageMagick and QRcode plugins must be installed. A Virtuoso distribution includes both the ImageMagick plugin (im.so or im.dll) and QRcode plugin (qrcode.so or qrcode.dll) as standard. The plugins should be enabled in the Plugins section of your virtuoso.ini file, e.g.:

Load6=plain, im
Load7=plain, qrcode

The supporting ImageMagick or QRcode libraries used by these plugins are statically linked, so there is no need to install them separately.

Finally, ensure that the DefaultHost parameter under the URIQA section of your virtuoso.ini file is set appropriately for the host. If you anticipate viewing collections served by a host other than the one hosting HtmlPivotViewer, cross-origin resource sharing must be enabled as detailed in Part 4: HtmlPivotViewer, section "Cross-Origin Collection Browsing".

Verifying your installation


To confirm that the ImageMagick plugin is working, execute the following query through Conductor's ISQL interface, and check that the return value is 3.

select "IM XYtoMorton"(1,1)

CXML Generation from /sparql

The sparql_cxml VAD creates a small amount of test data for verifying the installation. In Conductor's /sparql interface (e.g., http://localhost:8890/sparql), enter the query below and select 'CXML' from the 'Format Results As' dropdown before running it.

prefix ski: <http://www.openlinksw.com/ski_resorts/schema#>
prefix camp: <http://www.openlinksw.com/campsites/schema#>

describe ?s where {{ ?s a ski:SkiResort . } union { ?s a camp:Campsite . }}

Your browser should display the generated CXML file. Copy the URL from your browser's address bar, open a browser window to /HtmlPivotViewer (e.g., http://localhost:8890/HtmlPivotViewer/), and paste the SPARQL query URL into the Pivot collection URL textbox. After clicking Submit, you should see a collection view similar to that below:

Click to enlarge


A Virtuoso /sparql endpoint can render a SPARQL query result in numerous serialization formats, one of which is CXML. The query driving the CXML generation may be a SELECT, DESCRIBE, or CONSTRUCT. The CXML generation process essentially consists of mapping the entities, attributes, and values contained in the SPARQL result set to CXML Items, Facets, and Facet values. SparqlCxml allows a high degree of control over the SPARQL to CXML mapping through the use of reserved query variables and a configuration graph, virtcxml.

HtmlPivotViewer & Collection XML - Key Parts

Before describing the Virtuoso RDF to Pivot interface, it's worthwhile to identify the main components of the HtmlPivotViewer UI and Collection XML to establish the terminology for the remainder of the text.

HtmlPivotViewer UI

The main parts of the HtmlPivotViewer UI are identified below:

Collection XML

The following sample XML specifies a simple collection with only one item.

    <CollectionName="Hello World Collection"SchemaVersion="1.0" 
            <FacetCategory Name="Hello World Facet Category One" Type="String"/>
        <Items ImgBase="helloworld.dzc">
            <Item Img="#0" Id="0" Href="http://www.getpivot.com"Name="Hello World!">
                <Description>This is the only item in the collection.</Description>
                    <Facet Name="Hello World Facet Category One">
                        <String Value="Hello World Facet Value"/>

Each part of the XML is surfaced in the HtmlPivotViewer as depicted below:

SPARQL Endpoint CXML Extensions

The Virtuoso SPARQL protocol endpoint /sparql supports a number of extensions to the SPARQL standard specifically for CXML.

"text/cxml" and "text/cxml+qrcode" have been added to the list of formats recognized by the query string 'format' parameter. The "text/cxml" output format is available for SELECT, DESCRIBE, and CONSTRUCT. "text/cxml+qrcode" is available only for DESCRIBE and CONSTRUCT. e.g.


When using the /sparql UI, these format options are activated by selecting either CXML (Pivot Collection) or CXML (Pivot Collection with QRcodes) from the "Format Results As" dropdown. For DESCRIBE and CONSTRUCT, the "text/cxml+qrcode" output option adds a QRcode adjacent to the custom or default image for each item. For SELECT, the addition or omission of a QRcode alongside the item image is controlled through the ?qrcode reserved query variable.

Corresponding pragmas define output:format "CXML" and define output:format "CXML;QRCODE" are available when executing SPARQL queries directly from Virtuoso PL or from a Virtuoso client, e.g. --

sparql define output:format "CXML"
 { ?href a foaf:Person; 
   foaf:name ?name; 
   foaf:depiction ?image

RDF to CXML Mapping - The Basic Approach

The way in which SPARQL result sets are converted to CXML differs for SELECTs and DESCRIBEs or CONSTRUCTs. The broad approach is outlined below to explain the default mapping. Options for tailoring the default mapping are described in subsequent sections.


Each distinct subject URI forms a CXML Item whose Name and Href are the subject URI. Each variable in the query which is not a subject URI becomes a facet named virtcxml:Facetvariable; where the variable name is converted to title case. The variable values become the values of the corresponding facet. The default type of each facet is CXML String. A query such as:

prefix ski: <http://www.openlinksw.com/ski_resorts/schema#>
prefix camp: <http://www.openlinksw.com/campsites/schema#>

select * where 
  { ?s a ski:SkiResort . ?s ?p ?o . optional { ?s foaf:depiction ?image } }
  { ?s a camp:Campsite . ?s ?p ?o . optional { ?s foaf:depiction ?image } } 

creates a collection with facet categories virtcxml:FacetP and virtcxml:FacetO, as shown in the CXML document tree fragment below. ?image is one of several special variables reserved for controlling the RDF to CXML mapping. These reserved variables are not used as facets.

The default facet names generated by a SELECT, in this example virtcxml:FacetP and virtcxml:FacetO, can be changed through graph virtcxml: as described below.


Like SELECT, each distinct subject URI again forms a CXML Item whose Name and Href are the subject URI. Unlike SELECT, no other query variables play a part in determining the facet categories. Instead they are derived directly from the DESCRIBE or CONSTRUCT output, whereby each distinct predicate, i.e. property, becomes a facet category. Each predicate value becomes a facet value. The full set of facet categories is a union of the properties of each entity type in the result set. Again, the default type of each facet is CXML String. The query:

describe ?s where {{ ?s a ski:SkiResort } union { ?s a camp:Campsite . }}

creates the facet categories shown below. Notice that the reserved variable ?image is not used to identify which entity property will provide the Deep Zoom images. Only SELECTs use reserved variables to tailor the CXML mapping. DESCRIBE and CONSTRUCT instead use entries in graph virtcxml: to both identify which entity properties supply particular CXML names and attributes and, if automatic facet type detection isn't possible, to override the default facet type of CXML String.

Default Settings

The default settings for the CXML generation include the following defaults:

Auto-Detection of Facet Type

RDF typed literals are automatically mapped to the appropriate CXML type. The xsd:integer, xsd:float, xsd:double, and xsd:decimal types are mapped to CXML Number, xsd:datetime is mapped to CXML DateTime and strings longer than 100 characters are mapped to CXML LongString. e.g.

:Le_Jard oplcamp:last_updated "2010-12-01T00:00:00Z"^^xsd:dateTime .
:La_Plagne oplski:expert_slopes "18"^^xsd:integer .

Properties of type CXML Number can be filtered using the HtmlPivotViewer "slider filter" (shown below associated with the ski_resort:expert_slopes property/facet) to restrict the displayed items to those within a certain range of the property.

RDF values which are URIs are handled as a special case. They may be mapped to a CXML Link, String or LongString depending on the SparqlCxml link typing options in force. These are described in section Link Typing

SparqlCxml Reserved Query Variables

SparqlCxml defines a list of reserved variables for use in SPARQL SELECTs to identify which predicates are to be used to set particular CXML element and attribute values. These reserved variables, or identifiers, are defined in graph virtcxml: which is created when the sparql_cxml VAD is installed. Each identifier is prefixed by virtcxml:Facet. The table below lists the virtcxml:Facetxxx Facet descriptors defined in virtcxml:, the reserved variable used to denote each Facet descriptor in a SELECT and the corresponding CXML element or attribute whose value is set by the reserved variable.
virtcxml Entity Reserved SELECT variable CXML Element/Attribute Description
virtcxml:FacetCopyright ?copyright Item/d1p1:Copyright A facet for a copyright link about the item
virtcxml:FacetDescription ?description Item/Description A facet for a text description of an item to be displayed in the info panel
virtcxml:FacetHref ?href Item/@Href A facet for an URL associated with an item. Double-clicking the item in the Pivot client will navigate the user to this URL
virtcxml:FacetId ?id Item/@Id A facet for an unique id of an item in the item list. If not specified the decimal representation of serial is used
virtcxml:FacetImage ?image Item/@Img A facet for an absolute URI of the image that will be added to an automatically created single-use collection
virtcxml:FacetImg ?img Item/@Img A facet for a relative image URI (for use with a third party Deep Zoom Collection)
virtcxml:FacetItemtype ?itemtype - A facet for an item type that is used, for example, to choose the fallback icon for the item
virtcxml:FacetName ?name Item/@Name A facet for a name of the item which will appear at the top of the info panel
virtcxml:FacetQrcode ?qrcode Item/@Img A facet for QRcode data that should be placed in the image
virtcxml:FacetRelated ?related Item/d1p1:Related A facet for links to web-pages or collections related to the item

The reserved variables are only used for a SELECT, not a DESCRIBE or CONSTRUCT. The last two query forms instead use graph virtcxml: to control the CXML mapping. For the moment, we'll confine our attention to SELECTs.

The following example, which selects a number of products (or services), illustrates the use of the reserved ?name and ?image variables to identify which predicates, i.e. product attributes, will act as a product's name and image - in this case rdfs:label and foaf:depiction respectively.

select distinct * where { ?s a gr:ProductOrServiceModel; foaf:depiction ?image; rdfs:label ?name } limit 500

The prefix gr: identifies the GoodRelations namespace, http://purl.org/goodrelations/v1#. (An explicit prefix declaration is not needed because this is a well-known namespace predefined in Virtuoso in table DB.DBA.SYS_XML_PERSISTENT_NS_DECL.)


The queries below further illustrate the use of the reserved query variables. All can be executed against http://uriburner.com/sparql. A snippet of the generated CXML is shown to highlight how an element or attribute is influenced by the variable(s). Refer to the graphic presented earlier for details of how different CXML elements are surfaced in HtmlPivotViewer.

Example 1:

define input:inference "virtrdf-label"
prefix geosp: <http://rdf.geospecies.org/ont/geospecies#>

select distinct * 
where { ?s a <http://lod.taxonconcept.org/ontology/txn.owl#GeoSpeciesTaxon> ; 
foaf:depiction ?image ; 
virtrdf:label ?name ; 
geosp:hasCommonName ?cn ; 
geosp:hasClassName "Insecta" .
} limit 2000

View Pivot collection

Example 2:

define input:inference "virtrdf-label"
select * where { ?s a <http://purl.org/ontology/wo/Species> ;
<http://purl.org/ontology/wo/order> <http://www.bbc.co.uk/nature/order/Primate#order> ;
<http://purl.org/ontology/wo/family> ?family ;
<http://purl.org/ontology/wo/livesIn> ?lives_in ;
foaf:depiction ?image; virtrdf:label ?name; dcterms:description ?description. ?href foaf:primaryTopic ?s .
} limit 2000

View Pivot collection

Example 3:
define input:inference "virtrdf-label"
prefix geosp: <http://rdf.geospecies.org/ont/geospecies#>

select distinct * 
{ ?s a <http://lod.taxonconcept.org/ontology/txn.owl#GeoSpeciesTaxon> ; 
foaf:depiction ?image ; 
virtrdf:label ?name ; 
foaf:isPrimaryTopicOf ?href ; 
rdfs:seeAlso ?related ; 
geosp:hasCommonName "Tiger" ; 
geosp:hasClassName ?className . } 
limit 3000

View Pivot collection

Example 4:
prefix mtp: <http://www.openlinksw.com/schemas/meetup/>
prefix dc: <http://purl.org/dc/elements/1.1/>

select * from <http://www.meetup.com/semweb-25/members/>  where  {
?href ?p ?o ; foaf:depiction ?image; foaf:name ?name; mtp:id ?id; dc:description ?description .filter (!isBlank(?description)) 
} limit 200

View Pivot collection

Example 5:
PREFIX wo: <http://purl.org/ontology/wo/>

SELECT DISTINCT  ?s as ?href ?name ?kname ?image ?hname ?description 
WHERE {?s wo:kingdom ?kingdom; 
dcterms:description ?description; 
rdfs:label ?name; 
wo:livesIn ?habitat.  
?kingdom rdfs:label ?kname. 
?habitat rdfs:label ?hname 
OPTIONAL {?s foaf:depiction ?image.} 
} limit 2500

View Pivot collection

Example 6:
# Capture subject URI in QR code
# Use foaf:name as anchor text and subject URI as href of main item link in PivotViewer properties panel
# Use foaf:depiction image for Tiles (DZI) for DZC

                ?href as ?qrcode
                ?name ?page
                ?description ?created ?modified
                ?country ?postcode ?region  ?locality ?latitude ?longitude
       ?href a foaf:Person;
           foaf:depiction ?image; foaf:name ?name; foaf:homepage ?page;
           dc:description ?description; dcterms:created ?created; dcterms:modified ?modified;
           vcard:Country ?country; vcard:Pcode ?postcode; vcard:Region ?region; vcard:Locality ?locality;
           geo:lat ?latitude; geo:long ?longitude.      

Example 7:

The next example shows the use of ?image to identify the custom image to be displayed for an item, and ?qrcode to identify the property (in this case the item URI) to be encoded in the QRcode adjacent to the main image. It also illustrates the use of Virtuoso's federated SPARQL capability (SPARQL-FED) via the service keyword.

prefix gr: <http://purl.org/goodrelations/v1#>
prefix icecat: <http://linkedopencommerce.com/schemas/icecat/v1/>

select distinct ?href ?href as ?qrcode ?name ?description ?manufacturer ?image where { service <http://linkeddata.uriburner.com/sparql> {
  ?href a gr:ProductOrServiceModel . 
  ?href icecat:hasModelName ?name .
  ?href icecat:hasLongSummaryDescription ?description .
  ?href gr:hasManufacturer ?manufacturer .
  ?href ?p ?o .
  ?href foaf:depiction ?image .
  ?o bif:contains '(SONY AND CAMCORDER)' . 

The following image slice taken from the resulting HtmlPivotViewer display shows product images with their generated QRcodes.

Example 8:

If no custom image can be found for an item, an icon is used instead. Here we see the image being provided by the foaf:depiction property which is included in an optional triple pattern so as to include in the collection items without a foaf:depiction. These items are depicted by the default icon, the 'RDF molecule', as shown below.

select distinct ?s ?image ?id 
from <http://www.meetup.com/linkeddata/members/> where { 
	service <http://uriburner.com/sparql> {
		?s ?p ?o; foaf:name ?id .
		optional { ?s foaf:depiction ?image }
} limit 100

Rather than use the same icon for all items without an image, it's possible to display a class-specific icon to give an indication of the item's type. The icon to be displayed is selected from a small extensible set provided by the SparqlCxml VAD, based on the item's rdfs:type. (More details of how this is done can be found in SparqlCxmlDeep Zoom Collections in section 'Default Images for Pivot Collections'.)

When generating CXML from a DESCRIBE or CONSTRUCT, an item's rdfs:type is readily available and the choice of icon is made automatically. For a SELECT, it's necessary to indicate an item's type through the ?itemtype reserved query variable. When the above query is modified to include ?itemtype, the default icon is replaced by a 'silhouette' which depicts items of type foaf:Person.

select distinct ?s ?image ?id ?itemtype 
from <http://www.meetup.com/linkeddata/members/> where { 
	service <http://uriburner.com/sparql> {
		?s ?p ?o; foaf:name ?id ; a ?itemtype .
		optional { ?s foaf:depiction ?image }
} limit 100 

Example 9:

Using a fixed image as the item depiction, with the item's URI forming the QRcode

prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#>
prefix xsd: <http://www.w3.org/2001/XMLSchema#>
prefix frbr: <http://purl.org/vocab/frbr/core#>
prefix dct: <http://purl.org/dc/terms/>

select ?work as ?href ?date ?description ?valid ?subject as ?qrcode ?image ?name
      (<http://i.telegraph.co.uk/telegraph/multimedia/archive/01590/backbenchers-pa_1590370c.jpg>) as ?image
{ service <http://gov.tso.co.uk/legislation/sparql> ( define lang:dialect 65535 )
    select *
      ?work a frbr:Work .
      ?work dct:title ?description .
      ?work dct:created ?date .
      ?work dct:subject ?subject .
      ?work dct:valid ?valid .
      ?work rdfs:label ?name .
      optional {?work foaf:depiction ?image} .
      filter (?date >= "2010-10-15"^^xsd:date)
    limit 100


select ?s as ?href 
(<http://www.railaroundbirmingham.co.uk/graphics/crests.jpg>) as ?image 
from <http://spending.lichfielddc.gov.uk/data> 
where {?s ?p ?o} 
limit 1000

Query Variable Order

If a wildcard SELECT list isn't used, the query variables in the SELECT list must include one representing the subject URI. The variable corresponding to the subject URI must also be first in the select list.

In the next example, ?href is, as well as setting each Item's Href attribute, also acting as a placeholder for the subject URI (in place of the more usual ?s).

PREFIX wo: <http://purl.org/ontology/wo/>

SELECT DISTINCT  ?href ?name  ?kingdom ?image ?habitat ?page ?adaptation ?prog
WHERE {?href wo:kingdom ?kingdom; 
wo:adaptation ?adaptation; 
wo:livesIn ?habitat; 
rdfs:label ?name. 
OPTIONAL {?prog po:subject ?href.} 
OPTIONAL {?href foaf:depiction ?image.} 
OPTIONAL {?page foaf:primaryTopic ?href} }  
limit 500

A re-ordered select list of 'SELECT DISTINCT ?name ?href ...' would be rejected by the SPARQL compiler.

Reserved Query Variables for Geodata

HtmlPivotViewer's "Map" view displays any geo-data contained in the collection as map markers. Currently HtmlPivotViewer looks for two kinds of facet to find geo-location points:
  1. A pair of numeric facets named "latitude" and "longitude" (or which have these substrings in the facet name). The facet values are combined to form a geo-marker.
  2. Any facet named "location" (or which contains the substring "location") is used to do geo-location lookups, with some special handing for facet values in the formats below. Any other format is passed to a geocoding service to try to retrieve a co-ordinate pair.
    • Generic co-ordinates: format - POINT(<double>, <double>);
    • Generic co-ordinates: format - <double>, <double>

The searches for "latitude", "longitude" or "location" in a facet name are case-insensitive. HtmlPivotViewer uses an internal cache to minimize the number of lookups.

In addition to plotting a map marker for each item in a collection, HtmlPivotViewer also looks for a facet called "geometry" and attempts to use the value of the facet to plot a geometric area. This can be useful, for example, to highlight where a dataset has been filtered to only include values that lie in a particular defined geometry. The value of the "geometry" facet should be a string in WKT format. Only the first value found will be plotted.

Alternatively, a separate area can be associated with each item in a collection using a facet called "area". Again, the values should be a string in WKT format. Once an item in the collection is selected then the associated area will be highlighted on the map. This facet has to be used in conjuction with an exact point from either the "latitude", "longitude" or "location" facet. An item with just an area is not plotted.

Thus ?latitude, ?longitude, ?location, ?geometry and ?area can be viewed as special variables specifically for PivotViewer. Conversely, the previously mentioned special variables ?image, ?qrcode etc are only for CXML generation.

Graph virtcxml

Graph <http://www.openlinksw.com/schemas/virtcxml#> can be used to customize the CXML generation for DESCRIBEs and CONSTRUCTs. The graph can also be referred to by its short name virtcxml: which is declared in table DB.DBA.SYS_XML_PERSISTENT_NS_DECL (e.g. select * from virtcxml: where { .... ) The graph is created by the sparql_cxml VAD and populated with some default settings for popular ontologies such as the Dublin Core and FOAF. Add entries to this graph as needed for your own collections.


Customization of the DESCRIBE output is done primarily by associating a virtcxml:FacetType with an entity property. The entity property to be modified is declared as being an instance of the class virtcxml:FacetCategoryPattern. The property's datatype can then be set by assigning the desired CXML type to the virtcxml:FacetType property. The supported types include the standard CXML datatypes, String, LongString, Number, DateTime and Link; but also some pseudo datatypes for setting certain item attributes from RDF property values. These special datatypes correspond to the reserved query variables presented earlier. They are:

Datatype Equivalent Reserved SELECT Variable CXML Element/Attribute
Item/d1p1:Copyright ?copyright A facet for a copyright link about the item
Item/Description ?description A facet for a text description of an item to be displayed in the info panel
Item/@Href ?href A facet for an URL associated with an item. Double-clicking the item in the Pivot client will navigate the user to this URL
Item/@Id ?id A facet for an unique id of an item in the item list. If not specified the decimal representation of serial is used
Item/@virt:Image ?image A facet for an absolute URI of the image that will be added to an automatically created single-use collection
Item/@Img ?img A facet for an relative URI of the image in the collection
Item/@virt:Itemtype ?itemtype A facet for an item type that is used, for example, to choose the fallback icon for the item
Item/@Name ?name A facet for a name of the item which will appear at the top of the info panel
Item/@virt:QRcode ?qrcode A facet for QRcode data that should be placed in the image
Item/d1p1:Related ?related A facet for links to web-pages or collections related to the item

Examples of their use follow.

Setting an Item's Image

An item's foaf:depiction property supplies the item's image:

  a virtcxml:FacetCategoryPattern ;
  virtcxml:FacetType "Item/@virt:Image" ;
  virtcxml:FacetDescription "A facet for absolute URI of depiction of a collection item" .

As with the other pseudo datatypes, more than one property can be declared as being of type "Item/@virt:Image". This would typically be the case where the collections to be generated include more than one entity type. In which case, there would be one "Item/@virt:Image" declaration for each included type, except where they share the same image property. For any entity type without a "Item/@virt:Image" property, an icon is displayed as a fallback image.

Building on the above definition, graph virtcxml supports a virtcxml:aliasOf property as a shorthand notation for declaring properties as having FacetType "Item/@virt:Image". As an illustration of how the graph can be customized by a developer, the SPARUL insert statement below declares mo:image as an alias of foaf:depiction to enable the use of images available through the mo:image property as custom images (aka ABox images) in the DESCRIBE query which follows. The query itself can be executed against endpoint http://lod.openlinksw.com/sparql.

prefix virtcxml: <http://www.openlinksw.com/schemas/virtcxml#> 
prefix mo: <http://purl.org/ontology/mo/>
prefix foaf: <http://xmlns.com/foaf/0.1/>

insert data into virtcxml:
  mo:image virtcxml:aliasOf foaf:depiction .

Without this declaration (which is included in the graph virtcxml by default), only icons (aka TBox images) would be displayed by HtmlPivotViewer.

DESCRIBE ?s FROM <http://www.bbc.co.uk/music/> 
WHERE { ?s a mo:SoloMusicArtist ; mo:image ?img .} LIMIT 100

?img ensures that each artist has a custom image. It should not be confused with the reserved query variable ?image, used only by SELECTs.

Setting an Item's Name

If an item has any of the properties foaf:name, rdfs:label, dc:title or dcterms:title, use it as the item's name:
  a virtcxml:FacetCategoryPattern ;
  virtcxml:FacetType "Item/@Name" ;
  virtcxml:FacetDescription "A facet for full name of a person" .
  a virtcxml:FacetCategoryPattern ;
  virtcxml:FacetType "Item/@Name" ;
  virtcxml:FacetDescription "Facet to use for name of item" .
  a virtcxml:FacetCategoryPattern ;
  virtcxml:FacetType "Item/@Name" ;
  virtcxml:FacetDescription "Facet to use for name of item" .

  a virtcxml:FacetCategoryPattern ;
  virtcxml:FacetType "Item/@Name" ;
  virtcxml:FacetDescription "Facet to use for name of item" .

Setting an Item's Description

For any item with a Dublin Core description property, use this for the CXML Item/Description element. For campsite items, use the camp:description property, and for ski resorts use the ski:description property. Note that the property to use can have any name, it need not be named prefix:description.

  a virtcxml:FacetCategoryPattern ;
  virtcxml:FacetType "Item/Description" ;
  virtcxml:FacetDescription "Facet to use for item description" .

  a virtcxml:FacetCategoryPattern ;
  virtcxml:FacetType "Item/Description" ;
  virtcxml:FacetDescription "Facet to use for item description" .
  a virtcxml:FacetCategoryPattern ;
  virtcxml:FacetType "Item/Description" ;
  virtcxml:FacetDescription "Facet to use for campsite description" .
  a virtcxml:FacetCategoryPattern ;
  virtcxml:FacetType "Item/Description" ;
  virtcxml:FacetDescription "Facet to use for ski resort description" .

Setting a Facet's Name

Use 'Type of Resort' in place of 'camp:resort_type' as the name of the facet:

  a virtcxml:FacetCategoryPattern ;
  virtcxml:FacetName "Type of Resort" ;
  virtcxml:FacetDescription "Type of resort" .

The ability to override a facet's name is not limited to DESCRIBEs. The default facet names generated by a SELECT can also be overridden in the same way. For instance, the facet name 'virtcxml:FacetO' created for query variable ?o in a SELECT starting:

select ?s ?p ?o where ...

can be changed to 'Object' with an entry in virtcxml of the form:

  a virtcxml:FacetCategoryPattern ;
  virtcxml:FacetName "Object" ;
  virtcxml:FacetDescription "Renaming of facet associated with ?o" .

Likewise for

select ?s ?p ?o as ?loc ...

  a virtcxml:FacetCategoryPattern ;
  virtcxml:FacetName "Location" ;
  virtcxml:FacetDescription "..." .

Setting a Facet's Datatype

Overriding the default CXML String datatype for item properties:

The following examples show how to set the facet type using the virtcxml:FacetType property. However this should not be necessary for numeric and datetime values. If the values are RDF typed literals, the facet datatype is set automatically.

  a virtcxml:FacetCategoryPattern ;
  virtcxml:FacetType "Number" ;
  virtcxml:FacetDescription "Resort altitude (m)" .
  a virtcxml:FacetCategoryPattern ;
  virtcxml:FacetType "Link" ;
  virtcxml:FacetDescription "Related item" .

Setting the Facet Used to Generate a QR code

For SELECTs, reserved query variable ?qrcode identifies the property to use for generating QR codes. However this variable is unavailable for DESCRIBEs and CONSTRUCTs. For the latter, two options exist. Either designate the property to use through an entry in graph virtcxml: or, in the absence of such an entry, use the default behaviour whereby the subject URI is automatically used as the basis for the QR code, assuming QR code generation has been enabled through the /sparql query string setting "format=text/cxml+qrcode", or through the SPASQL pragma 'define output:format "CXML;QRCODE"' as described above in SPARQL Endpoint CXML Extensions.

To specify a QR code source-property in virtcxml:, assign the property a virtcxml:FacetType property of "Item/@virt:QRcode". e.g.

	a virtcxml:FacetCategoryPattern ;
	virtcxml:FacetType "Item/@virt:QRcode" ;
	virtcxml:FacetDescription "Use a product's GoodRelations EAN-UCC-13 code (Universal Product Code) for it's QRcode" .

In most circumstances it will probably be preferable to QR-encode the item (subject) URI, in which case an entry in virtcxml: is not needed.

Excluding a Facet from the Filter Panel

There may be occasion where there is little reason to include a facet in the HtmlPivotViewer filter panel, for instance if the facet value is unique for each item and you are unlikely to want to filter on this facet. To exclude a facet from the filter panel, set its virtcxml:PivotIsFilterVisible property to "false" in virtcxml: e.g.

  a virtcxml:FacetCategoryPattern ;
  virtcxml:FacetName "SHA1 of a Personal Mailbox" ;
  virtcxml:FacetType "String" ;
  virtcxml:PivotIsFilterVisible "false" .

Using Labels in Place of URIs

In the world of RDF, a URI is the primary currency for naming and identifying classes, properties and individuals. In contrast, in HtmlPivotViewer items and properties are identified by name, and property values are in the main character strings. The screenshot on the left below shows the visual impact of property values which are resource URIs rather than literals, if HtmlPivotViewer?'s auto-label feature (see below) is disabled. (This feature cannot be disabled by end-users, but has been in the following example to illustrate its effect). The values for facet virtcxml:FacetHabitat were returned by the query:

SELECT DISTINCT  ?s as ?href ?kingdom ?image ?habitat 
WHERE {?s <http://purl.org/ontology/wo/kingdom> ?kingdom; 
<http://purl.org/ontology/wo/livesIn> ?habitat. 
OPTIONAL {?s foaf:depiction ?image . } } 
limit 2500

The values for property http://purl.org/ontology/wo/livesIn (corresponding to facet virtcxml:FacetHabitat) are individuals identified by URIs, e.g. http://www.bbc.co.uk/nature/habitats/Tropical_and_subtropical_grasslands,_savannas,_and_shrublands#habitat, which while great for machine-readable data does little for end-user readability. From a user's perspective, it's clearly preferable that URIs should be hidden as far as possible.

One option would be to display a property's label in place of its URI and keep the property type as CXML String. While for some properties, such as rdfs:seeAlso, it may be desirable to type the property as CXML Link, and use the property label for the anchor text, in most cases using CXML String makes for a better user experience and more searchable collections. The reason being that facets of type CXML Link are not exposed in the Filter panel for filtering, whereas facets of type CXML String are. In order to achieve the desired effect, the query could be recast to retrieve labels attached to properties through the rdfs:label predicate:

SELECT DISTINCT  ?s as ?href ?name ?kname ?image ?hname 
WHERE {?s <http://purl.org/ontology/wo/kingdom> ?kingdom; 
rdfs:label ?name; 
<http://purl.org/ontology/wo/livesIn> ?habitat. 
OPTIONAL {?s foaf:depiction ?image.} 
?kingdom rdfs:label ?kname. 
?habitat rdfs:label ?hname} 
limit 2500

View Pivot collection

The screenshot above right shows the effect of ?hname in the HtmlPivotViewer Filter panel. (The order of the images is different because HtmlPivotViewer orders on virtcxml:FacetHabitat in the first case and virtcxml:FacetHname in the second.) Those below show the impact of ?name on the item name in the Info panel, replacing the aardvark URI by the much more readable 'Aardvark'.

Exploiting Inferencing

The last example showed rdfs:label being used to supply human-friendly labels for some properties. Inferencing provides a more generic approach to the problem, particularly where a collection may consist of more than one entity type. Virtuoso defines an inference context, virtrdf-label, specifically for inferring property labels. The context, defined in graph virtrdf-label, declares many properties routinely used for labels as being a subproperty of virtrdf:label. e.g.
rdfs:label rdfs:subPropertyOf <http://www.openlinksw.com/schemas/virtrdf#label>
dcterms:title rdfs:subPropertyOf <http://www.openlinksw.com/schemas/virtrdf#label>

By including the pragma 'define input:inference "virtrdf-label" before the last query above, label inferencing is enabled and any reference to rdfs:label could be replaced by virtrdf:label.

Automatic Labels

When handling property values which are URIs, rather than require the query to convert these URIs to labels through the inclusion of additional rdfs:label (or virtrdf:label) patterns, such as "?kingdom rdfs:label ?kname . ?habitat rdfs:label ?hname ." as illustrated above, Virtuoso will attempt to infer such labels automatically. Similarly for property URIs, Virtuoso will attempt to find the property's label and use this for the property (i.e. facet) name rather than use the CURIE format of the property URI. For example, "Creator" is used as a property name is place of "dc:creator". When searching for auto-labels, SparqlCxml looks first in graph virtcxml: for facet names set using the virtcxml:FacetName property, before querying the local quad store for labels revealed through the virtrdf-label inference context.

The next example shows how the aesthetics of a recipe description can be improved using auto-labels. Starting from the Conductor's /sparql UI and executing the illustrated query

generates a query result URI which when passed to HtmlPivotViewer results in

Notice that the values for facet dv:ingredient are labels, not URIs. The ingredient URIs have been hidden by the auto-labelling. The full RDF description of the ingredient 'almonds' is given by:

SELECT * WHERE { <http://kingsley.idehen.net/about/id/entity/http/www.bbc.co.uk/food/recipes/almondymincepies_14566#hrecipe55g%2F2oz%20ground%20almonds%20> ?p ?o}

which returns

p o
http://www.w3.org/1999/02/22-rdf-syntax-ns#type http://rdf.data-vocabulary.org/Ingredient
http://rdf.data-vocabulary.org/name almonds
http://rdf.data-vocabulary.org/amount 55g/2oz ground almonds

and which reveals that the auto-labelling has used property <http://rdf.data-vocabulary.org/name> in place of the ingredient URI. In order for this to happen, the virtrdf-label inference context declares dv:name as an rdfs:subPropertyOf virtrdf:label.

SELECT * from <virtrdf-label> WHERE {?s ?p ?o}

includes in its output

s p o
http://rdf.data-vocabulary.org/name http://www.w3.org/2000/01/rdf-schema#subPropertyOf http://www.openlinksw.com/schemas/virtrdf#label

Also note that in the above DESCRIBE query, ?image functions as an ordinary variable, not a reserved query variable, as is the case with a SELECT. It is used to ensure each returned recipe has an accompanying image. For this DESCRIBE, in order to specify that the images for the DZC are provided by dv:photo, an appropriate entry was added to graph virtcxml using a simple SPARUL statement:

prefix dv: <http://rdf.data-vocabulary.org/>

insert data into virtcxml:
    a virtcxml:FacetCategoryPattern ;
    virtcxml:FacetType "Item/@virt:Image" ;
    virtcxml:FacetDescription "A facet for use as an item image" .

Auto-label extraction for URIs will not be performed if a facet is explicitly typed as CXML Link in graph virtcxml. An entry such as

prefix dv: <http://rdf.data-vocabulary.org/>
. . .
  a virtcxml:FacetCategoryPattern ;
  virtcxml:FacetName "Ingredient" ;
  virtcxml:FacetType "Link" .

for facet "Ingredient" will result in the facet values using a URI for both the Link's anchor text and href, instead of an inferred label for the anchor text. Auto-label extraction is only done when auto-type detection takes place.

Label Localization

In order to exploit URI to label conversion to maximum effect, it is important that ontologies and instance data use language tags in multi-lingual scenarios to localize property values, as illustrated by this made-up extract

:EWD1036 a bibo:Document ;
  dc:creator "E. W. Dijkstra"@en ;
  dc:subject  "computer science"@en ;
  dc:subject  "informatique"@fr ;
  dc:title "The Cruelty of Really Teaching Computer Science"@en .

Graph virtrdf-label declares dc:title rdfs:subPropertyOf virtrdf:label; so that dc:title serves as a potential label. Queries used for CXML generation can then restrict results, and hence labels, to a particular language by specifying the appropriate language tag. e.g.

DESCRIBE ?s WHERE { ?s a bibo:Document ; dc:subject "computer science"@en }
SELECT * WHERE { ?s dc:subject ?subject ; ... FILTER ( lang(?subject) = "EN" ) }

At present the onus is on the query writer to use language tags in the base query, but it is anticipated that SparqlCxml will in future perform this kind of filtering automatically through detection of the locale being used by the user agent.

Link Typing

The default setting for the auto-label feature is to set the type of the replacement labels to CXML String. However, the property URIs can also be typed as CXML Link, if required, with the auto-label acting as the anchor text and the property URI as the href. In the /sparql UI the dropdown 'Facet link behaviour' controls the target CXML type used by auto-labelling. In the Pivot Edit interface, the drop down is labelled 'Attribute and Value Link Style'. The choice of CXML String as the default is because it isn't possible to filter on properties typed as CXML Link; HtmlPivotViewer? does not display Link properties in the filter panel. Typing a facet like foaf:topicOfInterest as Link would prevent a user from selecting a group of people with the same interest, e.g. http://dbpedia.org/resource/Semantic+Web, via the filter panel.

Our recipe example used the default 'Local Faceted Navigation Link' option, so the properties are CXML Strings. This can be seen in the raw CXML for the dv:ingredient facet

<Collection SchemaVersion="1.0" Name="SPARQL Query Results">
    <FacetCategory Name="dv:author" Type="String"/>
    <FacetCategory Name="dv:cookTime" Type="String"/>
    <FacetCategory Name="dv:ingredient" Type="String"/>
    <FacetCategory Name="dv:instructions" Type="LongString" d1p1:IsFilterVisible="false"/>
    <FacetCategory Name="dv:name" Type="String"/>
    <FacetCategory Name="dv:yield" Type="String"/>
    <FacetCategory Name="Subject Type" Type="String" d1p1:IsFilterVisible="true" d1p1:IsMetaDataVisible="true" d1p1:IsWordWheelVisible="true"/>
  <Items ImgBase="http://kingsley.idehen.net/DeepZoomCache/cxml35073.dzc">
    <Item Id="0" Img="#0" Href="http://kingsley.idehen.net/about/id/entity/http/www.bbc.co.uk/food/recipes/almondymincepies_14566#hrecipe" Name="Almondy mince pies">
        <Facet Name="dv:author">
          <String Value="Silvana Franco"/>
        <Facet Name="dv:cookTime">
          <String Value="10 to 30 mins"/>
        <Facet Name="dv:ingredient">
          <String Value="cream"/>
          . . .
          <String Value="almonds"/>
          . . .

If 'Facet link behaviour' or 'Attribute and Value Link Style' is set to 'External Resource Links', the output for dv:ingredient becomes

<Collection SchemaVersion="1.0" Name="SPARQL Query Results">
    . . .
    <FacetCategory Name="dv:ingredient" Type="Link" d1p1:IsFilterVisible="false"/>
    . . .
        . . .
        <Facet Name="dv:ingredient">
          <Link Href="http://kingsley.idehen.net/about/id/entity/http/www.bbc.co.uk/food/recipes/almondymincepies_14566#hrecipecream%2C%20custard%2C%20or%20brandy%20butter%2C%20to%20serve" 
           . . .
           <Link Href="http://kingsley.idehen.net/about/id/entity/http/www.bbc.co.uk/food/recipes/almondymincepies_14566#hrecipe55g%2F2oz%20ground%20almonds%20"
           . . .

Should you want to override the 'Facet link behaviour' dropdown settings and force the type of a suitable property to Link, you can do so by adding an entry for the property in graph virtcxml:, assigning it a virtcxml:FacetType of "Link". The installed virtcxml: does this for rdfs:seeAlso,

  a virtcxml:FacetCategoryPattern ;
  virtcxml:FacetType "Link" ;
  virtcxml:FacetDescription "Related item" .

A property with virtcxml:FacetType "Link" will be typed as Link only if all its values are URIs and none are literals. Again, where auto-labels can be found, these provide the anchor text.

The href associated with each entity that becomes the link behind the title in the Info Panel can also be controlled in a similar way. In the /sparql UI the option is labelled 'External resource link' and in the Pivot Edit UI it is labelled 'Entity Link Style'. The value is not used for filtering so there is no benefit to defaulting to a CXML String. The default value is 'External Resource Link'. Other options allow you to, for example, do a SPARQL DESCRIBE on the the item.

Link Typing Options

Facet Values

The full set of link-typing options available in the /sparql UI are detailed below. The Facet Pivot Bridge and HtmlPivotViewer (via the 'Edit' feature) also expose the same options. Pick one of these options when executing SPARQL from any of these services.

'Facet link behaviour' or 'Attribute and Value Link Style' Dropdown:


The full set of link-typing options for the link behind the entity titles that are available in the /sparql UI are detailed below. The Facet Pivot Bridge and HtmlPivotViewer (via the 'Edit' feature) also expose the same options. Pick one of these options when executing SPARQL from any of these services.

'External resource link' or 'Entity Link Style' Dropdown:

Controlling Link Typing through /sparql Query String Parameters

The link styling options provided by the /sparql UI ultimately reduce to two query string parameters in the SPARQL query URI it builds. Query string parameter CXML_redir_for_hrefs corresponds to the 'Facet link behaviour' or 'Attribute and Value Link Style' dropdown. It can take one of the following values:

Parameter CXML_redir_for_subjs corresponds to the 'External resource link' or 'Entity Link Style' dropdown. It can take one of the following values:

Controlling Link Typing through Virtuoso/PL

When executing a SPARQL query directly from Virtuoso/PL, the controls used by the /sparql and /fct UIs for CXML link styling are not available. To specify the required CXML link typing in Virtuoso/PL, set the connection variables SPARQL_CXML_redir_for_subjs and SPARQL_CXML_redir_for_hrefs. In the code extract below:

  connection_set ('SPARQL_CXML_redir_for_subjs', redir_for_subjs);
  connection_set ('SPARQL_CXML_redir_for_hrefs', redir_for_hrefs);

  qry := 'sparql define output:format "CXML;QRCODE" define sql:describe-mode "SPO" describe ?s1 where { . . . } ';
  sqls := '00000';
  exec (qry, sqls, msg, vector(), 1, meta, result);
  . . .

variables redir_for_subjs and redir_for_hrefs take the same values as the CXML_redir_for_subjs and CXML_redir_for_hrefs query string parameters.