Pivot Collections (Part 8)

Frequently Asked Questions

SparqlCxml FAQs

Which Virtuoso releases support SparqlCxml and HtmlPivotViewer?

SparqlCxml and HtmlPivotViewer require Virtuoso Commercial Edition V6.2 (Build 3129 or later). SparqlCxml is not supported in Virtuoso Open Source Edition (VOS).

Are third-party (non-Virtuoso) SPARQL endpoints supported?

Third-party SPARQL endpoints are supported through Virtuoso's SPARQL-FED (Federated SPARQL) support. CXML can be generated from a third-party endpoint without this native facility by routing the query through a Virtuoso SPARQL endpoint, stipulating the third-party endpoint in a SERVICE clause.

Which SPARQL query types does SparqlCxml support?

SparqlCxml, that is CXML generated directly from the /sparql endpoint, supports SELECT, DESCRIBE and CONSTRUCT queries. All CXML generated by the Facet Pivot Bridge uses an internal DESCRIBE.

Are there limits on the size of a collection? How are large result sets handled?

Memory constraints limit the maximum collection size the HtmlPivotViewer control can handle. To mitigate the effects of this limit, the Facet Pivot Bridge supports paged collections. Large collections are split into pages whose size is controlled through a 'Page size' control in the bridge interface. Each page constitutes a sub-collection, containing no more than the number of items specified by the page size. The sub-collections are linked through first, last, next and previous links under the 'Related Collections' section of the HtmlPivotViewer display. The bridge also imposes a configurable limit (default 5MB) on the size of the CXML files it generates.
Collections generated directly from queries against a Virtuoso /sparql endpoint can be controlled through an appropriate LIMIT qualifier.

Do I need to install ImageMagick?

No. Virtuoso Commercial Edition releases include a bundled ImageMagick plugin. The plugin is standalone and includes all the functionality needed to generate Deep Zoom images.

Why are my collections sometimes slow to load?

HtmlPivotViewer collections can be significantly slower to load than a simple web page. Many factors can influence the load time, however there are several actions you can take to reduce it. Choosing between them requires understanding which parts of the collection delivery pipeline can be the most time-intensive. Possible factors to consider include:
- The characteristics of the SPARQL query and the result set size
  - Virtuoso's query execution speed is generally excellent. Obviously though, the query time is heavily influenced by the result set size. SPARQL-FED queries to a remote sparql endpoint will, unsurprisingly, be slower than to the local endpoint.
- Loading of the collection by the HtmlPivotViewer control
  - The CXML file for large collections can be several MBs. Depending on the available bandwidth, there may be a noticeable delay while HtmlPivotViewer retrieves and loads a file of this size.
- Image processing and rendering
  - In addition to the normal query processing and conversion of the result set to CXML, the collection generation pipeline must generate a 3-level Deep Zoom image (DZI) pyramid for each item.

How can I reduce the collection load time?

Reduce the query resultset size
- by adding a LIMIT clause to the query, if generating the CXML from a /sparql endpoint or
- by using the Facet Pivot Bridge's collection paging feature
Exploit SparqlCxml's image cache
- When a collection is first generated, all the images for the DZC must be downloaded, resized and converted to 3-level DZIs. The source images and the DZIs are then cached for 24 hours. Subsequent renderings of the collection inside this window will not carry this image processing overhead
Use a materialized view
- Rather than generating a collection dynamically each time it's viewed, use a materialized view in the form of a SparqlCxml DET (a virtual hypermedia resource), or a CXML snapshot generated by the Facet Pivot Bridge.

HtmlPivotViewer FAQs

Which browsers and platforms support HtmlPivotViewer?

HtmlPivotViewer is supported on all browsers that support HTML5.

Can I create social bookmarks to HtmlPivotViewer collections?

Yes. HtmlPivotViewer integrates with the AddThis social bookmarking service via an AddThis widget. From the HtmlPivotViewer interface, you can create a permalink for the collection being viewed and post it to a variety of social network sites including Facebook, Twitter, Google Bookmarks, Delicious etc. In order to use this service, you require a key. HtmlPivotViewer looks for an AddThis key in the Virtuoso registry. If one isn't found, it uses a temporary key. You can add your personal AddThis key to the Virtuoso registry using Virtuoso's registry_set function, using '_pivot_AddThis' for the key name.

How do I decouple presentation from representation?

One of the key tenets of Linked Data is the decoupling of the visual presentation of an entity from its data representation. HtmlPivotViewer supports this decoupling by making a collection's CXML data available in other raw Linked Data formats. These other formats (N-Triples, N3/Turtle, JSON, XML and OData) are accessible through links at the foot of the HtmlPivotViewer display which include alternate &output= query string settings appended to the SPARQL query URI. e.g.
- N3/Turtle: http://linkeddata.uriburner.com/sparql?default-graph-uri=?&query=DESCRIBE?&output=ext%2Frdf%2Bn3?
- JSON: http://linkeddata.uriburner.com/sparql?default-graph-uri=?&query=DESCRIBE?&output=application%2Frdf%2Bjson
Other (i.e. non-HtmlPivotViewer) presentations of an entity, for instance in HTML, can provide access to the base entity data using the same approach. Simply include links similiar to that below in the <head> section of your (static or dynamically generated) HTML page.
```
  <link rel="alternate" title="My data in Linked Data form"
   type="application/rdf+xml"
   href="http://linkeddata.uriburner.com/sparql?default-graph-uri=?&query=DESCRIBE?&output=ext%2Frdf%2Bn3/>" 
```
As an alternative to explicitly stating the required data format through &output=, Virtuoso's support for HTTP content negotiation allows clients to request the desired format through HTTP Accept headers.

My collection has loaded but why are my images blank?

If the collection has loaded and the Filter Pane contains the facets but there are no images in the tile pane just a blue box when highlighted with the the mouse then there there is is a problem with the deep zoom images associated with the collection. If the collection has been generated by Virtuoso then it is likely that the Virtuoso ImageMagick plugin has not been correctly loaded. In your virtuoso.ini in the [Plugins] section make sure you have a lines for "plain, im" and "plain, qrcode" like this:
```
[Plugins]
LoadPath=../hosting
...
Load6=plain, im
Load7=plain, qrcode
```
Remember to restart Virtuoso if you make any changes to your virtuoso.ini.
If the collection has loaded but you do not see individual images for each item but instead you see the the generic triple symbol you may need to reset the deepzoom cache. This can be done from isql or the Virtuoso conductor by executing the following stored procedure:
```
db.dba.dzc_reset();
```

HtmlPivotViewer has loaded but why is it empty?

This can happen when the HtmlPivotViewer editor is used to enter a new SPARQL query but the SPARQL-CXML VAD has not been installed. Download the VAD from sparql_cxml_dav.vad, install it on the server where you are executing the query then retry the query.

How can I fix a HtmlPivotViewer collection parsing error?

On opening the collection you get a message box with the title HTML5 PivotViewer and the message Error parsing CXML Collection. This error indicates that there is a syntax error in the cxml file. The message box gives the URL of the collection and also a further message from the xml parser which may contain a line and character number to pinpoint exactly where the problem is. To resolve this kind of problem it is often helpful to look at the cxml file directly. Click on the CXML link in the Raw Linked Data formats section at the bottom of the page. Note: in FireFox, Opera and Internet Explorer the cxml is displayed as xml in the browser window, while in Safari and Chrome you will have to look at the page source. However, the xml formatting in FireFox, Opera and Internet Explorer may mean that the line numbers in the error message do not match the line number in the browser window so in all cases it may be best to look at the page source. Alternatively, you could use curl to download the file and then examine it in a text editor.

How can I fix a HtmlPivotViewer collection downloading error?

On opening the collection you get a message box with the title HTML5 PivotViewer and the message Error loading CXML collection. This error is a very general error that means the HtmlPivotViewer has not been able to download a collection from the supplied URL and can occur for a number of reasons. The simplest is that the URL has been typed incorrectly. The message box contains the URL. Try pasting that URL in the address bar of a browser. If it is correct you should be able to see the contents of the collection cxml file. Note: in FireFox, Opera and Internet Explorer the cxml is displayed as xml in the browser window, while in Safari and Chrome it is best to look at the page source. If the URL is in the form
```
http://your server name:your server port/proxy/?url=the collection url
```
then it can be helpful to try pasting just the collection url in to a browser. If just the collection url loads the cxml file correctly but the proxy url fails then the proxy service on your server may not be enabled.
If the collection has been generated dynamically from a SPARQL query and there is a problem generating the collection then you will also get a download error. In the message box you may get an additional message with an HTTP response code 500 and message "internal server error" This typically means that the SPARQL query has failed. A good troubleshooting method when this error occurs is to click on the CXML link in the Raw Linked Data formats section at the bottom of the page. If the query is failing then you will get a more detailed error message that should help you resolve the problem.

How can I fix a HtmlPivotViewer DeepZoom cache downloading error?

* On opening the collection you get a message box with the title HTML5 PivotViewer and the message Error loading from DeepZoom cache. This error can occur if the server hosting the collection has not allowed access to the DeepZoom cache using Cross Origin Resource Sharing (CORS) or the permissions on the DeepZoom collection do not allow read access. Cross Origin Resource Sharing must be enabled on the server using the Virtuoso Conductor. See the Virtuoso Tips and Tricks Guide.