Pivot Collections (Part 8)

Contents

Part 1: Introduction

Part 2: SparqlCxml

Part 3: SparqlCxml Deep Zoom Collections

Part 4: PivotViewer

Part 5: Importing CXML

Part 6: Facet Pivot Bridge

Part 7: DETs : Persisting SPARQL Query Results to DAV

Part 8: Frequently Asked Questions (FAQs)

• Frequently Asked Questions
  • SparqlCxml FAQs
    • Which Virtuoso releases support SparqlCxml and PivotViewer?
    • Are third-party (non-Virtuoso) SPARQL endpoints supported?
    • Which SPARQL query types does SparqlCxml support?
    • Are there limits on the size of a collection? How are large result sets handled?
    • Do I need to install ImageMagick?
    • Why are my collections sometimes slow to load?
    • How can I reduce the collection load time?
  • PivotViewer FAQs
    • Which browsers and platforms support PivotViewer?
    • Can I create social bookmarks to PivotViewer collections?
    • Which web mapping services does PivotViewer support?
    • How do I decouple presentation from representation?
    • My collection has loaded but why are my images blank?
    • PivotViewer has loaded but why is it empty?
    • How can I fix a PivotViewer collection parsing error?
    • How can I fix a PivotViewer collection downloading error?

Part 9: Glossary

---

Frequently Asked Questions

---

SparqlCxml FAQs

Which Virtuoso releases support SparqlCxml and PivotViewer?

• SparqlCxml and PivotViewer require Virtuoso Commercial Edition V6.2 (Build 3129 or later). SparqlCxml is not supported in Virtuoso Open Source Edition (VOS).

Are third-party (non-Virtuoso) SPARQL endpoints supported?

• Third-party SPARQL endpoints are supported through Virtuoso's SPARQL-FED (Federated SPARQL) support. CXML can be generated from a third-party endpoint without this native facility by routing the query through a Virtuoso SPARQL endpoint, stipulating the third-party endpoint in a SERVICE clause.

Which SPARQL query types does SparqlCxml support?

• SparqlCxml, that is CXML generated directly from the /sparql endpoint, supports SELECT, DESCRIBE and CONSTRUCT queries. All CXML generated by the Facet Pivot Bridge uses an internal DESCRIBE.

Are there limits on the size of a collection? How are large result sets handled?

• Memory constraints limit the maximum collection size the Silverlight PivotViewer control can handle. To mitigate the effects of this limit, the Facet Pivot Bridge supports paged collections. Large collections are split into pages whose size is controlled through a 'Page size' control in the bridge interface. Each page constitutes a sub-collection, containing no more than the number of items specified by the page size. The sub-collections are linked through first, last, next and previous links under the 'Related Collections' section of the PivotViewer display. The bridge also imposes a configurable limit (default 5MB) on the size of the CXML files it generates.
• Collections generated directly from queries against a Virtuoso /sparql endpoint can be controlled through an appropriate LIMIT qualifier.

Do I need to install ImageMagick?

• No. Virtuoso Commercial Edition releases include a bundled ImageMagick plugin. The plugin is standalone and includes all the functionality needed to generate Deep Zoom images.

Why are my collections sometimes slow to load?

• PivotViewer collections can be significantly slower to load than a simple web page. Many factors can influence the load time, however there are several actions you can take to reduce it. Choosing between them requires understanding which parts of the collection delivery pipeline can be the most time-intensive. Possible factors to consider include:
  • The characteristics of the SPARQL query and the result set size
    • Virtuoso's query execution speed is generally excellent. Obviously though, the query time is heavily influenced by the result set size. SPARQL-FED queries to a remote sparql endpoint will, unsurprisingly, be slower than to the local endpoint.
  • Loading of the collection by the PivotViewer control
    • The CXML file for large collections can be several MBs. Depending on the available bandwidth, there may be a noticeable delay while PivotViewer retrieves and loads a file of this size.
  • Image processing and rendering
    • In addition to the normal query processing and conversion of the result set to CXML, the collection generation pipeline must generate a 3-level Deep Zoom image (DZI) pyramid for each item. Further, it appears that the Silverlight control doesn't work smartly with data, making no use of cursors or paging, and instead making a local cache of all the data (and images).
  • The quality of any source geo-location data
    • Geo-location data directly available in the source data as latitude and longitude requires no conversion and is fast to render in PivotViewer's map view. Less precise location information, like place names, is converted to geo-data by passing it to Bing Map's geocode service.

How can I reduce the collection load time?

• Reduce the query resultset size
  • by adding a LIMIT clause to the query, if generating the CXML from a /sparql endpoint or
  • by using the Facet Pivot Bridge's collection paging feature
• Exploit SparqlCxml's image cache
  • When a collection is first generated, all the images for the DZC must be downloaded, resized and converted to 3-level DZIs. The source images and the DZIs are then cached for 24 hours. Subsequent renderings of the collection inside this window will not carry this image processing overhead
• Use a materialized view
  • Rather than generating a collection dynamically each time it's viewed, use a materialized view in the form of a SparqlCxml DET (a virtual hypermedia resource), or a CXML snapshot generated by the Facet Pivot Bridge.
• Exploit PivotViewer's geo-data caching
  • Encoding geo-location information directly as latitude and longitude values in the source data is the most efficient option. If the data instead uses location names, the overhead in converting these names to geo-data is incurred when a map view of the collection is rendered for the first time. Subsequent map views should be much quicker to display because all geo-data generated from place names is permanently cached in local Silverlight Application Storage (until the 1MB cache limit is reached, at which point the action taken depends on a user's Silverlight preferences).

---

PivotViewer FAQs

Which browsers and platforms support PivotViewer?

• PivotViewer is supported on all Silverlight supported platforms and browsers:
  • Windows, Mac OS X
  • Firefox (3+), Internet Explorer, Safari (3+), Chrome (4+), Opera (11)
  • For precise details of Silverlight compatible browsers and operating systems (plus any required service packs), refer to Microsoft's Get Silverlight page.

Can I create social bookmarks to PivotViewer collections?

• Yes. PivotViewer integrates with the AddThis social bookmarking service via an AddThis widget. From the PivotViewer interface, you can create a permalink for the collection being viewed and post it to a variety of social network sites including Facebook, Twitter, Google Bookmarks, Delicious etc. In order to use this service, you require a key. PivotViewer looks for an AddThis key in the Virtuoso registry. If one isn't found, it uses a temporary key. You can add your personal AddThis key to the Virtuoso registry using Virtuoso's registry_set function, using '_pivot_AddThis' for the key name.

Which web mapping services does PivotViewer support?

• PivotViewer's map view works with both Google Maps and Bing Maps, with Bing Maps the preferred option. Location information is displayed on a fully interactive Bing map. In order for Bing Maps to work, you must first add a Bing maps key to the Virtuoso registry using registry_set ('_pivot_BingMapsKey','yourkey'). A Bing map key can be obtained by creating an account at the Bing Maps Account Centre. If a valid Bing key is not found, PivotViewer defaults to using a static Google map.

How do I decouple presentation from representation?

• One of the key tenets of Linked Data is the decoupling of the visual presentation of an entity from its data representation. PivotViewer supports this decoupling by making a collection's CXML data available in other raw Linked Data formats. These other formats (N-Triples, N3/Turtle, JSON, XML and OData) are accessible through links at the foot of the PivotViewer display which include alternate &output= query string settings appended to the SPARQL query URI. e.g.
  • N3/Turtle: http://linkeddata.uriburner.com/sparql?default-graph-uri=?&query=DESCRIBE?&output=ext%2Frdf%2Bn3?
  • JSON: http://linkeddata.uriburner.com/sparql?default-graph-uri=?&query=DESCRIBE?&output=application%2Frdf%2Bjson
• Other (i.e. non-PivotViewer) presentations of an entity, for instance in HTML, can provide access to the base entity data using the same approach. Simply include links similiar to that below in the <head> section of your (static or dynamically generated) HTML page.
  

  
    <link rel="alternate" title="My data in Linked Data form"
     type="application/rdf+xml"
     href="http://linkeddata.uriburner.com/sparql?default-graph-uri=?&query=DESCRIBE?&output=ext%2Frdf%2Bn3/>" 
  

  
  
• As an alternative to explicitly stating the required data format through &output=, Virtuoso's support for HTTP content negotiation allows clients to request the desired format through HTTP Accept headers.

My collection has loaded but why are my images blank?

• If the collection has loaded and the Filter Pane contains the facets but there are no images in the tile pane just a blue box when highlighted with the the mouse then there there is is a problem with the deep zoom images associated with the collection. If the collection has been generated by Virtuoso then it is likely that the Virtuoso ImageMagick plugin has not been correctly loaded. In your virtuoso.ini in the [Plugins] section make sure you have a lines for "plain, im" and "plain, qrcode" like this:
  

  
  [Plugins]
  LoadPath=../hosting
  ...
  Load6=plain, im
  Load7=plain, qrcode
  

  
  Remember to restart Virtuoso if you make any changes to your virtuoso.ini.

PivotViewer has loaded but why is it empty?

• This can happen when the PivotViewer editor is used to enter a new SPARQL query but the SPARQL-CXML VAD has not been installed. Download the VAD from sparql_cxml_dav.vad, install it on the server where you are executing the query then retry the query.

How can I fix a PivotViewer collection parsing error?

• On opening the collection you get a message box with the title PivotViewer Error and the message Error parsing the collection. This error indicates that there is a syntax error in the cxml file. The message box gives the URL of the collection and also a further message from the xml parser which may contain a line and character number to pinpoint exactly where the problem is. To resolve this kind of problem it is often helpful to look at the cxml file directly. Click on the CXML link in the Raw Linked Data formats section at the bottom of the page. Note: in FireFox, Opera and Internet Explorer the cxml is displayed as xml in the browser window, while in Safari and Chrome you will have to look at the page source. However, the xml formatting in FireFox, Opera and Internet Explorer may mean that the line numbers in the error message do not match the line number in the browser window so in all cases it may be best to look at the page source. Alternatively, you could use curl to download the file and then examine it in a text editor.

How can I fix a PivotViewer collection downloading error?

• On opening the collection you get a message box with the title PivotViewer Error and the message Error downloading the collection. This error is a very general error that means the PivotViewer has not been able to download a collection from the supplied URL and can occur for a number of reasons. The simplest is that the URL has been typed incorrectly. The message box contains the URL. Try pasting that URL in the address bar of a browser. If it is correct you should be able to see the contents of the collection cxml file. Note: in FireFox, Opera and Internet Explorer the cxml is displayed as xml in the browser window, while in Safari and Chrome it is best to look at the page source. If the URL is in the form
  

  
  http://your server name:your server port/proxy/?url=the collection url
  

  
  then it can be helpful to try pasting just the collection url in to a browser. If just the collection url loads the cxml file correctly but the proxy url fails then the proxy service on your server may not be enabled.

  The second reason for encountering a download error is that the server hosting the collection has not allowed access using the clientaccesspolicy.xml or the permissions on the collection or deepzoom collection do not allow read access. In this case there will be a further message displayed in the message box saying that there was a Security Exception. For more information on the clientaccesspolicy.xml file see http://msdn.microsoft.com/en-us/library/cc197955%28v=vs.95%29.aspx.

  If the collection has been generated dynamically from a SPARQL query and there is a problem generating the collection then you will also get a download error. In the message box you may get an additional message with an HTTP response code 500 and message "internal server error" This typically means that the SPARQL query has failed. A good troubleshooting method when this error occurs is to click on the CXML link in the Raw Linked Data formats section at the bottom of the page. If the query is failing then you will get a more detailed error message that should help you resolve the problem.