• Topic
  • Discussion
  • VOS.VirtSparqlCxmlHtmlPivotViewer(Last) -- DAVWikiAdmin? , 2019-07-31 05:48:17 Edit WebDAV System Administrator 2019-07-31 05:48:17

    Pivot Collections (Part 4)


    Part 1: Introduction

    Part 2: SparqlCxml

    Part 3: SparqlCxml Deep Zoom Collections

    Part 4: HtmlPivotViewer

    Part 5: Importing CXML

    Part 6: Facet Pivot Bridge

    Part 7: DETs : Persisting SPARQL Query Results to DAV

    Part 8: Frequently Asked Questions (FAQs)

    Part 9: Glossary


    The HtmlPivotViewer control makes it easy to view and interact with massive amounts of data on the web. The Virtuoso HtmlPivotViewer provides a container for a customized (subclassed) version of the control offering a grid view of the data in addition to the standard tile view. The viewer can display both Virtuoso-generated and third-party collections, hosted locally on the viewer host or remotely. Other enhancements include the facility to bookmark static or dynamic collection URIs on popular social networking sites, in-place editing of SPARQL queries used for dynamic collection generation, Google translation and collection permalink creation with automatic URL shortening.


    After installing VAD html5pivotviewer_dav.vad, HtmlPivotViewer is exposed as http://{cname}/HtmlPivotViewer/. Any browser with support for HTML5 can launch the Virtuoso HtmlPivotViewer application.

    If the viewer is launched without specifying a collection URI, you will be prompted to supply one in the "Collection URL" textbox. Alternatively, the viewer will immediately load any collection specified in a query string, for example:


    In this instance the url parameter is a link to a dynamic "Solo Music Artist" collection, created through HtmlPivotViewer's permalink option. The query to generate the collection is:

    SDEFINE sql:describe-mode "SPO"
    DESCRIBE ?s FROM <http://www.bbc.co.uk/music/> 
    ?s a mo:SoloMusicArtist;
        mo:image ?image;
        mo:musicbrainz ?musicbrainz;
        foaf:page ?href;
        foaf:name ?name;
        rdfs:comment ?comments.
        optional {?s mo:wikipedia ?wikipedia}.
        optional {?s mo:biography ?bio}.
        optional {?s mo:myspace ?myspace}.
        optional {?s mo:imdb ?imdb}.
    LIMIT 100

    executed against SPARQL endpoint http://lod.openlinksw.com/sparql.

    You can re-create this collection from scratch by pasting the query into Virtuoso's SPARQL query UI hosted at /sparql and setting "Format Results As" to "CXML (Pivot Collection)"

    After executing the query, the browser address bar contains the full URL-encoded SPARQL query

    Paste this into the URL text box displayed by /HtmlPivotViewer to view the collection:

    Should you want to modify the source query to create a variant of the collection, it can be edited via HtmlPivotViewer's "Edit" link:

    HtmlPivotViewer offers two view styles - the standard tile view above and the table view.

    Table View

    The table view of the Solo Music Artist collection is shown below:

    The data in the table view can be manipulated by clicking on the cells in the table. Clicking on the name of an item in the Item column will select that item. The table will then only show rows that contain information about that particular item. The info pane for that item will also be displayed on the right hand side. Once a single item has been selected, clicking on any cell in the Item column will clear the selection and rows will be displayed for all items in the collection. Similarly, clicking on a facet in the Relation column will restrict the table to only showing that facet. Again, once a facet has been selected, clicking on any row in the Relation column will clear the selection. Where a facet is defined as filterable, the values of that facet, in the Value column, can be used to set a collection filter in the same way that clicking on a value in the info pane sets a filter. Filters are cleared using the filter pane on the left hand side. If a value is a URI as well as being used for setting a filter, then clicking the link out symbol next to the value will open that URI. If the value is a URI and cannot be used as a filter, then it is displayed as a traditional blue link and clicking it will open that URI. Where a URI has been associated with an item, it can be opened by clicking on the link out symbol next to the item name in the Item column. The table can be sorted on each column by clicking on the column headers.

    Map View

    An example map view, generated by a different query, is shown below:

    The SPARQL query providing the collection is:

    PREFIX  ski:   <http://www.openlinksw.com/ski_resorts/schema#>
    PREFIX  camp:  <http://www.openlinksw.com/campsites/schema#>
    PREFIX  geonames: <http://www.geonames.org/ontology#>
    SELECT * 
        { ?s  a                        ski:SkiResort         . 
          ?s  a                        ?itemtype             ; 
              geonames:countryCode     ?Country              ;
              ski:resort_name          ?name                 ;
              ski:resort_name          ?location             ;
              ski:beginner_slopes      ?beginner_slopes      ;
              ski:intermediate_slopes  ?intermediate_slopes  ;
              ski:advanced_slopes      ?advanced_slopes      ;
              ski:expert_slopes        ?expert_slopes        ;
              ski:altitude_m           ?altitude             .
          OPTIONAL { ?s  foaf:depiction  ?image } 
        { ?s  a                     camp:Campsite            . 
          ?s  a                     ?itemtype                ; 
              geonames:countryCode  ?Country                 ; 
              rdfs:label             ?name                   ;
              campsite:resort_type  ?resort_type             ;
              campsite:resort       ?resort                  ;
              campsite:resort       ?location                ; 
              campsite:region       ?region                  ;
              campsite:category     ?category                .
          OPTIONAL { ?s foaf:depiction ?image } 

    Notice the use of the reserved query variable ?location to indicate to SparqlCxml which property provides the map's geo-data. For more information, see see VirtSparqlCxmlHtml section "Reserved Query Variable for Geodata".

    The map view shown utilises the Google Maps API. In order to use this map view an API key is required. This may be obtained by following the instructions here. This key must be added to the Virtuoso registry using the HtmlPivotViewer configuration page. In the Virtuoso conductor select the System Admin tab and the The Packages sub menu. Scroll down the list of packages until you get to html5pivotviewer. Click on the Configure link in the right hand column. This will open up the HtmlPivotViewer configuration page.

    Alternatively, maps can be generated using OpenStreetMap and the Nominatim geocoding service. In this case, no API key is required. OpenStreetMap and Nominatim can be selected using the configuration page.

    Adding a Map Overlay

    The basic map view, whether you are using Google Maps or OpenStreetMap, can be enhanced by specifying an map overlay in the configuration page. The overlay box should contain a WMS getMap request but without the bbox, width or height specified. The PivotViewer can then display the collection with the correct map overlay whilst still allowing zooming, panning and scaling. In this example, house price data from the landregistry is overlayed with government flood risk information. The WMS overlay is:

    The query used to generate the collection is:

    PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
    PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> 
    PREFIX lrppi: <http://landregistry.data.gov.uk/def/ppi/> 
    PREFIX skos: <http://www.w3.org/2004/02/skos/core#> 
    PREFIX lrcommon: <http://landregistry.data.gov.uk/def/common/> 
    SELECT ?transx as ?href ?name ?paon ?saon ?street ?town ?county ?postcode ?amount ?date ?newBuild ?propertyType ?addr as ?address ?postcode as ?location where
       SERVICE <http://landregistry.data.gov.uk/landregistry/sparql>
    ?transx lrppi:transactionId ?name .
    ?transx lrppi:pricePaid ?amount . 
    ?transx lrppi:transactionDate ?date . 
    ?transx lrppi:propertyAddress ?addr. 
    ?transx lrppi:newBuild ?newBuild . 
    ?transx lrppi:propertyType ?propertyTypeURI . 
    ?propertyTypeURI <http://www.w3.org/2000/01/rdf-schema#label> ?propertyType . 
    ?addr lrcommon:town "BOSTON"^^xsd:string . 
    ?addr lrcommon:postcode ?postcode . 
    OPTIONAL {?addr lrcommon:county ?county .} 
    OPTIONAL {?addr lrcommon:paon ?paon .} 
    OPTIONAL {?addr lrcommon:saon ?saon .} 
    OPTIONAL {?addr lrcommon:street ?street .} 
    OPTIONAL {?addr lrcommon:town ?town .} 
    limit 100

    Query Results Restricted by Geometry

    The result set from a sparql query can be filtered so only those results that fall within a specified geometric are are returned. HtmlPivotViewer? can display the result on a map with the filter area superimposed on the map. To do this, the filter geometry must be included in the result set as the reserved query variable ?geometry. For example,

    ## Find items of each type
    ## whose coordinates fall within a bounded
    ## Geometry box shape: Polygon
    SELECT ?href
           ?name as ?Label
    FROM <http://linkedgeodata.org>
        ?href <http://www.w3.org/1999/02/22-rdf-syntax-ns#type>  ?f ;
              <http://www.w3.org/2000/01/rdf-schema#label>       ?name ;
              <http://www.w3.org/2003/01/geo/wgs84_pos#geometry> ?p ;
              geo:lat ?latitude ;
              geo:long ?longitude .
        ?f <http://www.w3.org/2000/01/rdf-schema#label> ?type .
        values ?geometry {"POLYGON((0.3412 43.5141, 0.3412 48.0141, 9.3412 48.0141, 9.3412 43.5141, 0.3412 43.5141))"} .
              "POLYGON((0.3412 43.5141, 0.3412 48.0141, 9.3412 48.0141, 9.3412 43.5141, 0.3412 43.5141))"
        FILTER langMatches( lang(?name), "EN" ) .
    LIMIT 1500

    Location Results that Represent an Area

    The items in the following query are associated with a geographical area rather than a single point. If the query results only include areas then the items will not be plotted on a map. A specific point is required for plotting the map marker so in this case a single point within the area has been found using a simple calculation to get a latitude and logitude. When the item is selected the actual area is highlighted.

    PREFIX naija-grid: <http://kingsley.idehen.net/DAV/home/danielhm/Public/grid3/grid-nigeria.ttl#>
      ?geometry as ?area   
      (bif:st_xMin(bif:st_ewkt_read(?geometry)) + bif:st_xMax(bif:st_ewkt_read(?geometry)))/2 as ?longitude 
      (bif:st_yMax(bif:st_ewkt_read(?geometry)) + bif:st_yMin(bif:st_ewkt_read(?geometry)))/2 as ?latitude 
    FROM <http://kingsley.idehen.net/public_home/kidehen/Public/Linked%20Data%20Documents/geospatial/grid-nigeria.ttl>
    WHERE {
              ?href a schema:Place; 
              naija-grid:hasState ?state ;
              naija-grid:hasLga ?lga ;
              naija-grid:hasWardCode ?wardCode ;
              geo:polygon ?geometry
    LIMIT 50

    Further examples can be found in this tutorial.

    Timeline View

    In the image below the set of Nigerian states has been plotted on timeline using the Founding Date facet.

    The query used to generate the collection is:

    select distinct ?s1 as ?href
                     ?s7 as ?name
                     ?s7 as ?state
                     ?s2 as ?leader
                     ?s6 as ?topic_of
                     ?s3 as ?longitude
                     ?s4 as ?latitude
                     ?s8 as ?description
                     ?s9 as ?founding_date
                     ?s10 as ?image
    {  SERVICE <http://dbpedia.org/sparql> 
        ?s1 a <http://dbpedia.org/class/yago/StatesOfNigeria> .
        ?s1 <http://dbpedia.org/property/leaderName> ?s2 .
        ?s1 <http://www.w3.org/2003/01/geo/wgs84_pos#long> ?s3 .
        ?s1 <http://www.w3.org/2003/01/geo/wgs84_pos#lat> ?s4 .
        optional {?s1 <http://purl.org/dc/terms/subject> ?s6} .
        optional {?s1 <http://xmlns.com/foaf/0.1/name> ?s7} .
        optional {?s1 <http://dbpedia.org/ontology/abstract> ?s8} .
        optional {?s1 <http://dbpedia.org/ontology/foundingDate> ?s9} .
        ?s1 <http://xmlns.com/foaf/0.1/depiction> ?s10 .
        filter (isIRI(?s2))

    HtmlPivotViewer looks for datetime facets to plot items on a timeline. If there is more than one datetime facet then the facet to use can be selected from a dropdown box. The collection can be filtered and items selected in the same was as for the other views.

    Google Translation

    The HtmlPivotViewer can be configured to allow automatic Google Translation of the page. This feature can be enabled by a dba from the HtmlPivotViewer configuration page. In the Virtuoso conductor select the System Admin tab and the The Packages sub menu. Scroll down the list of packages until you get to html5pivotviewer. Click on the Configure link in the right hand column. This will open up the HtmlPivotViewer configuration page.

    Once enabled, the Google Translation drop down box is added to the page and the translation destination language can be selected. The translation source language will be detected as English. However, if you know that the data in your collection is predominantly in a different language then you can override the detected source language also from the Html5PivotViewer configuration page.

    Cross Domain Collection Browsing

    As well as displaying collections from the same site as itself, HtmlPivotViewer can display remote collections, provided that the collection publisher has granted the necessary access.

    Cross-origin resource sharing (CORS) allows Javascript on a web page to make XMLHttpRequests? to another domain. If CORS is not enabled, cross domain requests are not allowed. CORS defines a way in which the browser and the server can interact to determine whether or not to allow the cross-origin request.[2] It is more powerful than only allowing same-origin requests, but it is more secure than simply allowing all such cross-origin requests.

    The security model in modern browsers does not typically allow sharing of resources across domains. Cross Origin Resource Sharing (CORS) provides a mechanism for allowing javascript to communicate using XMLHttpRequests? to a server on a different domain which otherwise would not be allowed.

    To view a collection served from a different site to that serving HtmlPivotViewer, CORS must be set up on the server hosting the collection. You may need to contact the system administrator for the server to ensure this is the case. If the collection is being generated by another Virtuoso server, the DeepZoomCache? Virtual Directory must have a CORS rule set up to allow access across domains. See the Virtuoso Tips and Tricks Guide.

    HtmlPivotViewer - ACL Support

    HtmlPivotViewer now includes support for integration with the Virtuoso Authentication Layer (VAL). VAL provides an internal Virtuoso API for handling authentication in Virtuoso and provides a framework for setting up access control lists (ACL). This new feature can be used to manage access to HtmlPivotViewer. Use of this feature is dependant on the VAL VAD which can be downloaded from the Virtuoso downloads page.

    If the VAL VAD is not present when HtmlPivotViewer is installed then the feature will not be enabled. HtmlPivotViewer will be installed and anyone can use it.

    If the VAL VAD is subsequently installed ACL support will then be available but the default scopes etc. will have to be set up by hand. See HtmlPivotViewer - Configuring Support for Access Control Lists. However, the necessity to do any configuration by hand can be avoided by simply reinstalling the HtmlPivotViewer VAD after VAL has been installed.

    Once the VAL VAD is installed the name of the currently logged in user or links to login and logout pages are displayed in the top right hand corner of the HtmlPivotViewer page.

    If the VAL VAD has already been installed when HtmlPivotViewer is installed then the default configuration will be setup automatically at install time. However, by default the ACL will not be enabled until the box is checked to Enable HtmlPivotViewer ACLs in the VAL VAD configuration page. The default ACL requires that a user is logged in before they can use HtmlPivotViewer.

    Allow Users to Request Access to HtmlPivotViewer

    An additional feature of the VAL framework is that a user denied access to a resource can automatically request access from the resource owner. To make use of this feature an owner must be defined for the resource, in this case the HtmlPivotViewer or <urn:virtuoso:access:pivotviewer>. If a user is denied access then an email is sent to the owner requesting that they are added to the list of allowed users. Defining the owner of the resource can be easily done using the VAL API. In this example the owner is 'dba', the database administrator. Executing this call in isql or the sql editor in the Virtuoso Conductor will enable the feature.
    VAL.DBA.set_resource_ownership (
       serviceId=>VAL.DBA.user_personal_uri ('dba')