• Topic
  • Discussion
    • VOS.SPARQLSponger(1.1) -- DAVWikiAdmin? , 2017-06-13 05:41:51 Edit
    [TXT]
    [PLAIN]
    [DOCBOOK]
    WebDAV System Administrator 2017-06-13 05:41:51

    An Introduction to the Virtuoso Sponger

    What is the Virtuoso Sponger?

    The Sponger is a piece of built-in Virtuoso SPARQL Processor middleware, for extracting RDF "on the fly" from non-RDF Web Data Sources.

    Why Is It Important?

    A majority of the world's data naturally resides in non-RDF form at the current time. The "Sponger" is middleware that accelerates the bootstrap of the Semantic Data Web by unobtrusively generating RDF from non-RDF data sources.

    How Does It Work?

    When an RDF-aware client requests data from a network-accessible resource via the Sponger, the following events occur:

    1. A request is made for data in RDF form, and if RDF is returned, no further transformation happens -- the RDF Entities are returned to the client
    2. If RDF isn't returned, then the Sponger passes the data through a Metadata Extraction Pipeline process (using Metadata Extractors)
    3. The extracted data is transformed to RDF via a Mapping Pipeline process (RDF is extracted by ontology matching and mapping) that generates RDF Entities (instance data)
    4. RDF Entities are returned to the client

    The imported data forms a local cache with invalidation rules conforming to those of traditional HTTP clients (Web Browsers). That is to say, expiration time is determined based on subsequent data fetches of the same resource (note: the first data load will record the 'expires' header) with current time compared to expiration time stored in the local cache. If HTTP 'expires' header data isn't returned by the source data server, then the Sponger will derive it's own invalidation time frame by evaluating the 'date' header and 'last-modified' HTTP headers. Irrespective of path taken, local cache invalidation is driven by an assessment of current time relative to recorded expiration time.

    Sponger Architecture

    Architecturally, the Sponger is comprised of Cartridges which are themselves comprised of RDF Extractors (RDFizers) and Ontology (Schema) Mappers.

    The Schema Mappers are typically written with XSLT (e.g., GRDDL and other OpenLink Mapping Schemes) or Virtuoso PL. The Metadata Extractors may be developed in Virtuoso PL, C/C++, Java, or any other language that can be integrated into Virtuoso via its server extension APIs.

    Sponger Components

    Web Service Interface

    The Sponger can be used also via a Virtuoso built-in REST style Web Service, through the Proxy endpoint of any Virtuoso installation.

    Note: The cartridges_filesystem.vad must be installed for the actual extraction and mapping to occur.

    The Sponger is very much like an implementation of cURL, exposed as a built-in Virtuoso Web Service (so you can interact with it as you do with Triples).

    Packaging & Deliverables

    The RDF Cartridges (mappers and extractors) are packaged as Virtuoso VAD packages, easily installed via Virtuoso's ISQL interface or the browser-based Virtuoso Conductor.

    Simple Usage Examples

    Enter the following URIs into 3rd Party RDF Client Application or Service:

    Related

    CategoryRDF CategoryVirtuoso