---+Virtuoso Sponger Query Language Cartridges

%TOC%

---++What Are They?

Most of the Sponger's extractor cartridges work on a pool of data, such as an HTML webpage containing metadata, which are extracted into triples. Some of the cartridges make API calls to upstream web services to retrieve the relevant pool of data first.

Query Language cartridges are for situations where the data arises from a query executed on the upstream site, ie the data being sponged is the result-set of another query. 

---++Why?

Maybe the upstream server holds control over, or offers limited or restricted access to, the data; maybe you have a pre-existing application that uses the given language to express its queries.

---++How?

Virtuoso's Sponger has 3 cartridges that support upstream queries:

---+++Freebase Query Editor (MQL)

Upstream query editor interface: http://www.freebase.com/queryeditor

Language: MQL, a means of expressing filters in JSON.

Sample:

<verbatim>
{
  "album": [{
    "id":    null,
    "limit": 2,
    "name":  null,
    "track": []
  }],
  "id":   null,
  "name": "The Police",
  "type": "/music/artist"
}
</verbatim>

Permalink to query URL: http://www.freebase.com/queryeditor?q=%7B%22album%22%3A%5B%7B%22id%22%3Anull%2C%22limit%22%3A2%2C%22name%22%3Anull%2C%22track%22%3A%5B%5D%7D%5D%2C%22id%22%3Anull%2C%22name%22%3A%22The%20Police%22%2C%22type%22%3A%22%2Fmusic%2Fartist%22%7D

---++++Cartridge Configuration Requirements

There are no special requirements; because it doesn't even need an app key to
use it, it should be enabled by default after installing the Cartridges VAD
package.


---++++Examples

   * The query: sports players born in Brooklyn: http://www.freebase.com/queryeditor?q=%5B%7B%22id%22%3Anull%2C%22name%22%3Anull%2C%22type%22%3A%22%2Fsports%2Fpro_athlete%22%2C%22sports_played_professionally%22%3A%5B%7B%22sport%22%3A%7B%22id%22%3A%22%2Fwikipedia%2Fru_id%2F30265%22%7D%7D%5D%2C%22%2Fpeople%2Fperson%2Fplace_of_birth%22%3A%7B%22id%22%3A%22%2Fen%2Fbrooklyn%22%7D%2C%22key%22%3A%5B%7B%22value%22%3Anull%2C%22namespace%22%3A%22%2Fauthority%2Fgoogle%2Fgoogle_plus%22%7D%5D%7D%5D&autorun=1
   * Data extracted and presented in HTML: http://uriburner.com/about/html/http://www.freebase.com/queryeditor?q=%5B%7B%22id%22%3Anull%2C%22name%22%3Anull%2C%22type%22%3A%22%2Fsports%2Fpro_athlete%22%2C%22sports_played_professionally%22%3A%5B%7B%22sport%22%3A%7B%22id%22%3A%22%2Fwikipedia%2Fru_id%2F30265%22%7D%7D%5D%2C%22%2Fpeople%2Fperson%2Fplace_of_birth%22%3A%7B%22id%22%3A%22%2Fen%2Fbrooklyn%22%7D%2C%22key%22%3A%5B%7B%22value%22%3Anull%2C%22namespace%22%3A%22%2Fauthority%2Fgoogle%2Fgoogle_plus%22%7D%5D%7D%5D&autorun=1
   * Data Explorer view: http://linkeddata.uriburner.com/ode/?uri=http://www.freebase.com/queryeditor?q=%5B%7B%22id%22%3Anull%2C%22name%22%3Anull%2C%22type%22%3A%22%2Fsports%2Fpro_athlete%22%2C%22sports_played_professionally%22%3A%5B%7B%22sport%22%3A%7B%22id%22%3A%22%2Fwikipedia%2Fru_id%2F30265%22%7D%7D%5D%2C%22%2Fpeople%2Fperson%2Fplace_of_birth%22%3A%7B%22id%22%3A%22%2Fen%2Fbrooklyn%22%7D%2C%22key%22%3A%5B%7B%22value%22%3Anull%2C%22namespace%22%3A%22%2Fauthority%2Fgoogle%2Fgoogle_plus%22%7D%5D%7D%5D&autorun=1


---+++Yahoo Query Language (YQL)

Upstream query editor interface: http://developer.yahoo.com/yql/console/

Language: a web-based SQL-style dialect

Sample:

<verbatim>
select * from local.search where query="sushi" and location="san francisco, ca"
</verbatim>

Permalink to query URL: http://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20local.search%20where%20query%3D%22sushi%22%20and%20location%3D%22san%20francisco%2C%20ca%22&diagnostics=true

---++++Cartridge Configuration Requirements

There are no special requirements; because it doesn't even need an app key to
use it, it should be enabled by default after installing the Cartridges VAD
package.


---++++Examples

   * The query: sushi restaurants in San Francisco: http://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20local.search%20where%20query%3D%22sushi%22%20and%20location%3D%22san%20francisco%2C%20ca%22&diagnostics=true
   * Data extracted and presented in HTML: http://uriburner.com/about/html/http://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20local.search%20where%20query%3D%22sushi%22%20and%20location%3D%22san%20francisco%2C%20ca%22&diagnostics=true
   * Data Explorer view: http://linkeddata.uriburner.com/ode/?uri=http://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20local.search%20where%20query%3D%22sushi%22%20and%20location%3D%22san%20francisco%2C%20ca%22&diagnostics=true

   * The query: http://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20geo.places%20where%20text%3D%22san%20francisco%2C%20ca%22&diagnostics=true&@Lookup@=&refresh=clean
   * Data extracted and presented in HTML: http://uriburner.com/about/html/http://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20geo.places%20where%20text%3D%22san%20francisco%2C%20ca%22&diagnostics=true&@Lookup@=&refresh=clean
   * Data Explorer view: http://linkeddata.uriburner.com/ode/?uri=http://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20geo.places%20where%20text%3D%22san%20francisco%2C%20ca%22&diagnostics=true&@Lookup@=&refresh=clean

   * The query: http://query.yahooapis.com/v1/public/yql?q=select%20woeid%20from%20geo.places%20where%20text%3D%22san%20francisco%2C%20ca%22&diagnostics=true&@Lookup@=&refresh=clean
   * Data extracted and presented in HTML: http://uriburner.com/about/html/http://query.yahooapis.com/v1/public/yql?q=select%20woeid%20from%20geo.places%20where%20text%3D%22san%20francisco%2C%20ca%22&diagnostics=true&@Lookup@=&refresh=clean
   * Data Explorer view: http://linkeddata.uriburner.com/ode/?uri=http://query.yahooapis.com/v1/public/yql?q=select%20woeid%20from%20geo.places%20where%20text%3D%22san%20francisco%2C%20ca%22&diagnostics=true&@Lookup@=&refresh=clean

   * The query: http://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20geo.places.parent%20where%20child_woeid%20in%20(select%20woeid%20from%20geo.places%20where%20text%3D%22san%20francisco%2C%20ca%22)&diagnostics=true&@Lookup@=&refresh=clean
   * Data extracted and presented in HTML: http://uriburner.com/about/html/http://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20geo.places.parent%20where%20child_woeid%20in%20(select%20woeid%20from%20geo.places%20where%20text%3D%22san%20francisco%2C%20ca%22)&diagnostics=true&@Lookup@=&refresh=clean
   * Data Explorer view: http://linkeddata.uriburner.com/ode/?uri=http://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20geo.places.parent%20where%20child_woeid%20in%20(select%20woeid%20from%20geo.places%20where%20text%3D%22san%20francisco%2C%20ca%22)&diagnostics=true&@Lookup@=&refresh=clean

   * The query: http://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20local.search%20where%20query%3D%22sushi%22%20and%20location%3D%22san%20francisco%2C%20ca%22&diagnostics=true&@Lookup@=&refresh=clean
   * Data extracted and presented in HTML: http://uriburner.com/about/html/http://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20local.search%20where%20query%3D%22sushi%22%20and%20location%3D%22san%20francisco%2C%20ca%22&diagnostics=true&@Lookup@=&refresh=clean
   * Data Explorer view: http://linkeddata.uriburner.com/ode/?uri=http://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20local.search%20where%20query%3D%22sushi%22%20and%20location%3D%22san%20francisco%2C%20ca%22&diagnostics=true&@Lookup@=&refresh=clean

   * The query: http://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20local.search%20where%20query%3D%22sushi%22%20and%20location%3D%22san%20francisco%2C%20ca%22%20and%20Rating.AverageRating%3D%224.5%22&diagnostics=true&@Lookup@=&refresh=clean
   * Data extracted and presented in HTML: http://uriburner.com/about/html/http://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20local.search%20where%20query%3D%22sushi%22%20and%20location%3D%22san%20francisco%2C%20ca%22%20and%20Rating.AverageRating%3D%224.5%22&diagnostics=true&@Lookup@=&refresh=clean
   * Data Explorer view: http://linkeddata.uriburner.com/ode/?uri=http://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20local.search%20where%20query%3D%22sushi%22%20and%20location%3D%22san%20francisco%2C%20ca%22%20and%20Rating.AverageRating%3D%224.5%22&diagnostics=true&@Lookup@=&refresh=clean

   * The query: http://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20upcoming.events%20where%20woeid%20in%20(select%20woeid%20from%20geo.places%20where%20text%3D%22North%20Beach%22)&diagnostics=true&@Lookup@=&refresh=clean
   * Data extracted and presented in HTML: http://uriburner.com/about/html/http://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20upcoming.events%20where%20woeid%20in%20(select%20woeid%20from%20geo.places%20where%20text%3D%22North%20Beach%22)&diagnostics=true&@Lookup@=&refresh=clean
   * Data Explorer view: http://linkeddata.uriburner.com/ode/?uri=http://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20upcoming.events%20where%20woeid%20in%20(select%20woeid%20from%20geo.places%20where%20text%3D%22North%20Beach%22)&diagnostics=true&@Lookup@=&refresh=clean

   * The query: http://query.yahooapis.com/v1/public/yql?q=select%20title%20from%20rss%20where%20url%3D%22http%3A%2F%2Frss.news.yahoo.com%2Frss%2Ftopstories%22&diagnostics=true&@Lookup@=&refresh=clean
   * Data extracted and presented in HTML: http://uriburner.com/about/html/http://query.yahooapis.com/v1/public/yql?q=select%20title%20from%20rss%20where%20url%3D%22http%3A%2F%2Frss.news.yahoo.com%2Frss%2Ftopstories%22&diagnostics=true&@Lookup@=&refresh=clean
   * Data Explorer view: http://linkeddata.uriburner.com/ode/?uri=http://query.yahooapis.com/v1/public/yql?q=select%20title%20from%20rss%20where%20url%3D%22http%3A%2F%2Frss.news.yahoo.com%2Frss%2Ftopstories%22&diagnostics=true&@Lookup@=&refresh=clean

   * The query: http://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20html%20where%20url%3D%22http%3A%2F%2Ffinance.yahoo.com%2Fq%3Fs%3Dyhoo%22%20and%20xpath%3D'%2F%2Fdiv%5B%40id%3D%22yfi_headlines%22%5D%2Fdiv%5B2%5D%2Ful%2Fli%2Fa'&diagnostics=true&@Lookup@=&refresh=clean
   * Data extracted and presented in HTML: http://uriburner.com/about/html/http://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20html%20where%20url%3D%22http%3A%2F%2Ffinance.yahoo.com%2Fq%3Fs%3Dyhoo%22%20and%20xpath%3D'%2F%2Fdiv%5B%40id%3D%22yfi_headlines%22%5D%2Fdiv%5B2%5D%2Ful%2Fli%2Fa'&diagnostics=true&@Lookup@=&refresh=clean
   * Data Explorer view: http://linkeddata.uriburner.com/ode/?uri=http://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20html%20where%20url%3D%22http%3A%2F%2Ffinance.yahoo.com%2Fq%3Fs%3Dyhoo%22%20and%20xpath%3D'%2F%2Fdiv%5B%40id%3D%22yfi_headlines%22%5D%2Fdiv%5B2%5D%2Ful%2Fli%2Fa'&diagnostics=true&@Lookup@=&refresh=clean

   * The query: http://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20rss%20where%20url%20in%20(select%20title%20from%20atom%20where%20url%3D%22http%3A%2F%2Fspreadsheets.google.com%2Ffeeds%2Flist%2Fpg_T0Mv3iBwIJoc82J1G8aQ%2Fod6%2Fpublic%2Fbasic%22)%20and%20description%20like%20%22%25Wall%20Street%25%22%20LIMIT%2010%20%7C%20unique(field%3D%22title%22)&diagnostics=true&@Lookup@=&refresh=clean&@Lookup@=&refresh=clean
   * Data extracted and presented in HTML: http://uriburner.com/about/html/http://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20rss%20where%20url%20in%20(select%20title%20from%20atom%20where%20url%3D%22http%3A%2F%2Fspreadsheets.google.com%2Ffeeds%2Flist%2Fpg_T0Mv3iBwIJoc82J1G8aQ%2Fod6%2Fpublic%2Fbasic%22)%20and%20description%20like%20%22%25Wall%20Street%25%22%20LIMIT%2010%20%7C%20unique(field%3D%22title%22)&diagnostics=true&@Lookup@=&refresh=clean&@Lookup@=&refresh=clean
   * Data Explorer view: http://linkeddata.uriburner.com/ode/?uri=http://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20rss%20where%20url%20in%20(select%20title%20from%20atom%20where%20url%3D%22http%3A%2F%2Fspreadsheets.google.com%2Ffeeds%2Flist%2Fpg_T0Mv3iBwIJoc82J1G8aQ%2Fod6%2Fpublic%2Fbasic%22)%20and%20description%20like%20%22%25Wall%20Street%25%22%20LIMIT%2010%20%7C%20unique(field%3D%22title%22)&diagnostics=true&@Lookup@=&refresh=clean&@Lookup@=&refresh=clean

   
---+++<nowiki>Wolfram|Alpha</nowiki>

Upstream query editor interface: http://www.wolframalpha.com/

Language: custom, pseudo-natural-language

Sample:

<verbatim>
airspeed of a laden swallow
</verbatim>

Permalink to query URL: http://www.wolframalpha.com/input/?i=airspeed+of+a+laden+swallow&dataset=

---++++Cartridge Configuration Requirements

Both the <nowiki>Wolfram|Alpha</nowiki> cartridge and meta-cartridge require an App-ID issued from the 
[[https://developer.wolframalpha.com/portal/myapps/][Wolfram Alpha developer dashboard]]. For a 
detailed overview, see the [[VirtCartConfigWolframAlpha][cartridge configuration instructions]].

---++++Examples

   * The query: [[http://www.wolframalpha.com/input/?i=airspeed+of+a+laden+swallow&dataset=][airspeed of a laden swallow]]
   * Data extracted and presented in HTML: http://uriburner.com/about/html/http://www.wolframalpha.com/input/?i=airspeed+of+a+laden+swallow&dataset=
   * Data Explorer view: http://linkeddata.uriburner.com/ode/?uri=http://www.wolframalpha.com/input/?i=airspeed+of+a+laden+swallow&dataset=

   * The query: [[http://www.wolframalpha.com/input/?i=What+is+the+weather+in+Boston%3F][what is the weather in Boston?]]
   * Data extracted and presented in HTML: http://uriburner.com/about/html/http://www.wolframalpha.com/input/?i=What+is+the+weather+in+Boston%3F
   * Data Explorer view: http://linkeddata.uriburner.com/ode/?uri=http://www.wolframalpha.com/input/?i=What+is+the+weather+in+Boston%3F

   * The query: [[http://www.wolframalpha.com/input/?i=Pb&dataset=][Pb]]
   * Data extracted and presented in HTML: http://uriburner.com/about/html/http://www.wolframalpha.com/input/?i=Pb&dataset=
   * Data Explorer view: http://linkeddata.uriburner.com/ode/?uri=http://www.wolframalpha.com/input/?i=Pb&dataset=

   * The query: [[http://www.wolframalpha.com/input/?i=play+sin%28440t%29][play sin(440t)]]
   * Data extracted and presented in HTML: http://uriburner.com/about/html/http://www.wolframalpha.com/input/?i=play+sin%28440t%29
   * Data Explorer view: http://linkeddata.uriburner.com/ode/?uri=http://www.wolframalpha.com/input/?i=play+sin%28440t%29


---+++Treasury.IO

Upstream query editor interface: http://treasury.io/

Language: a web-based SQL-style dialect

Sample:

<verbatim>
SELECT "table", "date", "year_month", "year", "month", "day", "weekday", 
"account", "account_raw", "is_total", "close_today", "open_today", "open_mo", 
"open_fy", "url"
FROM "t1" 
limit 10
</verbatim>

Permalink to query URL: http://api.treasury.io/cc7znvq/47d80ae900e04f2/sql/?q=SELECT%20%22table%22,%20%22date%22,%20%22year_month%22,%20%22year%22,%20%22month%22,%20%22day%22,%20%22weekday%22,%20%22account%22,%20%22account_raw%22,%20%22is_total%22,%20%22close_today%22,%20%22open_today%22,%20%22open_mo%22,%20%22open_fy%22,%20%22url%22%20FROM%20%22t1%22%20%20limit%2010

---++++Cartridge Configuration Requirements

There are no special requirements; because it doesn't even need an app key to
use it, it should be enabled by default after installing the Cartridges VAD
package.


---++++Examples

   * The query: Federal Tax Deposits: http://api.treasury.io/cc7znvq/47d80ae900e04f2/sql/?q=SELECT%20%22table%22,%20%22date%22,%20%22year_month%22,%20%22year%22,%20%22month%22,%20%22day%22,%20%22weekday%22,%20%22type%22,%20%22classification%22,%20%22classification_raw%22,%20%22is_total%22,%20%22today%22,%20%22mtd%22,%20%22fytd%22,%20%22url%22%20FROM%20%22t4%22%20%20limit%2020
   * Data extracted and presented in HTML: http://uriburner.com/about/html/http://api.treasury.io/cc7znvq/47d80ae900e04f2/sql/?q=SELECT%20%22table%22,%20%22date%22,%20%22year_month%22,%20%22year%22,%20%22month%22,%20%22day%22,%20%22weekday%22,%20%22type%22,%20%22classification%22,%20%22classification_raw%22,%20%22is_total%22,%20%22today%22,%20%22mtd%22,%20%22fytd%22,%20%22url%22%20FROM%20%22t4%22%20%20limit%2020
   * Data Explorer view: http://linkeddata.uriburner.com/ode/?uri=http://api.treasury.io/cc7znvq/47d80ae900e04f2/sql/?q=SELECT%20%22table%22,%20%22date%22,%20%22year_month%22,%20%22year%22,%20%22month%22,%20%22day%22,%20%22weekday%22,%20%22type%22,%20%22classification%22,%20%22classification_raw%22,%20%22is_total%22,%20%22today%22,%20%22mtd%22,%20%22fytd%22,%20%22url%22%20FROM%20%22t4%22%20%20limit%2020