History of Virtuoso
Virtuoso is the offspring of a genealogy tree comprising Universal Data Access Middleware from
Universal Data Access Middleware
In 1992 Kingsley Idehen departed Unisys (in the United Kingdom) to set up a consulting firm called PAL Consulting Ltd.
(which stood for Products, Applications, and Languages consulting).
His initial aim was to set up a consulting firm that specialized in data access middleware and associated integration services; a natural progression since he possessed significant knowledge of all the major RDBMS engines (at the time), and every first generation data access middleware product.
Unfortunately, the competitive realities of partnering with some early middleware vendors forced him change course and defend his vision and passion for data access middleware.
The net effect of this strategic change in direction was the creation of
In the early 90's,
By 1997 the company's initial portfolio of platform independent ODBC specific data access middleware had grown to include drivers for JDBC, and OLE-DB; all available in a variety of architectural formats (Single-Tier and Multi-Tier).
KublIn early 1994, Orri Erling was tasked with overseeing the porting of an existing DBMS known as VIA/DRE to ten different Unix platforms and all versions of Windows then-extant. The portation effort was successful, but the performance of the ports turned out to be inadequate for the network management application for which they had been commissioned. While managing this effort, Orri was also developing a portable distributed computing platform for C/C++ that provided asynchronous RPC?, streaming?, serialization of C and C++ data structures?, persistent? and distributed objects?, etc.
The net effect of the VIA/DRE experience, combined with the completion of the Distributed Computing Platform, spurred Orri toward a new goal. He embarked on a scientific pursuit, pushing the boundaries of SQL, aiming to achieve the lofty goal of the fastest OLTP-oriented DBMS engine that could possibly be written. This effort bore fruit, in Kubl.
While working on Kubl, Orri was also engaged with Infosto Group, Finnish publisher of Keltainen Porssi, a "Consumer to Consumer" ("C2C") national marketplace journal. He was tasked with technical oversight of Infosto's participation in the United Nations Trade Point initiative. Trade Point's aim was to help network small businesses worldwide via an online marketplace on the then-nascent Web. This effort became the first full-blown application of the Kubl DBMS Engine, which by then was a client-server RDBMS core with basic SQL, transactions, and a stored procedure language. This project was soon followed by a web version of Keltainen Porssi, which quickly gained 500.000 registered users out of a demographic of only 5 million. As Finland of the mid 90s still had relatively little household Internet use, the market penetration of this effort was remarkable.
Virtuoso Project Commencement
In early 1998 Kingsley Idehen, Founder & CEO of
The extensive search and evaluation process lead Kingsley to Kubl, and ultimately Orri.
Kingsley pitched the
Virtual Database Engine Development
As a vendor of high-performance ODBC Drivers covering all major backend DBMS engines,
IObox (one of the earliest hosted internet services) was one of the web applications/services built using Virtuoso.
It served up to half a million registered users from a single Sun SPARC 300MHz server with 32G of disk and catered for about 3000 concurrent sessions.
With the addition of more application logic in PHP, the data center became larger and the end user count reached 1 Million.
Telefonica eventually acquired IObox for about $300M in 2001.
Transition to Virtual DBMS for SQL and XML
In early 1999, XML started to emerge as the preferred candidate for standardizing data representation for data access, protocol definition, and data modeling.
Virtuoso also supported shredded storage of XML for a short while but this was later dropped in favor of document-style XML storage and XML specific Free Text indexing?.
By exposing native and third party SQL Data as XML based views hosted by the Virtuoso Database engine, it was clear to
In addition to its sophisticated XML functionality, Virtuoso also possesses its own dynamic web page generation language called VSP (Virtuoso Server Pages). The language enables the development of dynamic web pages using Virtuoso's SQL procedure language interspersed in HTML (as is the case with PHP, ASP, JSP, and others). It also includes a declarative XML based dynamic web page generation language called VSPX, which is conceptually similar to ASP.NET. A key advantage that VSP/VSPX pages bring to bear is the fact that SQL data access occurs in-process, as long as the tables being accessed are stored in Virtuoso.
Virtuoso the Universal Server
By 2000-2001, SOAP? (Simple Object Access Protocol) started to gain prominence as the foundation protocol of what is known today as the WS-* protocol stack. This protocol stack catalyzed the nascent Web Services paradigm (including Service Oriented Architecture and ESB derivative paradigms). As Virtuoso already possessed DBMS hosted (in-built) HTTP/WebDAV support (its web server functionality), and a very powerful SQL Stored Procedures language, it was a very natural progression for the product to expose functionality that would simplify comprehension and exploitation of the burgeoning Service Oriented Architecture (SOA) paradigm. The following features resulted:
- Execution of native or third party SQL Stored Procedures over HTTP
- Execution of native or third party SQL Stored Procedures via SOAP (by publishing SQL Stored Procedures as WSDL and SOAP compliant Web Services)
- Proxy Generation for third party Web Services
- Exposure of saved SQL, SQL-XML, XQuery, and XPath queries as Web Services
Orri's Lisp and AI background provided Virtuoso with a rich programming language endowed with runtime data-typing and self-hosted compilation making the incorporation of SOAP within an already powerful technology cocktail a triviality. In addition, the Database / Data-Access Middleware essence of Virtuoso synergistically extended the SOAP functionality to include Stored Procedures hosted in any third party ODBC accessible DBMS engine. Over time Virtuoso has continued to implement relevant portions of the WS-* protocol stack as and when they are published. It had one of the very first implementations of WS-Security and till this very day the only DBMS hosted implementation of the Business Process Execution Language for Web Services (BPEL nee. BPEL4WS).
By the summer of 2001, Virtuoso underwent a major engine-rewrite that re-galvanized an erstwhile dormant aspect of Virtuoso, namely its Object-Relational DBMS functionality. Outputs from this major development effort include:
- DBMS hosting of the Java, Microsoft .Net CLR, and Mono runtime environments
- User Defined Type (UDT) support with implementation in SQL, Java or .net
- Use of UDTs for abstracting web services, generating a UDT from a WSDL.
- SQL Stored Procedures extensibility via code associated with hosted runtimes
- Dynamic Language & Web page hosting for PHP, ASP.NET, JSP, Python, Perl, and Ruby (a recent addition)
- Procedure Views (Table Valued Functions in SQL Server and Table Functions in Oracle)
- Improved Cost based Distributed Query Optimizer for handling heterogeneous SQL joins
- Bi-Directional Transaction Replication
- XA-based 2-Phase Commit for Distributed Transactions
The combination of SQL-ORDBMS, HTTP/WebDAV, Web Services Platform for SOA, and other functionality realms covered by Virtuoso, led to fundamental incompatibility between the product moniker "Virtual Database" and the actual product feature set. Re-branding Virtuoso as a ?Universal Server? ultimately alleviated the aforementioned product-branding challenge; a single server product offering that implements a plethora of industry-standard protocols.
From a traditional marketing and positioning point of view Virtuoso in its Universal Server is a complete renegade by virtue of the fact that it straddles a number of functionality realms concurrently. At the same time, a closer look at the product architecture and its evolution , unveils deep understanding and anticipation of the platform requirements that will ultimately define next generation Intranet/Internet/Extranet (SOA, Web 2.0, Semantic Web/Data Web, and beyond) solutions. This is all the more so as these solutions will demand loose coupling of application logic, separation of data access & data storage, with an inherent need for collaboration driven integration that comes naturally to Virtuoso.
Virtuoso and the Data Space Frontier
It is no secret that Virtuoso is a product way ahead of its time.
- Web Application Framework
- Weblog Publishing Platform
- RSS/Atom/RDF Feed Aggregator
- Photo Sharing system
- Discussion Server
- Wiki Engine
- BPEL Process Manager (application layer above the in-built BPEL core)
- Unified Storage (that includes automatic metadata extraction and resource classification using RDFS and OWL)
- Social Networking Framework
As was the case when Virtuoso transitioned from a Virtual Database engine to a Universal Server that included Virtual Database functionality, the suite of Virtuoso applications posed a number of marketing related product branding challenges. Ultimately it was decided that the Universal Server would be packaged as a platform solution (workbench of sorts) while the applications would be packaged as discrete parts of a collective DataSpace (since all of the data and application logic that expressed the application behaviors resided in a Virtuoso Database).
Having completed a slew of unreleased Virtuoso applications (as enumerated above), it was then decided in late 2005 that the time was right for finalizing engineering work that would unveil and revisit a deferred development effort aimed at the "Semantic Web".
There emergence of a definitive query language for RDF Data Management systems called SPARQL, accompanied by a SOAP based protocol also called SPARQL, provided
A large part of Virtuoso was released in open-source form (dual-license mode similar to MySQL) in April 2006 with the following strategic goals in mind:
- Developing and Fostering community around Virtuoso
- Making a significant contribution that potentially accelerates comprehension and exploitation of the "Semantic Web"/"Data Web" vision
- Providing technology in a form palatable to early adopters and the extensive RDF research community
- Quicker product release cycles
- Bury the "Best-Kept Secret" and generally enigmatic status of Virtuoso
CategoryMarketing CategoryCorporateComms? CategoryWebSite CategoryVirtuoso CategoryOpenSource