%META:TOPICPARENT{name="VOSBuild"}% ---+How to Install Virtuoso Open Source (VOS) on Fedora Core Linux %TOC% ---++Starting Point We start from a fresh Fedora Core Linux (14) installation. ---+++Configuring the OS 1 As a baseline, we favor using sudo to run certain commands as the privileged root user, set up the instance for access via ssh and configure the firewall to permit remote access to ssh and Virtuoso: bash$ su bash# yum update bash# yum install openssh-server sudo bash# visudo bash# /etc/init.d/sshd start bash# chkconfig sshd on bash# setup 1 If you wish to access Virtuoso remotely, you need to open ports: * 8890/tcp - the web interface * 1111/tcp (optional) - the iSQL/ODBC port 1 To this end, we run setup from a terminal command line. ---++Installing Virtuoso Fedora carries pre-built binary packages of Virtuoso, although they only give a minimalist server installation; you have the option of installing Virtuoso using these packages, or if a newer version is available upstream or you want to experience more of Virtuoso, you can build from source directly. ---+++Installing Virtuoso with Fedora Packages 1 Fedora ship three packages for Virtuoso, tailored for minimalist usage patterns: [tim@fedora ~]$ yum search virtuoso Loaded plugins: langpacks, presto, refresh-packagekit [...] ============================== Matched: virtuoso =============================== redland-virtuoso.i686 : Virtuoso storage support for Redland virtuoso-opensource.i686 : A high-performance object-relational SQL database virtuoso-opensource-utils.i686 : Utilities 1 The packages contain: * virtuoso-opensource -- the bare minimum to run the Virtuoso server with a default config file * virtuoso-opensource-utils -- the isql command for commandline interaction and the virt_mail SMTP handler * redland-virtuoso -- a backend to use Virtuoso for storage behind the Redland RDF libraries 1 For the purposes at hand, we install the first two packages: [tim@fedora ~]$ sudo yum install redland-virtuoso virtuoso-opensource{,-utils} [sudo] password for tim: Loaded plugins: langpacks, presto, refresh-packagekit Adding en_GB to language list Setting up Install Process Resolving Dependencies [...] Dependencies Resolved ================================================================================ Package Arch Version Repository Size ================================================================================ Installing: redland-virtuoso i686 1.0.12-1.fc14 updates 26 k virtuoso-opensource i686 6.1.2-1.fc14 fedora 3.8 M virtuoso-opensource-utils i686 6.1.2-1.fc14 fedora 99 k Installing for dependencies: libiodbc i686 3.52.7-1.fc12 fedora 163 k Transaction Summary ================================================================================ Install 4 Package(s) Total download size: 4.0 M Installed size: 13 M [tim@fedora ~]$ ---++++Starting an Instance 1 Virtuoso instances center around a configuration file, nominally called virtuoso.ini, so typically you copy this file into a directory of its own and the first time Virtuoso runs against it, it creates the empty database (virtuoso.db) and other support files alongside. 1 Fedora have packaged virtuoso.ini in two locations; one, in /etc/virtuoso/, for future reference; and a symbolic link in /var/lib/virtuoso/db/virtuoso.ini so the instance can be run under /var/lib/ where it belongs. 1 If you wish to run virtuoso as a non-root user (for security reasons), you should change the ownership on /var/lib/virtuoso/db/ accordingly: [tim@fedora db]$ cd /var/lib/virtuoso/db [tim@fedora db]$ sudo chown tim . [sudo] password for tim: [tim@fedora db]$ ls -la total 8 drwxr-xr-x. 2 tim root 4096 May 18 12:55 . drwxr-xr-x. 3 root root 4096 May 18 12:55 .. lrwxrwxrwx. 1 root root 37 May 18 12:55 virtuoso.ini -> ../../../../etc/virtuoso/virtuoso.ini 1 To start the instance using the default config file: [tim@fedora db]$ cd /var/lib/virtuoso/db [tim@fedora db]$ virtuoso-t -df Wed May 18 2011 16:33:22 INFO: { Loading plugin 1: Type `plain', file `wikiv' in `/usr/lib/virtuoso/hosting' 16:33:22 ERROR: FAILED plugin 1: Unable to locate file } 16:33:22 INFO: { Loading plugin 2: Type `plain', file `mediawiki' in `/usr/lib/virtuoso/hosting' 16:33:22 ERROR: FAILED plugin 2: Unable to locate file } 16:33:22 INFO: { Loading plugin 3: Type `plain', file `creolewiki' in `/usr/lib/virtuoso/hosting' 16:33:22 ERROR: FAILED plugin 3: Unable to locate file } 16:33:22 INFO: OpenLink Virtuoso Universal Server 16:33:22 INFO: Version 06.01.3127-pthreads for Linux as of Jul 22 2010 16:33:22 INFO: uses parts of OpenSSL, PCRE, Html Tidy 16:33:22 INFO: SQL Optimizer enabled (max 1000 layouts) 16:33:23 INFO: Compiler unit is timed at 0.003372 msec 16:33:26 INFO: Checkpoint started 16:33:26 INFO: Roll forward started 16:33:26 INFO: Roll forward complete 16:33:26 INFO: Checkpoint started 16:33:26 INFO: Checkpoint finished, log reused 16:33:27 INFO: Checkpoint started 16:33:27 INFO: Checkpoint finished, log reused 16:33:27 INFO: Checkpoint started 16:33:27 INFO: Checkpoint finished, log reused 16:33:27 INFO: Checkpoint started 16:33:27 INFO: Checkpoint finished, log reused 16:33:27 INFO: HTTP/WebDAV server online at 8890 16:33:27 INFO: Server online at 1111 (pid 2260) 1 From here, you can access the web-server by pointing your browser at http://hostname:8890/. 1 To access the server using the command-line isql tool, first you need to create an ODBC datasource; given this is a system-wide installation, edit /etc/odbc.ini to contain the following configuration: [Local Virtuoso] Driver=/usr/lib/virtodbc_r.so Address=127.0.0.1 Port=1111 UID=dba 1 Then you can run isql with an ODBC connect string on the command line: [tim@fedora ~]$ /usr/libexec/virtuoso/isql "DSN=Local Virtuoso;UID=dba;PWD=dba" OpenLink Interactive SQL (ODBC), version 0.9849b. Type HELP; for help and EXIT; to exit. SQL> tables; connected with connection string "DSN=Local Virtuoso;UID=dba;PWD=dba". Completed as: "DSN=Local Virtuoso;UID=dba;PWD=dba;HOST=127.0.0.1", length=49. Connected to OpenLink Virtuoso Driver: 06.01.3127 OpenLink Virtuoso ODBC Driver Showing SQLTables of tables like 'NULL.NULL.NULL', tabletype/colname like 'NULL' TABLE_QUALIFIER TABLE_OWNER TABLE_NAME TABLE_TYPE REMARKS VARCHAR VARCHAR VARCHAR VARCHAR VARCHAR _______________________________________________________________________________ DB DBA ADMIN_SESSION SYSTEM TABLE NULL DB DBA ADM_OPT_ARRAY_TO_RS_PVIEW SYSTEM TABLE NULL DB DBA ADM_XML_VIEWS SYSTEM TABLE NULL DB DBA ALL_COL_HIST SYSTEM TABLE NULL ---+++Building Virtuoso from Source ---++++Building from Upstream Source *Note*: Useful prior reading: please see the pages [[http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VOSBuild][VOSBuild]] and [[http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VOSMake][VOSMake]] in the VOS Wiki. ---+++++Dependencies Proceed to install the required build dependencies: [tim@fedora C]$ sudo yum install autoconf automake libtool flex bison gperf \ gawk m4 make openssl-devel readline-devel [sudo] password for tim: Loaded plugins: langpacks, presto, refresh-packagekit Adding en_GB to language list Setting up Install Process Package gawk-3.1.8-3.fc14.i686 already installed and latest version Package m4-1.4.14-1.fc14.i686 already installed and latest version Package 1:make-3.82-3.fc14.i686 already installed and latest version Resolving Dependencies --> Running transaction check ---> Package autoconf.noarch 0:2.66-2.fc14 set to be installed ... Installing : autoconf-2.66-2.fc14.noarch 28/31 Installing : automake-1.11.1-5.fc14.noarch 29/31 Installing : libtool-2.2.10-3.fc14.i686 30/31 Installing : openssl-devel-1.0.0d-1.fc14.i686 31/31 Installed: autoconf.noarch 0:2.66-2.fc14 automake.noarch 0:1.11.1-5.fc14 bison.i686 0:2.4.3-1.fc14 flex.i686 0:2.5.35-11.fc14 gperf.i686 0:3.0.4-2.fc14 libtool.i686 0:2.2.10-3.fc14 openssl-devel.i686 0:1.0.0d-1.fc14 ... ---+++++Unpacking Download the latest VOS archive (source tarball such as virtuoso-opensource-6.1.3.tar.gz) from [[http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VOSDownload#Latest%20source%20code%20on%20Github][GitHub]] and unpack it: [tim@fedora C]$ tar xvpfz virtuoso-opensource-6.1.3.tar.gz virtuoso-opensource-6.1.3/ virtuoso-opensource-6.1.3/bin/ virtuoso-opensource-6.1.3/bin/installer/ virtuoso-opensource-6.1.3/bin/installer/virtuoso.ini ... [tim@fedora C]$ cd virtuoso-opensource-6.1.3 [tim@fedora virtuoso-opensource-6.1.3]$ ---+++++Configuring 1 For the purposes of this demonstration, we use a simple configuration with no frills: [tim@fedora virtuoso-opensource-6.1.3]$ ./configure --prefix=/usr/local/ \ > --with-readline --program-transform-name="s/isql/isql-v/" checking for a BSD-compatible install... /usr/bin/install -c checking whether build environment is sane... yes checking for a thread-safe mkdir -p... /bin/mkdir -p checking for gawk... gawk ... 1 There are many other options that can be specified at this stage, to enable/disable the building of language-hosting plugins (Perl, Python, Ruby) or various VAD packages (sub-packages within Virtuoso); for more on these, read the relevant README files in the source distribution and run `./configure --help'. 1 Here we settle for asking for readline support, ie the ability to cursor-up/down and use typical readline key-combinations in the commandline isql tool once built. ---++++++Aside: VOS component locations 1 Virtuoso instances usually take the form of a designated directory somewhere in the filesystem, centred around a configuration file (nominally virtuoso.ini) which specifies operational parameters such as the filenames of database page-files to use, port-numbers to use, numbers of threads and buffers, etc. 1 In the above command, we specify a prefix of /usr/local to Virtuoso's ./configure script. This forms a base directory under which Virtuoso will create/use the following structure: * /usr/local/lib/ * various libraries for Sesame, JDBC, Jena, Hibernate, and hosting * /usr/local/bin * where the main executables (virtuoso-t, isql) live * /usr/local/share/virtuoso/vad * used to store VAD archives prior to installation in an instance * /usr/local/share/virtuoso/doc * local offline documentation * /usr/local/var/lib/virtuoso/db * the default location for a virtuoso instance * /usr/local/var/lib/virtuoso/vsp * various VSP scripts - the default homepage before the Conductor is installed 1 Other directory-structures are possible by specifying variously: * individual configure options such as --bindir, --libdir, etc. * --with-layout={GNU|Debian|Gentoo|RedHat|FreeBSD|opt|OpenLink}%BR% where each layout specifies a set of locations where the VADs, documentation (HTML and PDF), demo and default instance databases, and hosting libraries will be installed, each compatible with the packaging requirements of the named OS distribution ("opt" means a prefix of /opt and OpenLink is the default, detailed above). ---++++++Aside: isql name conflict Unfortunately, both unixODBC and Virtutoso provide a command `isql', used to access data-sources from the command-line (in unixODBC, an ODBC datasource; in Virtuoso, the SQL interface). In order to allow both packages to coexist, it is reasonable to rename Virutoso's isql command to isql-v. Given that we are installing Virtuoso from source to a different prefix (/usr/local), it is not absolutely necessary to do the same here, but to avoid having to specify different PATH considerations and make life easier, the configure command above reflects this as well. ---+++++Building and Installing 1 Having chosen some configure options, run make: [tim@fedora virtuoso-opensource-6.1.3]$ nice make Making all in . make[1]: Entering directory `/home/tim/C/virtuoso-opensource-6.1.3' make[1]: Nothing to be done for `all-am'. make[1]: Leaving directory `/home/tim/C/virtuoso-opensource-6.1.3' Making all in docsrc make[1]: Entering directory `/home/tim/C/virtuoso-opensource-6.1.3/docsrc' ... make[2]: Nothing to be done for `all-am'. make[2]: Leaving directory `/home/tim/C/virtuoso-opensource-6.1.3/appsrc' make[1]: Leaving directory `/home/tim/C/virtuoso-opensource-6.1.3/appsrc' [tim@fedora virtuoso-opensource-6.1.3]$ 1 Optionally, you can run `make check' to run the test-suite (beware, it takes some hours and multiple gigabytes of disk-space). [tim@fedora virtuoso-opensource-6.1.3]$ nice make check 1 Install it to the directory-structure chosen above: [tim@fedora virtuoso-opensource-6.1.3]$ sudo make install [sudo] password for tim: Making install in . make[1]: Entering directory `/home/tim/C/virtuoso-opensource-6.1.3' make[2]: Entering directory `/home/tim/C/virtuoso-opensource-6.1.3' make[2]: Nothing to be done for `install-exec-am'. test -z "/usr/local/share/virtuoso/doc" || /bin/mkdir -p "/usr/local/share/virtuoso/doc" /usr/bin/install -c -m 644 AUTHORS COPYING CREDITS INSTALL ... ---++Getting Started 1 Take a copy of the default virtuoso.ini and store it safely in case of making erroneous changes. 1 Change into the default database directory: [tim@fedora virtuoso-opensource-6.1.3]$ cd /usr/local/var/lib/virtuoso/db/ [tim@fedora db]$ ls virtuoso.ini virtuoso.ini.sample * (At this point, if you wish to run virtuoso as a non-root user you should change the ownership on this directory, e.g. sudo chown -R tim . . Virtuoso only requires root if you wish to use it as a webserver listening on port 80; by default it listens only on high ports.) 1 Start the server: [tim@fedora db]$ virtuoso-t -df Thu May 19 2011 01:33:03 INFO: { Loading plugin 1: Type `plain', file `wikiv' in `/usr/local/lib/virtuoso/hosting' 01:33:03 INFO: WikiV version 0.6 from OpenLink Software 01:33:03 INFO: Support functions for WikiV collaboration tool 01:33:03 INFO: SUCCESS plugin 1: loaded from /usr/local/lib/virtuoso/hosting/wikiv.so } 01:33:03 INFO: { Loading plugin 2: Type `plain', file `mediawiki' in `/usr/local/lib/virtuoso/hosting' ... 01:33:17 INFO: Checkpoint finished, log reused 01:33:17 INFO: Checkpoint started 01:33:18 INFO: Checkpoint finished, log reused 01:33:18 INFO: HTTP/WebDAV server online at 8890 01:33:18 INFO: Server online at 1111 (pid 27873) * Here we see the server running in foreground mode with maximum debugging information. Because it's the first time this instance has been run, Virtuoso has created a bunch of database files (virtuoso.db, virtuoso.trx (the transaction log), etc) and installed the Conductor VAD package into the database. * If you close your terminal while this command is running, Virtuoso will stop. To run it as a daemon instead, omit the "-df" options. You can still see most of what's happening by running tail -f virtuoso.log. ---+++Using Virtuoso ---++++Web-based UI You can access the Conductor menu by pointing your web-browser at http://localhost:8890/conductor/ (substitute hostname as appropriate). From the Conductor you can manage users and automated backups, install VAD packages, execute SQL commands in a web-based iSQL tool, configure the RDF Sponger and loads more. There are two system users of immediate importance: dba (the relational data administrative account) and dav (the WebDAV adminstrative account). By default both these accounts have their passwords set the same as the respective usernames; it is highly advised that you change these as soon as possible for security reasons. ---++++Command-line Virtuoso provides a SQL/ODBC listener on port 1111/tcp. You can connect directly to this and execute SQL statements using the isql tool: [tim@fedora ~]$ /usr/local/bin/isql-v OpenLink Interactive SQL (Virtuoso), version 0.9849b. Type HELP; for help and EXIT; to exit. SQL> tables; Connected to OpenLink Virtuoso Driver: 06.01.3127 OpenLink Virtuoso ODBC Driver Showing SQLTables of tables like 'NULL.NULL.NULL', tabletype/colname like 'NULL' TABLE_QUALIFIER TABLE_OWNER TABLE_NAME TABLE_TYPE REMARKS VARCHAR VARCHAR VARCHAR VARCHAR VARCHAR _______________________________________________________________________________ DB DBA ADMIN_SESSION SYSTEM TABLE NULL DB DBA ADM_OPT_ARRAY_TO_RS_PVIEW SYSTEM TABLE NULL [...] ---++Resource Usage Virtuoso is highly configurable: the same executables and packages will service anything from a small "lite-mode" installation with a database of 10Mb up to a huge multi-user enterprise installation with terabytes of data. The defaults with Virtuoso Open-Source give a 160MB process size in memory, about 29MB database and total 237MB footprint on disk; this has 20 threads running for database and/or web-server use. The wiki document [[http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VOSScale][VOSScale]] describes how to scale Virtuoso in some detail. ---++Related * [[VOSUbuntuNotes][Ubuntu]] * [[VOSDebianNotes][Debian]] * [[VOSCentosNotes][CentOS]]