Installation Guide

Kowari^TM is shipped in two forms, Kowari Lite^TM is shipped as binary files only or the full version as a source code download. If you have downloaded the binary files kowari-1.1.0.jar and itql-1.1.0.jar please proceed with Installing Java and then skip to Running a Kowari server. If you have obtained source code, you will need to build the binary files from a build environment by completing all sections in this document. All users should take note of the hints given in the document latest set of Release Notes.

Note - Throughout this document, $KOWARI_HOME is used to refer to the directory containing either the server or iTQL^TM shell binary JAR file, or the directory containing the source code. Where examples refer to $KOWARI_HOME below, you should substitute in this location.

In This Section

Installing Java

Checkout Setup

Build Targets

Generating Kowari javadoc

Generating the Server and the iTQL Shell

Running a Kowari Server

Kowari Configuration File

Kowari Command Line Options

Java System Properties

Running the Test Cases

Installing Java

Download a J2SE^TM 1.4.0 or above for your platform from http://java.sun.com/j2se/, and install it. Installation instructions for Windows^® and Linux^® operating systems are available. You should then check that the installation added the java commands to your path by typing:

$ java -version

You should get the following:

java version "1.4.2" Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.0-b28) Java HotSpot(TM) Client VM (build 1.4.2-b28, mixed mode)

If your shell reports that it cannot find the command, add <JAVA_HOME>/bin (where JAVA_HOME is the location where you installed J2SE to) to your path in the appropriate way for your shell.

Note. You must use J2SE 1.4.0 or higher for compiling and running Kowari.

Checkout Setup

After you've obtained a checkout of the Kowari Metastore^TM project, you'll need Add a JAVA_HOME environment variable pointing to the place that you have java installed to get your checkout working.

From a Windows command line:

$ set JAVA_HOME=C:\jdk1.4.2

The installation program should have set this setting automatically.

Under bash:

$ export JAVA_HOME=/usr/local/java

Note. $KOWARI_HOME refers to the place where you checked out the Kowari module from CVS. The version of the junit JAR file listed above may be different from the version included with your checkout.

Note - Under a Windows operating system you could set these environment variables in Control Panel -> System -> Advanced -> Environment Variables instead of in each shell. On most other versions of Windows you'll probably want to add these to your autoexec.bat file.

Build Targets

To get a list of the available targets in the build script, do the following:

Change to the Kowari^TM home directory:
$ cd $KOWARI_HOME
To list the targets on a system running a UNIX^® operating system:
$ ./build.sh -projecthelp

To list the targets on a system running a Windows^® operating system:

$ build -projecthelp

You can chain targets together instead of invoking ant multiple times. This saves the JVM startup time associated with each ant run. So for example, suppose that you made a change to the source, and wanted a clean build. You may want to bring the build back to an original state (no generated files), and generate all distributable files. Instead of two commands, you could type:

$ ./build.sh clean dist

Or, for systems running a Windows operating system:

$ build.bat clean dist

Generating Kowari javadoc

To generate the Kowari javadoc you'll need to perform the following steps:

Change to the Kowari home directory:
$ cd $KOWARI_HOME
Run the javadoc target.
For systems running a UNIX operating system:

$ ./build.sh javadoc

For systems running a Windows operating system:

$ build javadoc

You should then be able to view the Kowari javadoc.

Generating the Server and the iTQL Shell

To generate the server and iTQL Shell you'll need to perform the following steps:

Change to the Kowari home directory:
$ cd $KOWARI_HOME
The logging files are located in $KOWARI_HOME/conf and are named log4j-<component>.xml. To configure the logging system, you'll need to set the path to which logging information is sent. For example the default configuration for the iTQL component is:

<appender name="LOGFILE" class="org.apache.log4j.FileAppender">
<param name="File" value="./itql.log"/>
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d %-5p [%t] %C{2} - %m%n"/>
</layout>
</appender>

Which sends logging information to a log file named ./itql.log, which will create a logging file in the directory from where you run the interpreter. This may not be appropriate for your environment, so you may need to change this to a more sensible path.
Run the dist target.
For systems running a UNIX operating system:

$ ./build.sh dist

For systems running a Windows operating system:

$ build dist

This creates the two binary files called itql-1.1.0.jar and kowari-1.1.0.jar in $KOWARI_HOME/dist directory.
Run the executable (the version number for your installation may be different):
$ java -jar dist/itql-1.1.0.jar

If the iTQL Shell starts correctly, you get a prompt back. Enter the quit command at the prompt to exit the interpreter.

ITQL> quit ;

Running a Kowari Server

Executing an executable JAR file runs the server. To start Kowari you'll need to do the following:

Change to the directory (folder) into which you downloaded Kowari:
$ cd $KOWARI_HOME
Start the executable JAR:
$ java -jar kowari-1.1.0.jar.jar

You should see the following line when the server is starting, identifying the build number:

1 [main] INFO EmbeddedKowariServer - Kowari Metadata Store version 1.0 Build (v1.1.0)

Once you see a line similar to the following display in the console the server is ready to be used.

INFO EmbeddedKowariServer - Successfully started Kowari server at
rmi://localhost/server1 in directory /usr/local/kowari/server1

However, if the following message displays then the HTTP port is already occupied by another process. See the Command Line Options section for more information on how to change this.

ERROR EmbeddedKowariServer - java.net.BindException: Address already in use

If you see the following message, you may not have started the RMI registry correctly. As stated previously, see the Command Line Options section for more information on how to change this..

ERROR EmbeddedKowariServer - javax.naming.ServiceUnavailableException
[Root exception is java.rmi.ConnectException:
Connection refused to host: localhost; nested exception is:
java.net.ConnectException: Connection refused]
To verify your installation is working correctly open your web browser and enter the following URL:
http://localhost:8080

Note - Your HTTP port may be different if you have supplied a -p option.

Follow the links to the user documentation to learn more about using Kowari.

Kowari Configuration File

A default XML configuration file, packaged within the Kowari executable JAR file, configures how Kowari runs. This file is shown below.

<?xml version="1.0" encoding="UTF-8"?>



<TucanaConfig>


<ExternalConfigPaths>
<TucanaLogging>conf/log4j-tks.xml</TucanaLogging>
<WebDefault>conf/webdefault.xml</WebDefault>
</ExternalConfigPaths>





<Jetty>
<Listener>

<Port>8080</Port>
<MinThreads>5</MinThreads>
<MaxThreads>255</MaxThreads>
<MaxIdleTimeMs>60000</MaxIdleTimeMs>
<MaxReadTimeMs>60000</MaxReadTimeMs>
<LowResourcePersistTimeMs>5000</LowResourcePersistTimeMs>
</Listener>
</Jetty>


<ServerName>server1</ServerName>


<RMIPort>1099</RMIPort>


<PersistancePath>.</PersistancePath>


<TripleStoreImplementation>org.kowari.store.xa.XADatabaseImpl</TripleStoreImplementation>


<PersistentNodePoolClass>org.kowari.store.nodepool.xa.XANodePoolFactory
</PersistentNodePoolClass>
<TemporaryNodePoolClass>org.kowari.store.nodepool.memory.MemoryNodePoolFactory
</TemporaryNodePoolClass>
<PersistentStringPoolClass>org.kowari.store.stringpool.xa.XAStringPoolFactory
</PersistentStringPoolClass>
<TemporaryStringPoolClass>org.kowari.store.stringpool.memory.MemoryStringPoolFactory
</TemporaryStringPoolClass>
<SystemResolverClass>org.kowari.resolver.store.StatementStoreResolverFactory
</SystemResolverClass>


<ContentHandlerClass>org.kowari.content.rdfxml.RDFXMLContentHandler</ContentHandlerClass>
<ContentHandlerClass>org.kowari.content.mp3.MP3ContentHandler</ContentHandlerClass>



<ResolverFactoryClass>org.kowari.resolver.lucene.LuceneResolverFactory</ResolverFactoryClass>

<ResolverFactoryClass>org.kowari.resolver.url.URLResolverFactory</ResolverFactoryClass>
<ResolverFactoryClass>org.kowari.resolver.view.ViewResolverFactory</ResolverFactoryClass>


<StartupScript>foo.itql</StartupScript>

<Smtp></Smtp>

</TucanaConfig>

Kowari Command Line Options

You can override the default configuration file with individual command line options or with an external configuration file (specified via a command line option).

The individual command line options you can specify are as follows:

-h or --help
Display a list of the command line options available.

For example, java -jar kowari-1.1.0.jar --help, displays the following:

-h, --help display this help screen
-n, --normi disable automatic starting of the RMI registry
-x, --shutdown shutdown the local running server
-l, --logconfig use an external logging configuration file
-c, --serverconfig use an external server configuration file
-k, --serverhost the hostname to bind the server to
-o, --httphost the hostname for HTTP requests
-p, --port the port for HTTP requests
-r, --rmiport the RMI registry port
-s, --servername the (RMI) name of the server
-a, --path the path server data will persist to, specifying
'.' or 'temp' will use the current working directory
or the system temporary directory respectively
-m, --smtp the SMTP server for email notification

-n or --normi
Do not start an RMI server when you start Kowari.

By default, Kowari automatically starts an RMI server, if one does not already exist, on port 1099, or the port specified by the --rmiport option. In situations where you expect an RMI server to exist already (that is, you are running another Kowari server on the same host) and you do not want to automatically start another RMI server, use this option.

When running multiple Kowari servers on the same host, it is recommended that you:
1. Start an RMI server separately, via the command line using the rmiregistry program supplied by the Java SDK.
2. Start each Kowari server with the --normi option.

-x or --shutdown
Shutdown the running Kowari server.

For example, java -jar kowari-1.1.0.jar --shutdown.

-l or --logconfig
Specify a log4j configuration file to enable fine grain control on logging.

For example, java -jar tks-kowari-1.1.0.jar --logconfig file:<installation directory>/Resources/log4j-template.xml.

See the Logging section for more information on the --logconfig option.

-c or --serverconfig
Specify an external configuration file to override the default configuration file.

For example, java -jar kowari-1.1.0.jar --serverconfig file:<installation directory>/Resources/Kowari-config.xml.

-k or --serverhost
Specify the fully qualified host name for the embedded RMI server used for incoming iTQL queries. The default is the host name of your machine, automatically determined by your operating system's name resolution order.

See the --httphost option for more information.

-o or --httphost
Specify the host name for the embedded HTTP server used for web services, the KowariViewer and Descriptors. The default is the host name of your machine (the same as the RMI server).

You can find the host name bound to Kowari from the:
- Console output at startup
- Kowari Viewer (the server URI)
- TMC (the server URI)
- ipconfig -all command (for systems running a Windows operating system)
- /sbin/ifconfig command (for systems running a Linux or UNIX operating system)
- /etc/hosts file (for systems running a Linux or UNIX operating system)
  If you are unsure as to what your fully qualified host name is, consult your network administrator.
  
  Note - Machines with multiple host names can have problems with the RMI and HTTP servers binding to the wrong host name. If your machine has multiple host names, or fails to detect the correct host name during host name resolution, you can force binding to a specific host by using the --serverhost and --httphost options, either individually or together.
  
  For example, java -jar kowari-1.1.0.jar --serverhost localhost --httphost localhost.

-p or --port
Specify the port number for the embedded HTTP server. The default is 8080. A conflict occurs if this port is in use (by another application or another Kowari server).

For example, java -jar kowari-1.1.0.jar --port 8081.

-r or --rmiport
Specify the port number for the embedded RMI server. The default is 1099. A conflict occurs if this port is in use (by another application or another Kowari server).

For example, java -jar kowari-1.1.0.jar --rmiport 1299.

-s or --servername
Specify the RMI server name. The default is server1 (that is, rmi://mysite.com/server1). If you are running multiple servers on the same host, you must specify a new server name.

For example, java -jar kowari-1.1.0.jar --servername server2.

-a or --path
Specify the location where Kowari database files are stored. The default is the server1 directory (or the name specified using the --servername option) of your Kowari installation.

For example, java -jar kowari-1.1.0.jar --path /usr/local/tucana. For systems running a Windows operating system, java -jar kowari-1.1.0.jar --path c:\tucana.

Note - The directory you specify is the directory containing the server1 directory (or the name specified using the --servername option). Do not specify the server1 portion of the directory name, as Kowari appends this automatically to the directory you specify.

-m or --smtp
Specify an SMTP server to which e-mail notifications are sent.

For example, java -jar kowari-1.1.0.jar --smtp mail-mysite.com.

If you specify an external configuration file or individual command line options, they are evaluated in the following order:

Default configuration file
External configuration file
Command line options

Java System Properties

In addition to the command line options when starting Kowari, there are also Java^TM system properties that control how Kowari runs. These are the -D and -X options specified when you run the Kowari executable jar file.

Note - The -D and -X options must appear before -jar option.

The Java system properties are as follows:

tucana.xa.useByteOrder
tucana.xa.defaultIOType
tucana.xa.forceIOType
The above properties control file input and output in the Kowari data store. See the Controlling File Input and Output in Kowari section for more information on these properties.

itql.command.log
Specifies the log file that iTQL^TM commands are written to by the client application interacting with the ItqlInterpreterBean.

For example, java -Ditql.command.log=/tmp/itql.log -jar kowari-1.1.0.jar
shutdownhook.port
Specifies the port listening for a shutdown. If not specified, the default is 6789. Specifying this property is useful when running multiple Kowari servers on the same machine and you don't want to risk shutting down the wrong one.

For example, java -Dshutdownhook.port=6790 -jar kowari-1.1.0.jar starts Kowari with a shutdown port of 6790.

While java -Dshutdownhook.port=6790 -jar kowari-1.1.0.jar --shutdown performs a shutdown of this Kowari server.
tucana.rmi.prefetchsize
tucana.rmi.pagetimeout
The above properties control how the server returns query results to a client.

The server returns query result rows in groups, known as pages, in order to reduce network traffic. The tucana.rmi.prefetchsize property sets the size of a page. That is, the number of rows in a page. The default page size is 1000.

If this were the only mechanism in place, each new page fetch would result in a delay as the rows that make up the page are gathered. Reducing the page size reduces the delay, but increases the network traffic. To overcome this, a background thread fetches the next page before it is required.

When the next page is requested:
1. The system checks if the next page has already been requested on the background thread. If it has, and it is available, it is returned immediately. That is, there is no delay.
2. If the next page has already been requested, but it is not yet available, the system waits for a specified number of milliseconds before it decides the page cannot be returned, and the query times out.
  The tucana.rmi.pagetimeout property specifies the time the system waits for the next page before timing out. The default is 60000 milliseconds (60 seconds).
3. If the next page has not yet been requested on the background thread, it is requested.
tucana.sp.cachesize
Internally, the system maintains resources as numbers in order to optimize joins and storage space. For example, 1293969 could represent the URI http://purl.org/dc/elements/1.1/title. However, when the system needs to order or stream resources to the client, the system performs a lookup to convert the numbers to resources. In most situations a collection of statements have repeating subjects or predicates, and to reduce the amount of duplicate I/O requests, cache memory is used.

The tucana.sp.cachesize property sets this cache size. The default is 1000 entries.
tucana.rmi.compression
When a query result returns to the client, the content can be compressed in order to reduce bandwidth consumption. The acceptable values for this system property range from 0 to 9, where 0 is no compression and 9 is maximum compression. The higher the compression, the greater the load on the CPU to perform the compression. The default value is 1, which gives maximum performance at about 70% compression.
Xms and Xmx
The above two properties are Java Virtual Machine (JVM) specific settings that specify the minimum and maximum amount of memory allocate to the JVM, respectively.

Increasing the value of the tucana.sp.cachesize or tucana.rmi.prefetchsize properties, or if you have a large number of concurrent users, you can exceed the amount of memory allocated to the JVM (the default maximum is 64 MB) and an out of memory exception occurs. You then need to increase the amount of memory allocated to the JVM.

For example, the following sets the minimum memory allocated to 64 MB and the maximum to 128 MB.

java -Xms64m -Xmx128m -jar kowari-1.1.0.jar -d

Note - Do not over allocate memory to the JVM (using too high a value for Xmx). Kowari uses memory mapped files that are managed by the operating system. Over allocating memory to the JVM reduces the available memory available for the operating system, thus reducing its ability to map files efficiently.

Running the Test Cases

This section is only applicable if you have been supplied the build environment for Kowari^TM.

To run the Kowari test cases you'll need to perform the following steps:

Change to the Kowari home directory:
$ cd $KOWARI_HOME/
Run the test target.
For systems running a UNIX operating system:

$ ./build.sh test

For systems running a Windows operating system:

$ build test

Test output is sent to the $KOWARI_HOME/test/ directory. You can view the files in this directory to obtain more information on test runs.

Note - Individual test cases will usually load up logging configuration from the copy of the template logging configuration file written to $KOWARI_HOME/log4j-conf.xml. If you want to change the logging settings for tests, edit this file.

Latest News

Kowari 1.1.0 Pre-release 1 Released

Kowari 1.0.5 Released

Kowari 1.0.4.1 Released

Kowari 1.0.4 Released

DAWG Evaluates iTQL

Kowari article in XML.com

Kowari mentioned on XML.com

Kowari 1.0.3 Released

Kowari Lite Introduced

Kowari 1.0.2 Released

Kowari 1.0.1 Released

View all news items