Decompress and untar your download and then change into the unpacked directory.

Is java installed?

The steps in the following sections presume a 1.7 version of Oracle java is installed on your machine and available on your path; i.e. when you type java, you see output that describes the options the java program takes (DCS requires java 7 or better). If this is not the case, DCS will not start. Install java, edit conf/dcs-env.sh, uncommenting the JAVA_HOME line pointing it to your java install.

Is Trafodion installed and running?

DCS presumes a Trafodion instance is installed and running on your machine and available on your path; i.e. the TRAF_HOME is set and when you type sqcheck, you see output that confirms Trafodion is running. If this is not the case, DCS may start but you’ll see many errors in the DcsServer logs related to user program startup.

At this point, you are ready to start DCS.

Now start DCS:

You should now have a running DCS instance. DCS logs can be found in the logs subdirectory. Peruse them especially if DCS had trouble starting.

Stop your DCS instance by running the stop script.

Where to go next?

The above described setup is good for testing and experiments only. Next move on to configuration where we’ll go into depth on the different requirements and critical configurations needed setting up a distributed DCS deploy.

ssh must be installed and sshd must be running to use DCS’s' scripts to manage remote DCS daemons. You must be able to ssh to all nodes, including your local node, using passwordless login (Google "ssh passwordless login").

Both forward and reverse DNS resolving should work. If your machine has multiple interfaces, DCS will use the interface that the primary hostname resolves to.

DCS expects the loopback IP address to be 127.0.0.1. Ubuntu and some other distributions, for example, will default to 127.0.1.1 and this will cause problems for you. /etc/hosts should look something like this:

The clocks on cluster members should be in basic alignments. Some skew is tolerable but wild skew could generate odd behaviors. Run NTP on your cluster, or an equivalent.

This is the default mode. Single node is what is described in the quickstart section. In single node, it runs all DCS daemons and a local ZooKeeper all on the same node. Zookeeper binds to a well known port.

Multi node is where the daemons are spread across all nodes in the cluster. Before proceeding, ensure you have a working Trafodion instance.

Below sections describe the various configuration files that needs to be set up for starting DCS processes. Configuration is described in a section that follows, Running and Confirming Your Installation.

To set up a multi-node deploy, you will need to configure DCS by editing files in the DCS conf directory.

You may need to edit conf/dcs-env.sh to tell DCS which java to use. In this file you set DCS environment variables such as the heap size and other options for the JVM, the preferred location for log files, etc. Set JAVA_HOME to point at the root of your java install.

Before you start DCS make sure Trafodion is up and running first. Start and stop the Trafodion instance by running sqstart.sh over in the $TRAF_HOME/sql/scripts directory. You can ensure it started properly by testing with sqcheck. If you are managing your own ZooKeeper, start it and confirm its running else, DCS will start up ZooKeeper for you as part of its start process.

Start DCS with the following command:

Run the above from the DCS_HOME directory.

You should now have a running DCS instance. DCS logs can be found in the logs subdirectory. Check them out especially if DCS had trouble starting.

DCS also puts up a UI listing vital attributes and metrics. By default its deployed on the DcsMaster host at port 24410 (DcsServers put up an informational http server at 24430+their instance number). If the DcsMaster were running on a host named master.example.org on the default port, to see the DcsMaster’s homepage you’d point your browser at http://master.example.org:24410.

To stop DCS after exiting the DCS shell enter

Shutdown can take a moment to complete. It can take longer if your cluster is comprised of many machines.

To point DCS at an existing ZooKeeper cluster, one that is not managed by DCS, uncomment and set DCS_MANAGES_ZK in conf/dcs-env.sh to false

Next set ensemble locations and client port, if non-standard, in dcs-site.xml, or add a suitably configured zoo.cfg to DCS’s CLASSPATH. DCS will prefer the configuration found in zoo.cfg over any settings in dcs-site.xml.

When DCS manages ZooKeeper, it will start/stop the ZooKeeper servers as a part of the regular start/stop scripts. If you would like to run ZooKeeper yourself, independent of DCS start/stop, you would do the following

Note that you can use DCS in this manner to start up a ZooKeeper cluster, unrelated to DCS. Just make sure to uncomment and set DCS_MANAGES_ZK to false if you want it to stay up across DCS restarts so that when DCS shuts down, it doesn’t take ZooKeeper down with it.

For more information about running a distinct ZooKeeper cluster, see the ZooKeeper Getting Started Guide. Additionally, see the ZooKeeper Wiki or the ZooKeeper documentation for more information on ZooKeeper sizing.

You add site-specific configuration to the dcs-site.xml file, for DCS, site specific customizations go into the file conf/dcs-site.xml. For the list of configurable properties, see DCS Default Configuration below or view the raw dcs-default.xml source file in the DCS source code at src/main/resources.

Not all configuration options make it out to dcs-default.xml. Configuration that it is thought rare anyone would change can exist only in code; the only way to turn up such configurations is via a reading of the source code itself.

Currently, changes here will require a cluster restart for DCS to notice the change.

The documentation below is generated using the default dcs configuration file, dcs-default.xml, as source.

Set DCS environment variables in this file. Examples include options to pass the JVM on start of an DCS daemon such as heap size and garbage collector configs. You can also set configurations for DCS configuration, log directories, niceness, ssh options, where to locate process pid files, etc. Open the file at conf/dcs-env.sh and peruse its content. Each option is fairly well documented. Add your own environment variables here if you want them read by DCS daemons on startup. Changes done to this file requires restart of DCS.

Edit this file to change rate at which DCS files are rolled over and, to change the level at which DCS logs messages. Changes done to this file will require restart of DCS.

A plain-text file which lists of hostname or host IP address on which the primary and backup master process should be started. The first entry will be the primary DcsMaster and the renamining lines will be the backup DcsMaster nodes. Only one host per line is allowed

A plain-text file which lists hosts on which the DcsServer server process should be started. Only one host per line or the host name followed by the count or number of master executor servers. All servers listed in this file will be started and stopped when DCS start or stop is run.

This example shows a basic configuration for a distributed four-node cluster. The nodes are named example1,example2, and so on, through node`example4` in this example. The DCS Master is configured to run on node example4. DCS Servers run on nodes example1-example4. A 3-node ZooKeeper ensemble runs on example1, example2, and example3 on the default ports. ZooKeeper data is persisted to the directory /export/zookeeper. Below we show what the main configuration files, dcs-site.xml, servers, and dcs-env.sh, found in the DCS conf directory might look like.

The default value is 23400. This is the port the DcsMaster listener binds to waiting for JDBC/ODBC T4 client connections. The value may need to be changed if this port number conflicts with other ports in use on your cluster.

To change this configuration, edit conf/dcs-site.xml, copy the changed file around the cluster and restart.

The default value is 100. This is the total number of ports that MXOSRVRs will scan trying to find an available port to use. You must ensure the value is large enough to support the number of MXOSRVRs configured in conf/servers.

The default value is false. When set to true the floating IP feature in the DcsMaster is enabled via the bin/dcsbind.sh script. This allows backup DcsMaster to take over and set the floating IP address.

There is no default value. You must ensure the value contains the correct interface for your network configuration.

There is no default value. It is important that you set this to the dotted IP address appropriate for your network.

To change this configuration, edit dcs-site.xml, copy the changed file to all nodes in the cluster and restart dcs.

DCS, see figure Figure 1, is a framework that connects ODBC/JDBC Type 4 clients to Trafodion user programs (MXOSRVR servers). In a nutshell, clients connect to a listening DcsMaster on a well known port. DcsMaster looks in ZooKeeper for an "available" DcsServer user program (MXOSRVR) and returns an object reference to that server back to the client. The client then connects directly to the MXOSRVR. After the initial startup DcsMaster restarts any failed DcsServers. And, DcsServers restart any failed MXOSRVRs.

DCS provides the following:

The DcsMaster is started via the scripts found in the /bin directory. During startup it registers itself in Zookeeper. If the DcsMaster is started as a backup then after registering in Zookeeper it waits to become the next DcsMaster leader. If ever the backup becomes the leader it executes the bin/dcsbind.sh script to enable floating IP on a given interface and IP address.

A common question is what happens to an DCS cluster when the DcsMaster goes down. Because the DcsMaster doesn’t affect the running DcsServers or connected clients, the cluster can still function i.e., clients already connected to MXOSRVRs can continue to work. However, the DcsMaster controls critical functions such as listening for clients and restarting DcsServers. So, while the cluster can still run for a time without the DcsMaster, it should be restarted as soon as possible.

Please refer to section High Availability

The DcsMaster runs several background threads:

The DcsServer is started via the scripts found in the /bin directory. During startup it registers itself in Zookeeper.

The DcsServer can continue to function if the DcsMaster goes down. the cluster can still function in a "steady state." However, the DcsMaster controls critical functions such as DcsServer failure and. So while the cluster can still run for a time without the DcsMaster, the DcsMaster should be restarted as soon as possible.

The DcsServer runs a variety of background threads:

RAM, RAM, RAM. Don’t starve Dcs.

Use a 64-bit platform (and 64-bit JVM).

Watch out for swapping. Set swappiness to 0.

Symptoms are: When connection are viewed using DCS webUI, the server will be in "CONNECTING" state and the state does not change to "CONNECTED".

When such issues are seen, validate network communication works by using the linux utility 'netcat(nc)'command.

Another test would be to enable verbose when using ssh by using the public or private IP address

The third test would be to use linux tool 'traceroute'

The DCS Reference Guide is written in Asciidoc and built using AsciiDoctor. The following cheat sheet is included for your reference. More comprehensive documentation is available in the AsciiDoctor user manual.

Some parts of the DCS Reference Guide, most notably DCS default configuration, are generated automatically, so that this area of the documentation stays in sync with the code. This is done by means of an XSLT transform, which you can examine in the source at src/main/xslt/configuration_to_asciidoc_chapter.xsl. This transforms the dcs-common/src/main/resources/dcs-default.xml file into an Asciidoc output which can be included in the Reference Guide. Sometimes, it is necessary to add configuration parameters or modify their descriptions. Make the modifications to the source file, and they will be included in the Reference Guide when it is rebuilt.

It is possible that other types of content can and will be automatically generated from DCS source files in the future.

You can include images in the DCS Reference Guide. It is important to include an image title if possible, and alternate text always. This allows screen readers to navigate to the image and also provides alternative text for the image. The following is an example of an image with a title and alternate text. Notice the double colon.

Here is an example of an inline image with alternate text. Notice the single colon. Inline images cannot have titles. They are generally small images like GUI buttons.

When doing a local build, save the image to the src/main/site/resources/images/ directory. When you link to the image, do not include the directory portion of the path. The image will be copied to the appropriate target location during the build of the output.

If you want to add a new chapter to the DCS Reference Guide, the easiest way is to copy an existing chapter file, rename it, and change the ID (in double brackets) and title. Chapters are located in the src/main/asciidoc/_chapters/ directory.

Delete the existing content and create the new content. Then open the src/main/asciidoc/book.adoc file, which is the main file for the DCS Reference Guide, and copy an existing include element to include your new chapter in the appropriate location. Be sure to add your new file to your Git repository before creating your patch.

When in doubt, check to see how other files have been included.

The following documentation issues come up often. Some of these are preferences, but others can create mysterious build errors or other problems.