<< Click to Display Table of Contents >> Navigation:  Deployment > Basic Deployment

 

The easiest way to deploy DTS for testing or production purposes is in a Docker-only environment using Docker Compose for easy control and configuration and the DTS Agent for orchestration.

This section outlines the requirements and steps involved in setting up such a DTS deployment.

For information regarding integrating DTS into new or existing orchestrated clusters (e.g. Kubernetes), please contact us.

 

Hardware

 

A DTS standard (minimal) host machine spec is defined as:

•2 CPUs

•16 GB RAM

•100 GB Hard Drive

•1 Network Card

The recommended host machine spec can vary greatly and is defined by the type and numbers of containers expected to be running.

 

Host Operating System

 

Any Windows, Linux or Macintosh operating system that can run a supported Java version and meets the Docker Minimum Requirements will work. Other operating systems may work, but they are not tested by DTS.The most widely used operating system for Data Transit System is Linux and therefore customers should consider it the best tested platform.

 

Java

 

The Data Transit System agent will run outside the Docker network (without a container) and requires a Java 8 Runtime Environment (JRE).

The distributions for OSX and Windows include suitable runtime environments for the specific operating system.

The distributions for Unix do not include the runtime environment.

If you prefer to use an external runtime or use a Unix operating system, you can choose to install the full JDK or the JRE only.

You can confirm the installed Java version with the java -version command, for example:

$ java -version

openjdk version "1.8.0_191"

OpenJDK Runtime Environment (build 1.8.0_191-b12)

OpenJDK 64-Bit Server VM (build 25.191-b12, mixed mode)

 

When multiple JDK or JRE versions are installed, you need to ensure the correct version is configured by running the above command as the operating system user that is used to run the DTS agent.

In the event you have a non-standard location, you need to update the configuration to specify a specific JDK or JRE installation path.

 

Docker and Docker Compose

 

Docker is the containment engine of choice for DTS and is the only one the product is tested with. An appropriate Docker Engine should be installed on the machine.

For information on how to install Docker Engine on a specific host OS, please see the Docker Engine installation instructions

 

DTS is supported on Docker versions starting with version 18.01 but the latest version of Docker is advised for all customers

 

The most efficient way to configure an entire DTS deployment for a quick start of testing procedures is using Docker Compose. This should also be installed on the machine.

For information on how to install Docker Compose on a specific host OS, please see the Docker Compose installation instructions

 

Docker Images Repository

 

For easy access to the Docker images specific to the deployment, we recommend creating a bespoke repository.

This repository can run on the same host machine within a Docker container or can be installed on a different machine or even in the cloud.

For local installations, Nexus Repository OSS is the most common option

 

The initial DTS images and subsequent updates can be pulled from our central repository and pushed into the local one

 

Ports

 

A standard test deployment requires the following ports to be opened on the host machine:

•The GUI Webapp port (configurable - default is 7875).

•The ICB (Redis) port (configurable - default is 7877).

•The SSH port (22).

•[Optional] For remote debugging, the Artifact Datastore (MongoDB) port (configurable - default is 7879).

 

Deployment and Configuration

 

The functional deployment of DTS for testing purposes consists of a directory containing the following:

 

The configuration of the deployment is done in two places:

•docker-compose.yml is used to set up host paths and ports, as well as certain environment variables used to configure the behavior of core components.

▪Each container definition under services has an image parameter which should point to the name of the image to be used by the container (as known by Docker).

▪The ports parameter under the dts-redis, dts-mongo and dts-webapp container definitions allow customizing the mappings of the respective access ports on the host (left side value).

▪The dts-mongo container can be configured to map the mongodata volume to a custom directory on the host if desired (e.g. for backup purposes). If not mapped, the project data will be stored in Docker's own storage pattern.

▪The dts-controller container maps the licence directory under the volumes parameter and can be altered for a custom placement of the licence directory on the host.

▪The dts-gui-controller container maps the blades and wss directories from the host and can be altered for custom placements of the respective directories.

▪The environment parameter for each container should generally remain unmodified, but certain environment variables can be added:

oThe TZ environment variable can be defined to enforce a specific timezone on each container (e.g.: - TZ=EEST).

oDTS_DEBUG_LOGGING can be added to any container to activate its verbose output mode for debugging purposes (e.g.: - DTS_DEBUG_LOGGING=true).

•agent/start-agent.sh is used to set up the Agent's operation mode, connection points and local paths.

▪DTS_AGENT_MODE can be DOCKER or K8S and determines whether Kubernetes or Docker will be the platform used. In the case of the standard test deployment, we use Docker: export DTS_AGENT_MODE=DOCKER

▪DTS_CLUSTER_HOST is the URL that the agent will use to send commands to the container/cluster platform. In the case of the standard test deployment, we use Docker and run the agent on the same machine, so we use Docker's default socket: export DTS_CLUSTER_HOST=unix:///var/run/docker.sock

▪DTS_REDIS_HOST_NAME is the name or address of the machine that provides access to the Redis communication platform. In the case of the standard test deployment, it should be localhost or the IP address of the host machine.

▪DTS_REDIS_PORT is the port that provides access to the Redis communication platform. In the case of the standard test deployment, this should be the host port mapped in docker-compose.yml for the dts-mongo container.

▪DTS_HOST_BLADES_DIR is the absolute path to the blades directory of the deployment.

▪Optionally, a default producer image can be defined using DTS_PRODUCER_IMAGE (which will be used if none is provided in a given connector's definition).

▪Also optionally, the DTS_DEBUG_LOGGING variable can be set to true to enable verbose output from the agent for debug purposes. This option is also passed on to all producer containers that the agent starts.

 

Interaction

 

After deployment, the entire system can be started with a single command:

<dts_dir> $ ./dts-start.sh

 

The system can also be shut down with a single command:

<dts_dir> $ ./dts-stop.sh

 

The DTS core components can be started without agent support using:

<dts_dir> $ [sudo] docker-compose up -d

 

The DTS core components can also be shutdown independently using:

<dts_dir> $ [sudo] docker-compose down

 

The agent can be started individually using:

<dts_dir>/agent $ ./start-agent.sh

 

The agent can also be stopped individually using:

<dts_dir>/agent $ ./stop-agent.sh

 

Logs for various containers can be accessed using this command:

<dts_dir> $ [sudo] docker logs --tail=<n_lines> <container_name>

 

Logs for the current (or last run) agent instance can be found in the <dts_dir>/agent/agent.log file.

 

User Permissions

 

In order for the user which will interact with DTS on the host machine to be able to invoke all the necessary commands and pass enough authority to the agent to perform its tasks, the following are required:

•The user should have full rwx permissions on all files in the DTS deployment listed above in the Deployment and Configuration section.

•For the start and stop scripts, as well as the agent to be able to control Docker, either:

▪The user should be part of the docker group on the host machine.

 or

▪The user should have sudo permission for the docker, docker-compose and java commands (with SETENV and, optionally but preferably NOPASSWD).

•For the agent stop script to be able to kill the agent process, either:

▪The user should have sudo permission for the kill command.

 or

▪The user should have sudo permission to run the <dts_dir>/agent/stop-agent.sh script, on which write access for the user can be removed to ensure no alterations are made.