All DTS components communicate to the Core via the Internal Communication Bus (ICB), which is a Redis Server. Messages exchanged on the Redis channels are all in JSON format, following a specific protocol.
In this section we'll illustrate how an application can register as a DTS Client, make requests and parse responses using direct ICB communication.
Messages
All messages exchanged by DTS components have the following standard form:
{
"_id" : "",
"_correlationId" : "",
"replyToChannel" : "",
"replyEncryptionKey" : "",
"payload" : {
"payloadMetadata" : {
"payloadClassKey" : "",
"arrayPayload" : false
},
"payloadJSON" : ""
},
"error" : {
"name" : "",
"message" : "",
"type" : "",
"traceback" : ""
}
}
Where the attributes represent the following:
Attribute |
Value |
Note |
||
_id |
A unique identifier for the message - generally a random UUID |
The component creating the message must generate it and ensure uniqueness |
||
_correlationId |
The _id of another message that this message relates to - generally used for responses to reference requests |
A component responding to a certain message must use the _id of the request as the _correlationId of its response |
||
replyToChannel |
The name of the Redis channel on which replies to the message are expected |
A component responding to a certain message must send the response on the replyToChannel of the request. Conversely, when making a request, a component must set the replyToChannel where they expect the response. |
||
replyEncryptionKey |
The AES encryption key corresponding to replyToChannel |
This parameter is only used when communications are secured. Its purpose is for the requesting party to communicate to the responder how to encrypt messages that will be sent on a newly opened channel. It only needs to be included for newly opened channels or when a change of encryption key is desired. Any component receiving a request that includes the replyEncryptionKey must keep it stored and always use it for messages sent on replyToChannel. |
||
payload |
Wrapper for the actual payload of the message |
|
||
|
payloadMetadata |
Wrapper for the metadata of the message's payload |
|
|
|
payloadClassKey |
The DTS Type Key of the payload |
This is the key by which DTS components know what class to use for deserializing the payload. |
|
|
arrayPayload |
Flag that communicates whether the payload is a single object or an array |
|
|
|
payloadJSON |
The actual payload in JSON format |
The payload json is actually wrapped in a string. This means that all " (double quote) characters need to be escaped. |
|
error |
Wrapper for an error object |
Should only exist on responses when an error happened while processing the request. |
||
|
name |
The name of the error |
|
|
|
message |
The complete error message / description |
|
|
|
type |
The type of error |
|
|
|
traceback |
The stack trace of the error |
|
|
If a certain attribute is unknown or not applicable for a given message, it must be omitted. Including an attribute that is an empty string can have unforeseen consequences.
Client Commands
Client Commands are Request-Response entities that facilitate all the operations that a Client can perform within DTS. Requests and Responses are marshalled over the ICB wrapped in messages in the format above where they occupy the payload -> payloadJSON field.
The commands relevant to the Client are:
Standard Sequence
The standard sequence for a Client to connect to the DTS Core and make requests is the following:
•Register
▪[Secure Only] Build an Authentication Package using the component certificate, component private key and controller public key (see Registration and Authentication).
▪Send a Registration Command Request (including the Authentication Package if needed) with a temporary replyToChannel.
▪Read the Registration Command Response (RCResp) from the replyToChannel.
▪[Secure Only] Decrypt the signature and keys provided in the RCResp using the component private key.
▪[Secure Only] Verify that the signature in the RCResp matches the signature sent in the Authentication Package.
▪Register the Redis channel provided by the RCResp (hereafter referred to as componentChannel) and start a reader process on it.
▪[Secure Only] assign the controllerKey from RCResp to be used to encrypt all messages sent on DTS:MIDDLEWARE:CHANNEL.
▪[Secure Only] assign the componentKey from RCResp to be used to decrypt all messages received on the channel provided by RCResp.
▪[Optional] Setup a handler for unprompted Record Stream Command Responses received on the componentChannel (stream timeouts).
•Open a project
▪Send a Project Command Request to Open the desired project (use componentChannel as replyToChannel for convenience).
▪Read the Project Command Response from the replyToChannel.
▪[Optional] Store or verify the list of connector names
•Receive connector entries
▪Handle each Connector Command Request received on the componentChannel by constructing any structures you need based on the metadata the requests provide.
▪Send Connector Command Responses to the request's replyToChannel to acknowledge each connector entry. Use the error attribute of the response message if something went wrong on the client end.
▪[Optional] Wait for the required connectors to become available and/or implement some timeouts.
•Use
▪Send Record Stream Command Requests and/or Execute Remote Command Requests as desired.
▪Records Stream Command Responses for Open Stream Commands will include the channel where Advance commands for the opened stream should be sent ([Secure Only] they will also include the corresponding AES key). Make sure to use these when sending Advance Stream Requests.
•Shutdown
▪Send a Project Command Request to Close any open projects.