Cluvio Agents

Connecting through a Cluvio Agent allows Cluvio to securely access your database when it is on a private network or does not have a public IP, without requiring inbound connections from Cluvio.

The Cluvio Agent is a lightweight, open-source application that runs on your computer or server and opens connections to Cluvio and to your database to allow Cluvio to easily and securely connect to your database via the agent.

Cluvio can send queries through the agent to your database and return the results back to the Cluvio UI to display as reports. The advantage of running a Cluvio Agent is that there is no need to allow inbound connections from Cluvio to your servers.

Your Private Network

Make sure to enable and secure the network connectivity between your Cluvio Agent and your database. This can be achieved by running the agent on the same host as the database or by running both the agent and the database in a secured, private network.

You can find the management UI for your agents under Organization Settings > Datasources, where the agents are listed below the datasources:

Each Cluvio Agent is listed with its name, a unique ID assigned by Cluvio, information on its usage in datasources and its connectivity status.

Agent Setup

To connect a new agent, click on "Connect New Agent" on the Datasources page. A dialog like the following appears:

The agent name is pre-filled but can be changed to a name that makes it easy to identify it. The agent ID cannot be altered - Cluvio uses this ID to uniquely identify and authenticate an agent that wants to connect to Cluvio.

Below the agent ID you can find installation instructions for various environments:

• Docker: the easiest way to start and run the agent on Linux and Mac
• Mac (Homebrew): install and run the agent via the homebrew package manager on Mac
• Mac / Linux / Windows: download, install and configure the OS-specific agent binary

Follow the instructions for the environment of your choosing.

The dialog continuously monitors the connectivity of the new agent, which will not be connected until the agent is installed and running. Leave the dialog open until you successfully installed and started the agent and the connectivity status on the dialog indicates that the agent is now connected:

Click "Save" to complete the agent setup process. You can now continue with configuring datasources to use the new agent.

Datasource Configuration

To make use of an agent, it must be attached to at least one datasource. To do so, you can choose "Agent" as the connection mode when creating or editing a datasource, as seen in the following example where an agent is used to connect Cluvio to a local PostgreSQL database:

When one or more agents are selected for the datasource, click "Test Connection". If all selected agents are running and connected to Cluvio, the UI will indicate success:

info

When configuring a datasource with agents, a connection test will only succeed if all configured agents for this datasource are running and connected. This assists you in verifying that all the agents are fully functional. In order for report queries to the datasource to succeed, however, it is sufficient if at least one agent is up and running at a time. In this way you can use multiple agents for fault-tolerance as well as during upgrades to the Cluvio Agent software without affecting the availability of reports and dashboards. See also the documentation on disabling / enabling agents further below.

Click Save to complete the datasource configuration.

High Availability

We designed the agents and the datasource connectivity via agents with high availability setups in mind, where you can setup for redundancy or perform maintenance for upgrades without affecting your Cluvio users.

This is primarily achieved via two features:

1. The ability to connect 2 or more agents and use them for the same datasource. When a SQL query needs to be executed via the datasource, the load is balanced via all available agents and if an agent becomes unavailable (e.g. the machine running it becomes unresponsive), other agents are used transparently.

2. The ability to gracefully disable an agent, letting any existing queries that are currently running to finish (referred to as draining) without using the agent for any new queries. Once all running queries are finished, the agent can be stopped and any required maintenance performed on it (such as upgrading the agent or moving it to another server)

When a new agent is created, it is enabled by default. However, there are occasions like software upgrades where it is useful or necessary to temporarily disable an agent, so that it does not receive any new queries, while at the same time allowing currently executing queries to complete. In a setup with at least two agents, this allows you to take an agent (or a host on which an agent is running) temporarily down for maintenance without affecting the availability of your reports and dashboards. Cluvio thus allows you to disable an agent from its settings drop-down menu:

When you click on "Disable", you will be prompted with a request for confirmation:

Once the process of disabling an agent is started, Cluvio immediately stops sending new queries to that agent, but Cluvio allows all currently executing queries on that agent to complete, which may take from a few seconds to many minutes, depending on the complexity and performance of the queries that your reports are running. During this time, hovering over the agent status will indicate that the agent is "draining":

When there are no more active queries on the disabled agent, it is disconnected from Cluvio and its status changes to "disabled":

While an agent is disabled, the Cluvio Agent software running on your host will continuously try to reconnect to Cluvio, as the disabled state is considered temporary. The Cluvio servers will reject the connection attempts until the agent is enabled again. When an agent is in the disabled state, you can safely stop the Cluvio Agent software on your host, e.g. to perform an upgrade. When you restart the agent, it will still be rejected by the Cluvio servers until you re-enable the agent in the UI:

Once re-enabled, it may take a few seconds before the agent successfully reconnects and appears connected in the list of agents.

Deleting Agents

Agents can be permanently deleted from the agent settings drop-down menu:

Because deleting an agent is an irreversible operation, a prompt is shown that must be confirmed by selecting "OK". If the agent should still be connected when it is deleted, it will be disconnected and the Cluvio Agent software, upon receiving the disconnect, will terminate and not restart (unless restarted by service supervision, e.g. a Docker daemon, systemd or similar). Any future connection attempts by this agent are rejected by the Cluvio servers.

Note that only agents that are not used in any datasources can be deleted, in order to ensure that existing reports and dashboards are not affected. If you want to delete an agent that is still used by a datasource, you must first re-configure that datasource to use other agents or a different connection mode.