Configuration

Shotover proxy accepts a two seperate YAML based configuration files. A configuration file specified by --config-file and a topology file specified by --topology-file

configuration.yaml

The configuration file is used to change general behavior of Shotover. Currently it supports two values:

• main_log_level
• observability_interface (optional)

main_log_level

This is a single string that you can use to configure logging with Shotover. It supports env_filter style configuration and filtering syntax. Log levels and filters can be dynamically changed while Shotover is still running.

observability_interface

Shotover has an optional observability interface for you to collect Prometheus data from. This value will define the address and port for Shotover's observability interface. It is configured as a string in the format of 127.0.0.1:8080 for IPV4 addresses or [2001:db8::1]:8080 for IPV6 addresses. To disable metrics reporting for Shotover, do not specify this field. More information is on the observability page.

topology.yaml

The topology file is the primary method for defining how Shotover behaves.

Consider this example topology.yaml:

# This example listens on two different localhost ports and routes messages to a single valkey instance on localhost.
# Requests received on port 1000 will have metrics recorded on the types of messages sent, while port 1001 will not have those metrics.
---
# The list of sources
sources:
  # First we define the source that will listen for connections from the client and then communicate to the client once a connection is opened.
  - Valkey:
      name: "valkey"
      listen_addr: "127.0.0.1:1000"
      # Next we define the transform chain that will process messages received by this source
      chain:
        # The QueryCounter transform intercepts messages and records metrics on the types of messages that pass through.
        - QueryCounter:
            name: "Main chain"
        # The final transform is a sink, it receives requests from the previous transform and sends them to an actual DB instance.
        # When it receives a response back it routes the response back through every transform in the chain and finally back to the client.
        - ValkeySinkSingle:
            remote_address: "127.0.0.1:6379"
            connect_timeout_ms: 3000

  # A second source definition, this time we lack the QueryCounter transform.
  - Valkey:
      name: "valkey"
      listen_addr: "127.0.0.1:1001"
      chain:
        - ValkeySinkSingle:
            remote_address: "127.0.0.1:6379"
            connect_timeout_ms: 3000

The topology.yaml defines multiple sources. Each source defines an end to end connection to the database and any transformations that occur along the way. The chain section is an array of transforms and their respective configuration. The order in which a transform chain is defined, is the order in which a query will traverse it. So the first transform in the chain is the source and will get the request from client first, then it will pass it to the second transform in the chain and so on.

As each transform chain is synchronous, with each transform being able to call the next transform in it's chain, the response from the upstream database or generated by a transform down the chain will be passed back up the chain, allowing each transform to handle the response.

The last transform in a chain should be a "terminating" transform. That is, one that passes the query on to the upstream database (e.g. CassandraSinkSingle) or one that returns a Response on it's own ( e.g. DebugReturner).

Under the hood, each transform is able to call it's down-chain transform and wait on it's response. Each Transform has it's own set of configuration values, options and behavior. See Transforms for details.