Upgrades and restarts

Upgrade the SQL Flow control plane, log brokers and agents in the right order, and see what each component keeps across a restart.

Each SQL Flow component is upgraded on its own, while pipelines keep running. This page covers the version support window, the upgrade order and steps, and what each component keeps when it restarts.

Versions and the support window

Every control plane, agent and log broker release has a version of the form YYYY.M.D.BUILD: its release date, then a build number. The Version row on each agent and broker card shows it, and SqlFlow.Agent --version, SqlFlow.LogBroker --version and sqlflow --version print the version of an executable you have.

An agent or broker is supported for 12 months after the date in its version, whatever the control plane's version. The card's chip shows where it stands:

Registration of an out-of-window agent is refused before it uses the join code, so you can retry with a supported release and the same code.

Upgrade order

1. Control plane. It is always the newest component.
2. Log brokers, one host at a time.
3. Agents. For a relay pipeline, upgrade the apply agent before the capture agent.

You can stop between steps. Replication continues throughout.

Upgrade the control plane

Hybrid. Spectral Core upgrades the hosted control plane for you, continuously, and existing pipelines aren't affected. When an agent or broker release you run nears the end of its support, its card shows Upgrade soon well in advance.

Air-gapped. Upgrade a single instance as follows. For several instances, see High availability: Control plane.

1. Back up the PostgreSQL database.
2. Run the migration step with the new image and your existing settings:

docker run --rm --env-file /etc/sqlflow/control-plane.env \
  -v /etc/sqlflow/secrets:/run/secrets:ro -v sqlflow-keys:/keys \
  <new_control_plane_image> migrate

3. Replace the running container with one from the new image, with the same settings and volumes:

docker rm -f sqlflow-control-plane
docker run -d --name sqlflow-control-plane --restart unless-stopped -p 443:5100 \
  --env-file /etc/sqlflow/control-plane.env \
  -v /etc/sqlflow/secrets:/run/secrets:ro -v sqlflow-keys:/keys \
  <new_control_plane_image>

4. Check https://<control_plane_host>/healthz and sign in.

The web app and the CLI can't reach the control plane while the container restarts. Agents and brokers keep replicating, and reconnect on their own.

After migrate has run, an older control plane can refuse the database with store_schema_too_new. Don't restore an older database backup to undo an upgrade.

Upgrade a log broker

Download the new release from Downloads, or take it from your own copy in an air-gapped deployment. The broker's service runs the executable from the folder you installed it in. Replace that file with the new release:

Stop-Service -Name sqlflow-logbroker
Copy-Item -Path .\SqlFlow.LogBroker.exe -Destination 'C:\Program Files\SqlFlow\LogBroker\SqlFlow.LogBroker.exe' -Force
Start-Service -Name sqlflow-logbroker

Use the folder you installed the broker in. The broker keeps its identity and approval. While it is stopped, pipelines that read through it can't read the log; they recover when it is back. See High availability: Log brokers.

Upgrade an agent

1. Download the new release from Downloads, or take it from your own copy in an air-gapped deployment. Copy it to the agent machine.
2. Open Agents → Add agent, and run the install command the panel prints, from the folder that holds the new file. If the agent has a memory cap, add --memory-limit again; install clears a cap you don't repeat.

install keeps the agent's stored identity, so the agent needs no new approval. It replaces the executable and restarts the service. Pipelines continue from their last committed position; nothing is reloaded.

Roll back

• Agents and brokers. Install the previous release the same way. An agent can't read a spool written by a newer release and refuses it, so avoid rolling an agent back once it has run pipelines on the new release.
• Control plane. See the note under Upgrade the control plane.

What survives a restart

What happens when a component restarts

• An agent restarts, after a reboot, a service restart or an upgrade. Each pipeline continues from its last committed position. If the control plane can't be reached when the agent starts, the agent waits, and its pipelines resume once it connects.
• The control plane restarts or can't be reached. Agents and brokers keep running the work they already have for up to 24 hours, and reconnect on their own. Nothing new starts and no configuration changes until the control plane is back. A revoked agent stops.
• A pipeline faults. The agent restarts it with growing delays. See Automatic restart.

See also

• Agents
• Log brokers
• Air-gapped deployment
• High availability overview
• Lag, spool and log retention