Distributed acceleration

Overview

Learn why applying changes over a long-distance link is slow, and how relay agents near the source and the target remove that penalty.

Distributed acceleration splits a pipeline across two agents: a capture agent near the source and an apply agent near the target. The captured changes cross the long-distance link compressed and back to back, without waiting for each write to finish, so the link's round-trip time no longer limits how fast the target is written.

The two agents are called relay agents, and the link between them is the relay. Relay needs the Distributed acceleration capability pack; see Plans and licensing.

The long-distance penalty

Without a relay, one agent reads the source and writes the target. When the target is in another region or another cloud, every commit to it waits at least one network round trip for the target's answer.

That wait, not bandwidth or the database, then sets the pace. Throughput falls as the distance grows, and a target that could absorb far more sits idle between batches.

How relay removes it

One agentRelay agents
Where the target's round trips happenAcross the long-distance linkOn the target's local network
What crosses the long-distance linkThe target's database protocol, with a round trip for every commitCompressed changes, many in flight at once
What a slow or broken link doesStalls applyGrows the capture agent's spool; apply continues when the link is back
Where the target's credentials are usedOn the capture sideOnly on the apply agent

Initial loads cross the same link, compressed the same way.

The capture agent reads the source database and keeps a local spool. It sends compressed packets over TLS 1.3, many in flight, to the apply agent, which keeps its own local spool, returns durable acknowledgements and writes the target database.

When to use relay

  • The source and the target are in different regions, data centers or clouds, for example on-premises SQL Server to Azure SQL.
  • Initial loads or catch-up after an outage must cross a slow or expensive link. Relay compresses what it ships, and compresses harder while it catches up.
  • The target's credentials and network access should stay inside the target's network.

What you can relay

  • In the pipeline wizard: every source and capture mode, with the pipeline's target. See Relay.
  • On an existing pipeline: add a target that another agent applies. See Setting up relay agents.

Always Encrypted ciphertext doesn't travel over a relay. A SQL Server DirectLog pipeline that copies Always Encrypted ciphertext needs local targets, and is refused with always_encrypted_relay_unsupported.

See also

If this was useful,
our newsletter will be too.

Get monthly insights on data engineering, AI, and building critical infrastructure - direct from the Spectral Core team and CEO Damir Bulic.