Pipelines
Multiple targets
Add targets to an existing pipeline, read each target's state, pause one without stopping the others, and remove one.
A pipeline can write the same captured changes to more than one target. The wizard creates the pipeline with one target, the primary, and you add the others from the pipeline's page. Each target has its own connection, its own table names and rules, and its own state, so one slow or paused target doesn't stop the others.
Add a target
You can add a target while the pipeline runs or while it is paused.
- Open the pipeline and select Actions > Add target, or Add target on the Targets card of the Overview tab.
- In the Add target dialog, choose the Target engine.
- Choose where the target is applied. This pipeline's agent (local, no relay) is the default. To apply it from another online agent over a relay, choose that agent. See Relay.
- Enter the connection. The fields are the same as on the wizard's Target step.
- Select Add target. The dialog tests the connection first. A relayed target is tested from its apply agent, and needs both agents online and the relay dial verified.
- If the target lacks tables, a notice lists them. Select Show SQL to read what will run, then the create button, such as Create tables & keys. Or fix the target yourself and select Refresh. A column missing from an existing table isn't added for you.
The page confirms with Target added. The new target gets its own initial load of every replicated table, as described in Initial load, and its row reads initial load until the load finishes. The existing targets keep streaming, and are not emptied or loaded again.
If a table already holds rows in the new target and SQL Flow can't prove they are its own, the start stops instead of emptying it. See Rows already in the target.
When you can't add one
| Situation | What you see |
|---|---|
| The pipeline is archived | Restore the pipeline before editing targets. |
| The pipeline runs on two agents through a relay | Add target isn't offered. |
| Two targets would write one table | Table <table> in <target> would be written by <n> targets of this pipeline (<ids>). … Point one of the targets at another database, or remove one of them. |
| Another active pipeline writes the same target table | The start is refused until you pause or archive the other pipeline. See Lifecycle. |
Read a target's state
The Targets card shows one row per target: its name, a status line, its connection, primary target or additional target, and for a database connection its encryption setting.
| Status line | Meaning |
|---|---|
A lag value, for example 210ms | The target's own apply lag. |
no lag reported yet | The agent hasn't reported a lag for this target. |
initial load | The target is loading or reloading. |
restarting | Automatic restart is bringing the pipeline back. |
paused | You paused this target. |
paused (pipeline) | The whole pipeline is paused. |
agent offline | The pipeline's agent is offline. |
failing, degraded | The pipeline's status. |
dry run: nothing is written | The pipeline is a dry run. |
A target applied from another agent also shows a chip, such as on <agent> or on <agent>: offline, relay unavailable. The pipeline's overall lag is the worst of its targets.
Configure a target
Select Configure on a target to set its table names, row filters, output columns and masks. Each target has its own settings, so the same source table can have different names and rules on each. Names change only while the pipeline is paused, and a renamed table is written under its new name after you reload the pipeline (Actions > Reload from source). See Tables and configure.
Pause and resume one target
Select Pause on a target to stop writing to it. Source capture continues, and the changes wait in the spool, so the spool grows while the target is paused. The other targets keep applying. Select Resume to have the target apply the changes it missed.
Pause a target only for a short maintenance window. A long pause makes the spool grow and holds the source log longer. When the spool reaches its limit, capture waits (SQL Server DirectLog) or the paused target must be reloaded. See Lag, spool and log retention.
Remove a target
You can remove any additional target. The primary target can't be removed.
- Pause the pipeline. Otherwise the page says
Pause the pipeline first, then remove the target. - On the target's row, select Remove, and confirm Remove target?.
SQL Flow stops writing to the target. The tables and rows already there stay. Changing where a target is applied, local or over a relay, also needs the pipeline paused.
