Pipelines
Pipeline detail
Read a pipeline's page: the header actions, status banners, headline metrics, and what each of the six tabs shows.
Select a pipeline on the Pipelines page to open its page. The page shows a header with the lifecycle actions, any banner that needs your attention, three headline metrics, and six tabs: Overview, Tables, Config, Schema, Compare and Logs. The page updates live. If it can't refresh, the Overview charts dim and say Data may be stale. Retrying…
The header
The header shows the pipeline's status, its name and its source and target connections. A Live · updated just now chip warns when the live updates stall.
Other chips appear when something needs attention:
- ⚠ Agent offline: the assigned agent is offline, so the pipeline isn't running and the figures are stale.
- Unassigned: assign an agent →: no agent runs the pipeline. Select it to choose one.
- No target: the pipeline monitors the source only.
- No agent can run <engine> → add one: no registered agent can capture this source.
- archived: the pipeline is archived.
- queued: the initial load waits in its agent's load queue.
- Loading alongside <n> pipelines: other pipelines are loading on the same agent, source database or target database. Hover to see which ones.
The main button is Start when the pipeline is paused and Pause when it runs. See Lifecycle.
The Actions menu holds the rest:
| Item | What it does |
|---|---|
| Reload from source | Empties the target tables and copies them again. Needs the agent online. See Lifecycle. |
| Assign agent… | Chooses the agent that runs the pipeline, or Unassign. Not shown for a pipeline that runs on two agents. |
| Set project… | Chooses the pipeline's project, or No project. Shown when the account has projects. See Projects. |
| Archive | Stops the pipeline and moves it to the Archived tab. Pause the pipeline first. |
| Add target | Adds another target. See Multiple targets. |
| Restore, Delete | For an archived pipeline only. See Lifecycle. |
| Download diagnostics | Collects a bundle for SQL Flow support. See Diagnostics for support. |
| Save raw log bytes on agent (contains row data)… | Shown only to an Account admin, and only when an agent holds the raw log bytes of a stop. The file stays on the agent. See Diagnostics for support. |
Banners
One banner at a time appears under the header, the most urgent first:
| Banner | Meaning |
|---|---|
| Not started, not enough agent memory: | The agent doesn't have the memory the pipeline needs, and the start isn't retried. Change what the message names, then pause and start. |
| Failed, needs operator: | Automatic restarts are used up, or the fault needs a person. The banner shows the first and latest fault and the cause. When only a reload clears the stop, it tells you to select Actions > Reload from source. See Automatic restart. |
| A named target outage, for example the target is unreachable | Changes wait in the spool and lag grows until the target accepts writes again. Nothing is lost while the spool has room. See Lag, spool and log retention. |
| Pipeline failed, restarting automatically: | The agent is restarting the pipeline from its saved position. |
| Last run failed: | The pipeline is paused or its agent is offline, and this is why the last run failed. It is history, not a live fault. |
| Pipeline degraded: or Pipeline failed: | The running pipeline reports this error. |
| Nothing applied to the target: | Changes were captured, but none reached the target. Check the target tables, columns and connection, then pause and start. |
If the Failed, needs operator: banner says Refusal evidence recorded, SQL Flow met a SQL Server log record it doesn't recognize. It stopped before that record so no change is skipped, and it keeps holding the source log. Select Download diagnostics on the banner and send the bundle to SQL Flow support. Pause releases the hold, and Reload from source starts over from a fresh copy. See Diagnostics for support.
Above these, Agent offline: names the offline agent. An offline agent may still be running, so its pipelines can't be paused, archived or moved until it is back. If it is lost for good, revoke it on the Agents page: its pipelines then move to its standby, or pause. See Pause. Log broker access unavailable: appears when a pipeline that reads the log through a broker can't reach it. See High availability.
Headline metrics
Three tiles sit under the banner. They show averages over a recent window.
| Tile | Figures |
|---|---|
| Capture | changes / sec, data / sec and payload / sec at the source side. |
| Apply | changes / sec and data / sec written to the target. Not shown for a source monitor. |
| Lag | The end-to-end lag, with the capture and apply parts beneath it and goal <value>. It reads Lag · to landing zone for a Fabric Mirror target. For a source monitor the tile is Spool backlog. |
When the agent is offline every figure reads –. See Lag, spool and log retention.
While an initial load runs, a progress card sits under the tiles. See Initial load.
Open Details under the card to see the Change spool size and growth, the throughput for the last hour and last 24 h, and Last source event. That line turns red with capture may be stalled when the source has changes that capture hasn't seen for over a minute, and amber with apply may be behind when captured changes haven't been applied for a minute.
Overview tab
The Overview tab has these cards:
- Throughput and Replication lag over time. A source monitor shows Source monitor with spool facts instead of lag.
- Source queries/s: how many SQL commands the pipeline sent to the source each second.
- Agent CPU & RAM: resource use of the agent process. A relayed pipeline shows the capture agent.
- Changes: inserts, updates and deletes Observed at the source, Pending and Applied to the targets, since the current capture run started.
- Targets: each target, with Add target. See Multiple targets.
- Target apply plan: how the target applies changes, such as "Applies changes in source order, grouped by source transaction".
- Status: the status and the health reasons the agent reports.
- Auto-restart: restart count, the next attempt and the last fault, after a fault. See Automatic restart.
- Access: for a pipeline that reads a raw log, how it reads it, such as Log broker, and whether access is healthy.
- Log retention: what the pipeline holds on the source log. See The Log retention card.
- Availability: agent and source-log protection. See High availability.
- Details: the pipeline id, which you can copy for support or the CLI, and the connections, capture mode and assigned agent.
Tables tab
The Tables tab lists the Captured tables with their Rows at load, key and Initial load result, and has the Edit tables button and the per-table Resync and Repair keys… actions. Rows at load is the row count the most recent initial load saw, not a live count. See Changing tables.
Config tab
The Config tab shows the pipeline's configuration and its source and target connections.
- Select Edit on Source or Target to change the host, credentials or options. Enter the password again, then select Save connection. Editing needs the pipeline paused. For a failing or degraded pipeline the tab offers Pause & edit.
- Atomic transaction limits shows the limits per target and lets you change them while paused. See Target.
- Lag goal shows the goal this pipeline's lag is judged against. See Set a lag goal.
- Relay pairing appears for a pipeline that runs on two agents. See Relay.
- A SQL Server, Azure SQL or Fabric SQL database target also shows its Bookkeeping schema, which is fixed when the pipeline is created.
Schema, Compare and Logs tabs
- Schema lists Schema changes, the DDL events detected on the source. Tables held for review appear in a Schema review card on every tab. See Schema changes.
- Compare compares the source and the target and shows the history of runs. See Compare and Repairing drift.
- Logs shows the events for this pipeline, with errors first. See Logs.
