Pipeline wizard
Target
Choose where captured changes are written, enter and test the target connection, and set the apply options on the Target step.
The Target step is the second step of the New pipeline wizard. You choose where SQL Flow writes the captured changes, enter the connection, test it, and set the options for that target. A pipeline starts with one target. You can add more after you create it. See Multiple targets.
Choose the target
The step opens with Where do the changes go? Select a tile:
| Group | Tiles | Page |
|---|---|---|
| Relational databases | PostgreSQL, MySQL, Oracle, SQL Server, Azure SQL, Fabric SQL database, Sybase | PostgreSQL, MySQL, Oracle, SQL Server, Azure SQL, SQL database in Fabric, Sybase ASE |
| Event streaming & messaging | Kafka, Google Pub/Sub | Kafka, Google Pub/Sub |
| Data platform integration | Fabric Mirror | Fabric Open Mirroring |
If you chose monitor the source only on the Source step, this step shows No target: this pipeline monitors the source only and a Choose a target instead button. See Source monitor.
Choose where the target is applied
Under Where this target is applied, the default is This pipeline's agent (local, no relay): the agent that captures the source also writes the target. To apply the target from another online agent close to the target, choose <name> (over the relay). The target's credentials then stay on that agent, and the wizard pairs the two agents before you leave the step. See Relay.
Enter the connection
Fields depend on the target. Saved connections work as on the Source step, and never hold the password. See Use saved connections. The target pages describe each field.
| Target | Fields |
|---|---|
| PostgreSQL, MySQL, Oracle, SQL Server, Azure SQL, Sybase | The same fields as a source of that engine: Host, Port, Database (Oracle: Service name (PDB)), Username, the password or a secret reference, Authentication for SQL Server and Azure SQL, and Encryption for all but Azure SQL. See Enter the connection. |
| Fabric SQL database | Server, Database, Authentication (Microsoft Entra only), and Username and Password where the method needs them. The connection always encrypts and validates the certificate. |
| Kafka | Bootstrap servers, Topic routing, Topic prefix or Fixed topic, Initial snapshot, Security protocol, the SASL settings and the broker CA certificate. |
| Google Pub/Sub | Google Cloud project, Topic prefix. The agent uses the Google credentials available on its host. |
| Fabric Mirror | Target (OneLake (default) or Local folder), Landing zone URL, Microsoft Entra tenant ID, Client ID and the password, or for Local folder, Local path and Work directory. |
If the target is the same server and database as the source, the step shows, in red: "This is the source database. Replicating into it would empty the source tables; choose a different target database." Next stays disabled until you change it.
Test the connection
Select Test connection. Next runs the test for you when the current details haven't passed. The test runs on the agent that applies the target, and the failure tags are the same as on the Source step.
For a Fabric Mirror target, a refused test appears as a red alert under the heading. When mirroring isn't running, the alert offers Start mirroring. The wizard tests this target again, with your tables, when you leave the Tables step.
Set target options
Some options appear below the connection, depending on the target.
SQL Flow's own objects
For SQL Server, Azure SQL and Fabric SQL database, SQL Flow keeps its bookkeeping tables in a schema of its own, and creates the schema if it is missing. Name it in Bookkeeping schema. The default is sqlflow.
The name must start with a letter or underscore, contain only letters, digits and underscores, and be at most 128 characters. System schemas such as sys are refused. You can't change the schema after you create the pipeline. See SQL Server target.
Atomic transaction limits
For relational targets, the Atomic transaction limits section sets how large one source transaction may be. Apply stops if one source transaction exceeds either limit, keeping it intact for recovery.
| Field | Default | Range |
|---|---|---|
| Maximum records per transaction | 1,000,000 | 1 to 10,000,000 |
| Maximum transaction size (MiB) | 512 | Any positive size |
Raising the limits increases the apply agent's memory use. You can change them later on the pipeline's Config tab while it is paused.
Collapse row versions
For PostgreSQL, Oracle, SQL Server, Azure SQL and Fabric SQL database targets, Collapse row versions (faster) is selected by default. SQL Flow then writes only each row's final state when it can prove that is safe for the target, and otherwise applies every row version in order. Clear the box if a trigger or audit on the target must see every change.
Streamed large-object loads
For SQL Server, Azure SQL, Oracle and PostgreSQL targets, Allow streamed large-object loads despite triggers lets a table with large values load as an INSERT plus one update each, even when target triggers exist. See Large values and LOBs.
Leave the step
Next is enabled when the connection is complete and the last target test didn't fail. With a relay, both agents must be online and the relay dial verified. When something blocks Next, the red box says what, for example:
Choose where the changes are written.The last connection test failed. Fix the connection and test it again.The chosen apply agent is <status>. A relay needs both agents online.Host is required.
