Overview
About
Learn what SQL Flow does, who it is for, and where to start.
SQL Flow replicates changes between operational databases, and from them to event streams, by reading each source's transaction log. Agents and log brokers run on your network, next to your databases, and a control plane that you use through a browser or a CLI tells them what to do. Row data moves only between your databases and your agents, and never reaches the control plane.
SQL Flow is for DBAs and platform engineers who keep a second copy of a database current: for a migration, a reporting or analytics replica, or a feed of changes to Kafka, Google Pub/Sub or Microsoft Fabric.
What SQL Flow does
| Capability | What it gives you |
|---|---|
| DirectLog capture | Reads the SQL Server transaction log file, Oracle redo or the SAP ASE log device through a log broker on the database host, with lag in milliseconds and no query work on the server to find changes. See Why read the transaction log directly? |
| Sources and targets | Captures from SQL Server, Azure SQL, Oracle, PostgreSQL, MySQL, MariaDB and Sybase ASE. Writes to the same relational engines, SQL database in Microsoft Fabric, Kafka, Google Pub/Sub and Fabric Open Mirroring. |
| In-flight processing | Filters rows, chooses and computes columns, and masks values before they reach a target. |
| Mapping | Names tables and columns on each target, and maps every column to a target type. |
| Distributed acceleration | Splits a pipeline between a capture agent near the source and an apply agent near the target, so a long-distance link doesn't limit apply speed. Available for every source and capture mode. |
| High availability | Keeps replicating through the failure of the control plane, an agent or a log broker. A pipeline resumes from the position its target last committed. |
| Validation and auto-repair | Compares a target with its source while the pipeline streams, and repairs the rows that differ. |
| Hybrid or air-gapped deployment | Runs the control plane hosted on the web or on your own servers, with the same agents and brokers in both. |
| Diagnostics for support | Collects a bundle of logs, status and configuration from a pipeline, with no row data, to send to SQL Flow support. |
Start here
- Read How SQL Flow works for the components and how data moves.
- Choose a deployment model, then set up a hybrid or air-gapped deployment.
- Check that your engines are in Supported sources and targets, and choose a capture mode.
- Install an agent, and a log broker if your capture mode needs one.
- Create your first pipeline with Getting started.
