Sources
MySQL
Check which MySQL versions SQL Flow can capture from, and what binlog capture needs on your server.
SQL Flow captures changes from MySQL and MariaDB by reading the row-based binary log (binlog) over the replication protocol, the same way a MySQL replica does. There is one capture mode, so the decisions on this page are about whether your server can run it.
Supported versions
| Version | Supported |
|---|---|
| MySQL 8.0 and later, including 8.4 LTS and 9.x | Yes |
| MySQL 5.7 and earlier | No. SQL Flow needs binlog_row_metadata = FULL, which MySQL 8.0 introduced. |
| MariaDB | Yes. Choose MySQL in the wizard. |
Managed services
Managed MySQL services such as Amazon RDS and Aurora MySQL, Azure Database for MySQL and Google Cloud SQL can be sources when they meet the same requirements as any other server:
- The binlog settings below, set through the service's server parameters, with binary logging on and a binlog retention long enough for your pipelines.
REPLICATION SLAVEandREPLICATION CLIENTgranted to the capture user itself.- With Encryption at Require (default), a server certificate the agent can validate. If the service's certificates come from its own certificate authority, the agent host needs that authority's certificate. See Binlog: Encrypted data.
Capture modes
| Mode | What it reads | What you install on the database host |
|---|---|---|
| Binlog | Row events from the binary log, streamed to the agent over a replication connection | Nothing |
The wizard shows no Capture mode choice for MySQL. When you pick MySQL on the Source step, the form asks for Host, Port (default 3306), Database, Username, a password or a secret reference, and Encryption.
Choosing a capture mode
MySQL has a single capture mode. Answer these questions to check that your server can run it.
- Is the server MySQL 8.0 or later?
- No: upgrade the server.
- Yes: go to question 2.
- Can binary logging run with
binlog_format = ROW,binlog_row_image = FULLandbinlog_row_metadata = FULL? These are server-wide settings. With anything else, the binlog doesn't carry complete rows and column names, and SQL Flow refuses to start rather than capture wrong values.- No: MySQL can't be a SQL Flow source.
- Yes: go to question 3.
- Does the server accept TLS connections with a certificate the agent can validate, issued for the host name you connect to? With Encryption at Require (default), the agent encrypts its MySQL connections and validates the certificate.
- No: install such a certificate on the server, or trust its certificate authority on the agent. Encryption at Prefer skips the check, for a server you trust on a network you trust. See Binlog: Encrypted data.
- Yes: go to question 4.
- Can the capture login have
REPLICATION SLAVEandREPLICATION CLIENT?- Yes: use Binlog.
- No: MySQL can't be a SQL Flow source.
| Binlog | |
|---|---|
| Server settings | log_bin on, binlog_format = ROW, binlog_row_image = FULL, binlog_row_metadata = FULL |
| Login | REPLICATION SLAVE, REPLICATION CLIENT, SELECT on the captured tables |
| Installed on the database host | Nothing |
| Holds on the source | Nothing. The server's binlog retention decides how long a stopped pipeline can resume. |
| Resume position | Binlog file name and offset, not GTID |
| Before-images of updates and deletes | The whole row |
TRUNCATE | Captured |
See Choosing a capture mode for the comparison across every source engine.
Encrypted data
| Binlog | |
|---|---|
| Encrypted connection | Required by default, with the server certificate and host name validated. Encryption at Prefer relaxes this. |
Binary log encryption (binlog_encryption) | The server decrypts events before it sends them over the replication connection. SQL Flow needs no setting for it. |
| InnoDB data-at-rest encryption | Not visible to SQL Flow: the server decrypts rows before it returns them. |
| Column values encrypted by the application | Captured and copied as stored, as ciphertext |
See Binlog: Encrypted data for the TLS requirements, and Encrypted data support for every engine.
