Targets
Event streaming & messaging
Publish captured changes to Apache Kafka or Google Cloud Pub/Sub as JSON events, delivered at least once.
An event target receives each captured change as a JSON message instead of SQL. Use one to feed applications, microservices or downstream pipelines that consume a stream of changes.
Targets in this group
| Target | Highlights |
|---|---|
| Kafka | Keyed JSON events on Apache Kafka topics, with TLS and SASL authentication |
| Google Pub/Sub | Ordered JSON events on Google Cloud Pub/Sub topics, using the agent's Google credentials |
What every event target does
- Initial snapshot, then changes. SQL Flow first publishes the rows of each selected table, then one event for each insert, update and delete.
- At-least-once delivery. The committed position advances only after the service accepts a message. After a failure, SQL Flow publishes the unacknowledged changes again, so consumers should drop repeats. Each target's page says how to recognize one.
- Mapping and processing. Name mapping and in-flight processing shape the events before they are published.
- No Reload. A topic can't be emptied and refilled the way a table can, so Reload is refused. To publish a fresh snapshot, create a new pipeline.
