Monitoring
Logs
Read the live event log across your pipelines, filter it by level or text, and find out what each event means.
The Logs page, headed Event log, lists the operational events that agents report for your pipelines: a pipeline starting to load, starting to stream, or failing, and warnings about the source log. Use it to find out what happened to a pipeline and when.
Open it from Logs in the top navigation. The same log, filtered to one pipeline, is on the pipeline's Logs tab.
Read the log
The newest events are first. Each row has four parts:
| Column | Meaning |
|---|---|
| Time | The date and time as MM-DD HH:MM:SS, in your browser's time zone. |
| Level | DEBUG, INFO, WARN or ERROR. |
| Event | The event name, such as pipeline.failed. |
| Message | The pipeline's ID, which opens the pipeline, followed by the message. |
The page updates live and shows the most recent 200 events of your account. Select Pause to freeze the list while you read it, and Resume to continue.
Filter the log
- Select ALL, DEBUG, INFO, WARN or ERROR to show exactly that level (not that level and above).
- Type in Search logs… to keep the rows whose level, event name, pipeline ID or message contains the text.
Both filters work on the 200 events the page has loaded, so an older event isn't reachable from here.
On a pipeline's Logs tab, the list starts on ERROR, so a failing pipeline shows what is wrong at once. Select ALL to see every event.
Events you will see
An agent reports an event when a pipeline's state changes, so a quiet log means nothing changed.
| Event | Level | Reported when |
|---|---|---|
pipeline.loading | INFO | The pipeline starts its initial load or a reload. |
pipeline.streaming | INFO | The pipeline finished loading and is streaming changes. |
pipeline.degraded | INFO | The pipeline is running with elevated lag or errors. |
pipeline.dryrun | INFO | The pipeline is a dry run that moves no data. |
pipeline.failed | ERROR | The pipeline failed. The message is the failure reason. Each failure also raises a Pipeline failed alert. |
source_… | ERROR or WARN | A problem with the source log that a SQL Server DirectLog pipeline holds. Each also raises the alert of the same name, which says what to do. |
What to look for
- Select ERROR. Every failure is here, with the reason the agent gave.
- Read the newest
pipeline.failedmessage for the pipeline. Then open the pipeline for the full health reasons and any automatic restart state. - A pipeline that alternates between
pipeline.loading,pipeline.streamingandpipeline.failedis restarting. Automatic restart retries with growing delays, then stops and raises a critical alert. - For detail the log doesn't carry, read the agent's own log. See The data directory.
How long events are kept
Log events are deleted after 14 days.
