User Guide
This guide walks through everything Stream DaQ can do today, from defining a single check to running many monitoring tasks behind a REST API.
How Stream DaQ is organised
A task reads a stream, applies checks to it, and writes the results to an output. A session runs one or more tasks, each in its own process. You can build tasks in Python, or send them as JSON to the REST API.
flowchart LR
S[Input source] --> I
S --> W
subgraph T[Task]
I[Instant checks]
W[Window checks]
end
I -->|per-row results| O1[Output]
W -->|per-window results| O2["Output (_window)"]-
Tasks and Sessions
Define what to monitor, and run it.
-
Instant Checks
Check every incoming row: ranges, sets, patterns, comparisons.
-
Window Checks
Check groups of rows over time: windows, grouping, late data.
-
Measures
The full list of built-in measures for window checks.
-
Assessments
How to say what a "good" value is with
must_be. -
Inputs and Outputs
Read from Kafka, MQTT, files and more; write results anywhere.
-
Compact Data
Monitor compact IoT messages in the EVB format.
-
Custom Code
Your own measures, assessments, sources and sinks.
-
REST API and CLI
Run Stream DaQ as a service and manage tasks over HTTP.
-
Hot-swapping and Persistence
Change running tasks and survive restarts.
-
Logging
Control Stream DaQ's own logs.
-
Advanced Examples
Complete examples combining several features.
-
Good to Know
Practical notes, such as running tasks in Jupyter.