Every Binions daemon writes structured logs, and any workflow can be traced end-to-end. When something misbehaves, the fastest way to understand it is to read the right log file with jq and — for the trickier cases — follow a single workflow as one trace. This page shows where the logs live, how to filter them, how to change the log level without a restart, and how to turn on tracing.
Good to know. Logs and traces are kept on the host by design. The log files and the tracing UI stay on the machine and the local network — they are never exposed to the public internet.
Each daemon emits structured Log events. One service — the logger — collects them and writes them in two places at once:
/var/log/binions/<service>/events.jsonl. This is the machine-readable copy you filter and search.journalctl -u binions-<service> still works for a quick look.The .jsonl files are rotated daily and the last 7 are kept, so you always have about a week of history without the files growing without bound.
Because every line is a self-contained JSON object, jq is the natural tool. To pull out just the errors for one service:
jq 'select(.payload.level=="Error")' /var/log/binions/<service>/events.jsonlSwap "Error" for "Warn", "Info", or "Debug" to widen or narrow what you see. From there you can shape the output with ordinary jq — for example, projecting only the fields you care about, or piping the result into your pager.
Follow one run with the correlation id. Every log line carries a
correlation_idthat is shared by every event of a single workflow. Once you have it from one line, search for it across the other services’ files to reconstruct exactly what happened, step by step, even when several daemons were involved.
You do not need to edit config or restart a daemon to get more (or less) detail. Send the daemon a control event to set its level on the fly:
binions-cliconsole emit-control Control.<Service>.SetLogLevelThe available levels are Debug, Info, Warn, and Error. Turn a daemon up to Debug while you reproduce a problem, then put it back to Info when you are done so the log file stays readable.
For problems that span several daemons, distributed tracing shows the full path of a workflow as a single timeline. Tracing is opt-in: add an [otel] block to a daemon’s configuration to enable it.
[otel]
enabled = trueWith tracing on, spans are sent to a bundled Jaeger instance whose UI is reachable on the local network only. A standard W3C traceparent travels in each event’s metadata, so one workflow becomes one end-to-end trace — no matter how many daemons it touches.
Best of all, logs and traces are stitched together: they share the same trace id. You can pivot from a suspicious log line straight to the matching trace in Jaeger, or read the log lines that belong to a span you are inspecting.
Keep it on the host. The log files, the metrics endpoints, and the Jaeger UI are all loopback- or LAN-only on purpose. Reach them over SSH or from inside your network — don’t publish them to the open internet.