Binions is tuned through per-daemon configuration, not a central dashboard. You change a value in a daemon’s configuration file, restart that daemon to apply it, and confirm the effect with the numbers the daemon publishes on its own /metrics endpoint. There is no global “performance” switch and there are very few real knobs — Binions is built to run well at its defaults on a single host. This page is an honest guide to the knobs that genuinely exist, the ones that don’t, and how to size and measure the platform instead of guessing.
Good to know. Most installations never need to touch any of this. The defaults are sized for a small-to-medium host and verified in normal operation. Reach for tuning only when a specific metric tells you to — see Monitoring & tracing for how to watch those metrics.
Every daemon reads its own configuration file, runs as its own background service, and keeps its own state. So tuning is always local to one daemon: edit, restart, measure. The cycle is the same whichever value you change.
# 1. Edit the daemon's config (per-daemon application.toml)
sudo -e /opt/binions/logger/config/application.toml
# 2. Restart just that daemon to apply the change
sudo systemctl restart binions-logger.service
# 3. Confirm it came back healthy, then read its live metrics
binions-cliconsole status
curl -s http://127.0.0.1:9100/metrics | grep '^binions_'The platform dispatches work concurrently rather than serially. Every generic daemon processes incoming actions using an internal worker pool, so a slow operation — a long network call, a heavy query, an SSH command — does not block other work queued behind it. Head-of-line blocking is absent by design. On top of this, the playbook orchestrator decouples accepting triggers from running them: it can have many playbook runs in flight simultaneously, bounded by configurable caps, so throughput scales with available host CPU rather than an artificial serial ceiling.
| Surface | Tunable? | Notes |
|---|---|---|
| Orchestrator concurrency caps | Yes | A global in-flight cap and a per-playbook cap control how many playbook runs execute simultaneously. The defaults suit most workloads; raise them on a well-resourced host to increase throughput. |
| Log batching (logger only) | Yes | logger-service exposes [stream] / [batching] to control how it flushes log lines to disk. |
| Memory caps | Yes | Soft and hard ceilings per daemon, set on the service unit (see below). |
| Redis sizing | Yes | Each daemon’s private Redis has its own memory ceiling and persistence policy. |
| Scheduler tick | Yes | How often the scheduler wakes to check time-based triggers. |
| Tracing | Yes | Off by default; sample rate is configurable when you turn it on. |
The only daemon that lets you tune how it batches work is the logger, because it is the one daemon whose throughput is dominated by writing to disk. Its application.toml carries a [stream] / [batching] block:
# /opt/binions/logger/config/application.toml
[batching]
batch_size = 256 # log lines collected before a flush
block_ms = 500 # max wait for the next line before flushing anyway
flush_interval_ms = 1000 # hard upper bound between flushes to diskTip. Larger
batch_sizeand a longerflush_interval_msmean fewer, bigger disk writes — gentler on a slow SD card or spinning disk, at the cost of a slightly longer delay before a log line lands on disk. Leave these alone unless the logger is the bottleneck.
The playbook engine is designed for high concurrency. Accepting a trigger and running the resulting playbook are handled independently, so a burst of incoming events does not create a queue of waiting runners — many runs proceed in parallel. Two caps control this behaviour in the playbook service configuration:
Per-resource ordering is preserved even at high concurrency: actions that target the same resource (the same schedule name, the same Traefik route, the same database record) are still serialised in the order they arrive. Only independent actions run in parallel, so concurrency never reorders operations on a single resource.
In practice, a simple single-step run completes in roughly a tenth of a second. Sustained throughput scales into the tens of runs per second on a modest host, and the platform has been soak-tested at high load over extended periods with no failures, no memory growth, and no crashes. For most workloads the defaults are correct; tune the caps only when metrics show the orchestrator is the bottleneck.
How
mode: asyncinteracts with concurrency. A playbook withmode: asyncfires itsrun:steps as fire-and-forget actions and joins results with explicitwait_for:steps. This means one playbook run can itself fan out many concurrent actions without consuming multiple in-flight run slots. If your workloads fan out heavily,mode: asyncis more efficient than relying solely on raising the global cap. See Playbook anatomy for details on execution modes.
Each daemon runs under two memory ceilings set on its service unit. They behave differently, and knowing which is which saves a lot of guesswork:
MemoryHigh is the soft cap. When a daemon crosses it, the kernel throttles the process and reclaims memory aggressively, but the daemon keeps running. It is a back-pressure signal, not a kill switch.MemoryMax is the hard cap. A daemon that hits this ceiling is terminated by the kernel’s out-of-memory killer; the service then restarts under its normal recovery policy.The defaults are sized to each daemon’s real workload. The mailbox daemon, which buffers messages and attachments, gets the most headroom; the lightweight coordinators get the least.
| Daemon | MemoryHigh (soft) | MemoryMax (hard) |
|---|---|---|
| mailbox | 768M | 1G |
| database | 384M | 512M |
| playbook | 384M | 512M |
| aiinjector | 384M | 512M |
| logger | 128–192M | 256M |
| datatransporter | 128–192M | 256M |
| scheduler, webhookcaller, traefiklinker, dataanaliser, showman, modbus | 96M | 128M |
To raise a ceiling for a busy daemon, override it on the service unit rather than editing the packaged file — a drop-in survives upgrades:
# Give the mailbox daemon more headroom on a high-volume host
sudo systemctl edit binions-mailbox.service
# In the editor, add:
# [Service]
# MemoryHigh=1G
# MemoryMax=1280M
sudo systemctl restart binions-mailbox.service
systemctl show binions-mailbox.service -p MemoryHigh -p MemoryMaxWatch the soft cap first. If a daemon repeatedly bumps its
MemoryHighand slows down, that is the early warning. Raise the soft cap (and the hard cap with it) before the daemon ever reachesMemoryMaxand gets killed — restarts cost you in-flight work.
Every daemon has its own private Redis instance — that is where its event streams, transactional outbox, and idempotency keys live. Because each instance is dedicated to one daemon, you size them small and individually rather than running one large shared cache. Each one has a memory ceiling, never evicts data, and persists to disk so nothing is lost across a restart.
# Per-instance Redis settings (one instance per daemon)
maxmemory 256mb # ceiling: typically 64mb to 256mb depending on the daemon
maxmemory-policy noeviction # never drop events to free space — back-pressure instead
appendonly yes # append-only file: durable, replays on restart
save 900 1 # snapshot if ≥1 key changed in 15 min
save 300 10 # … or ≥10 keys in 5 min
save 60 10000 # … or ≥10000 keys in 1 minThe noeviction policy is deliberate and important: Binions would rather apply back-pressure and slow down than silently discard an event to make room. If a Redis instance fills up, that is a sign the matching daemon is falling behind — the fix is to find and clear the backlog (see Measure, don’t guess below), not to enable eviction.
Durability is the default. Append-only persistence plus the snapshot policy means a daemon’s in-flight state survives a crash or restart. Don’t disable
appendonlyto chase write throughput — you would be trading away the platform’s exactly-once guarantees.
There is exactly one time-driven knob in the whole platform, and it lives in the scheduler. Every other daemon is purely event-driven — it does work only when an event arrives on the bus, never by polling on a timer. So the scheduler’s tick is the single setting that controls how often Binions wakes up to check the clock for time-based triggers.
# /opt/binions/scheduler/config/application.toml
tick_interval_secs = 30 # how often the scheduler checks for due time-triggersThe default of 30 seconds means a schedule fires within half a minute of its due time, which is plenty for cron-style reports, syncs, and digests. Shortening the tick makes time-triggers fire more promptly but wakes the daemon more often; lengthening it is gentler on a very small host. Because nothing else polls, this one value is the entire “scheduling overhead” of the platform.
No hidden pollers. If you are coming from systems where every integration runs its own timer, note that Binions has none. The data-transporter offers only an optional offload toggle for heavy transfers; it does not poll. The scheduler tick is the only clock in the system.
Distributed tracing is genuinely useful when you are debugging a slow or misbehaving workflow, but it is not free — every span has to be built, sampled, and shipped. In steady-state operation you should keep tracing off, which is the default. A daemon with no [otel] block in its configuration does structured logging only and carries zero tracing overhead.
When you do need to investigate, turn tracing on for the daemons involved and sample rather than capture everything — a fractional sample rate keeps the overhead small while still giving you representative traces.
# Add [otel] only while investigating; remove it (or sample) afterwards
[otel]
endpoint = "http://127.0.0.1:4317" # OTLP export to the bundled collector
sample_rate = 0.1 # keep ~10% of traces; 1.0 = keep everythingTrace narrow, trace short. Enable
[otel]on the one or two daemons in the workflow you are chasing, reproduce the issue, then take the block back out. Leaving full-rate tracing on across every daemon in steady state is the most common self-inflicted performance cost. Full details in Monitoring & tracing.
The single most useful performance habit is to let the metrics tell you where the problem is before you change anything. Every daemon exposes a Prometheus endpoint on a loopback address, and the values below are the ones worth watching. They all share the binions_ prefix.
| Metric | Type | What it tells you |
|---|---|---|
binions_outbox_pending_size | gauge | How many results a daemon has produced but not yet handed off. Steadily rising = the daemon is falling behind. |
binions_event_processing_duration_seconds | histogram | How long each event takes to handle, by event type. Watch the upper percentiles for slow steps. |
binions_events_processed_total{status="failed"} | counter | Events that errored out. A climbing failure count usually explains a backlog faster than any timing graph. |
binions_healthcheck_last_activity_age_seconds | gauge | Seconds since the daemon last did real work. A large value on a daemon you expect to be busy means it is stalled or starved. |
binions_healthcheck_last_redis_ok_age_seconds | gauge | Seconds since the daemon last reached its Redis. A growing value points at a Redis or connectivity problem, not a workload one. |
Reading them is a quick curl against the daemon’s loopback metrics port:
# Backlog and failures for the playbook daemon
curl -s http://127.0.0.1:9108/metrics | grep -E 'outbox_pending_size|status="failed"'
# Where time is going, by event type
curl -s http://127.0.0.1:9108/metrics | grep 'event_processing_duration_seconds'One honest gap to be aware of: there is no live metric for consumer lag — how far a daemon’s reader trails the head of its stream. To measure stream depth and stuck messages directly, ask the daemon’s own Redis instead:
# How many messages are sitting in a stream (overall depth)
redis-cli -s /run/redis-binions-playbook.sock XLEN events:actions
# Messages delivered to the consumer but not yet acknowledged (true lag)
redis-cli -s /run/redis-binions-playbook.sock XPENDING events:actions playbook-groupOne number rarely tells the whole story. A rising
outbox_pending_sizewith a climbingfailedcounter means a downstream step is erroring, not that the host is too slow. A rising backlog with healthy processing times and a growinglast_activity_ageusually means the daemon is genuinely saturated — that is when more CPU or RAM on the host is the right answer. Read the metrics together before you tune.
That last case — a daemon that is busy, error-free, and still behind — is the one that calls for scaling. Because Binions runs on a single host by design, scaling means a bigger host and right-sized daemons, not more nodes. The next page covers exactly how to think about that.