# Vision

## The Signal Framing

A useful mental model for LogBus, borrowed from signal processing and signals intelligence:

**Layer 1 — Signal Conditioning** is the digital layer.  Raw signals are noisy, inconsistently formatted, and arrive at irregular rates.  Before you can make sense of a signal you have to clean it.  This is the job of DSP: filtering, normalization, sampling, transforms.

**Layer 2 — Signal Intelligence** is the analog layer.  Once you have a clean signal, you can start asking harder questions: what patterns are present, what do they mean, what is anomalous, how do events across sources correlate?

LogBus currently operates at Layer 1.  The work of wiring up sources, parsing formats, normalizing fields, enriching events, aggregating streams, and routing to sinks is all signal conditioning.

## Layer 1: Signal Conditioning

The plugins that exist today map cleanly onto classical DSP concepts:

| DSP concept | LogBus equivalent |
|:---|:---|
| Low-pass filter (remove high-frequency noise) | `filter`, `dedupe`, rate-limiting sinks |
| Normalization | `js`, `rename`, `keep` |
| Sampling | `sample`, `tap`, `agg` |
| Transforms & Enrichment | `js`, `geoip`, `sql` |
| Multiplexing / demultiplexing | fan-out / fan-in wiring |
| Buffering | bounded channel backpressure |

The goal is a conditioned signal: structured, normalized, enriched, routed.  A downstream consumer should be able to trust it.

## Layer 2: Signal Intelligence

Once the signal is conditioned, the interesting questions become tractable:

- **Pattern detection** — what sequences of events indicate a problem or an opportunity?
- **Anomaly detection** — what deviates from the baseline?
- **Cross-source correlation** — what events across different systems are causally related?
- **Meaning extraction** — what is this stream actually telling us?

This is closer to SIGINT than DSP.  It requires memory, models, and probabilistic reasoning — things that are architecturally different from a streaming pipeline of deterministic transforms.

Whether L2 lives inside LogBus (as a new class of "intelligence" plugins), as a separate system that consumes a LogBus-conditioned stream, or as some combination, is an open question.  What is clear is that L1 has to be solid before L2 is meaningful.  Garbage in, garbage out applies here as much as anywhere.

## The Feedback Loop: L2 Output → L1 Input

The layers are not strictly one-directional.  L2 observations can flow back into L1 to improve the conditioning — closing the loop:

```
[raw stream] → [L1: condition] → [clean stream] → [L2: analyze]
                    ↑                                     |
                    └──── pipeline adjustments ←──────────┘
```

Concretely: if L2 observes that 40% of all events are `GET /health` log lines at `info` severity, it can recommend that L1 aggregate or drop them.  The conditioned stream then improves, which in turn makes future L2 analysis less noisy.  This is a classic control loop: measure, compare to intent, correct.

In the limit, L2 feedback could drive L1 automatically — adaptive pipelines that tune their own filters and sampling rates in response to what the intelligence layer observes.  That is a long way off, but the architecture should not close the door on it.

## The Advisor Plugin: A Bridge

The `advisor` plugin sits at the L1/L2 boundary.  It rides along with a live L1 pipeline and observes the conditioned event stream over sliding windows, then emits structured `AnalysisEvent` objects containing **findings** about what it saw:

| Finding type | What it means |
|---|---|
| `agg-candidate` | A single message pattern dominates the window — probably safe to aggregate or drop |
| `low-severity-agg` | A high-volume pattern is almost entirely low-severity noise |
| `noisy-process` | One process is flooding the stream with low-value events |
| `rare-pattern` | A pattern is statistically unusual — may be high-signal |
| `novel-pattern` | A pattern has not appeared in recent history — worth attention |

This is lightweight signal intelligence applied directly to the conditioned stream.  The findings are not acted on automatically; they are emitted as events, and a human (or a downstream tool) uses them to decide how to adjust the L1 pipeline.

The advisor therefore demonstrates the feedback loop in its simplest form: L2-style analysis of the L1 stream, with output structured to drive L1 changes.  It answers the question "is my pipeline well-tuned?" and hands back enough context to fix it if not.

## What This Means for Development

- **Today:** continue deepening L1.  Better parsers, richer enrichment, smarter routing, stronger backpressure.
- **Near term:** expose the conditioned stream in ways that L2 consumers can trust — stable schemas, reliable delivery guarantees, good observability into what was dropped or rate-limited and why.  Develop the advisor's findings into actionable, copy-paste pipeline amendments.
- **Later:** evaluate whether L2 primitives (stateful pattern matching, windowed aggregation with ML hooks, anomaly scoring) belong in LogBus or should be left to purpose-built systems fed by LogBus.  Consider whether the feedback loop from L2 → L1 should ever be automatic rather than human-mediated.
- **Always:** be a Swiss army knife for Sys Admins.  The loftier L2 goals should be opt-in and never interfere with an operators ability to solve acute problems quickly & simply.