# Fanning Out & In

The event stream fans out when a stage belongs to multiple downstream stages' input channels. The event stream fans in when a stage contains multiple stages in its input channels. This is useful to sprinkle in conditional logic in a more composable way. Otherwise, users would be required to either:

• Run multiple instances. This approach can be prohibitively expensive, particularly at the ingest portion of the pipeline (eg reading files, kafka topics, opensearch indices).

• Modify existing plugins or create new ones, which can be unnecessarily complex, more prone to error, and less maintainable.

# Use Cases

# Debugging

Can easily tap into the pipeline with debug logic with little risk to impacting the production flow. This debug flow can be as simple or as complex as needed:

• Only want to log a sample of the events? Add a sample stage.
• Only want to debug specific kinds of events? Add a js stage that filters accordingly.
• Want the debugged events to go to a different destination? Add another output stage to ship those elsewhere.

# Aggregations / Roll-ups

One way to deal with noisy processes or processes that log in parser-unfriendly ways (eg multi-line tracebacks) is to aggegate those events into a single event. Allowing the event stream to be split into different roll-up logic and then merged back in for the common enriching & output stages can be useful.

For example, if a process starts spewing the same error repeatedly quickly, it is desirable to roll those up into a single event enriched with the number of times that error occurred. Another example is boot messages. All of those messages are better observed as a single multi-line "boot" event instead of individual per-line events.