# read-opensearch

Execute search at endpoint against index pattern and emit each hit as an event. Uses the scroll API to paginate through results. Supports dynamic index patterns and custom search bodies. Will shut down the pipeline when either max results or all results have been processed.

# Config

Required:

index: Function to return index pattern to search
search: Search body (see Query DSL)

Optional:

endpoint: Function to return server endpoint (defaults to http://localhost:9200)
scroll: Scroll context timeout (default: 1m)
max: Maximum hits to process
ssl.ca: Path to CA certificate file

# Example

pipeline:
  query-opensearch:
    config:
      index: !!js/function >-
        function() {
          const ts = this.moment.utc().subtract(1, 'day')
          return 'logs-' + ts.format('YYYY.MM.DD')
        }
      scroll: 1m
      search:
        size: 333
        query:
          range:
            timestamp:
              gte: now-1d
      endpoint: !!js/function >-
        () => 'http://localhost:9200'
      ssl:
        ca: ~/.certs/ca.pem