tail-file

Watch files matching globs patterns for new data, split by separator, and emit each record as {payload: string}. File positions and inodes are tracked in db to handle log rotation. Re-evaluates globs every glob-interval seconds to discover new files.

Config

Required:

• globs: Array of glob patterns for files to tail

Optional:

• separator: Record delimiter (default: \n)
• db: State file path (default: tail.taildb)
• glob-interval: Glob re-evaluation interval in seconds (default: 60)
• interval: File poll interval in milliseconds (default: 1000)
• from-start: Read new files from the beginning instead of current end (default: false)

Example

pipeline:
  tail-file:
    config:
      globs:
        - /var/log/app/*.log
      separator: "\n"
      db: taildb.json
      from-start: true