The following gist contains all files needed to take raw NGINX access logs and then conver them into a suitable format that is ready to ingest within OpenSearch. These components include the following:
- Mock NGINX logs
- Fluent Bit configuration file
- Fluent Bit parsers.conf file (This is the default and a user would not require to define)
- When converting to the proper format we use a lua script to perform this inline. To reduce complexity we do this in a single line, however a user may wish to abstract this and add as a seperate file
- TraceID and SpanID are set as the same values seen in the schema and are not dynamically generated
- timestamp is not seen in Standard Output and instead is only seen when sending the data to OpenSearch