Log Transformations

Overview

Textual logs are commonly used among organizations. However, there are some downsides to this approach, such as a longer search time due to a lack of indexing of the data). It is also harder to drill down and find the relevant logs when using textual logs. Transforming textual logs into JSON structured logs helps reduce those downsides.

To transform textual logs into JSON formatted logs, you need to apply code changes. In a fast paced environment, making these changes will usually not get prioritized.

To address this issue, Lumigo provides you with the option for transforming textual logs into JSON structured logs via transformation rules. Currently, this function is not available within the Lumigo UI. To use it, get in touch with Lumigo Support. We will transform your logs for you.

Benefits of structured logs in Lumigo

Lumigo is best suitable for structured logs due to several reasons:

• Metrics: using structured logs, you can store meaningful business details into fields (e.g. lambda function duration) and track them as metrics. These metrics can be then used within your dashboards, and shared with various stakeholders.
• Alerting: you can create advanced alerting based on structured log fields.
• Streamlined search results" Searching in structured logs provides you with a much faster search experience due to the indexing mechanism of these fields.

Create a transformation rule

To create a new transformation rule, navigate to the log transformations page in the Lumigo platform and set the following details:

1. Field name: This name will be the name used in the Lumigo platform, you will be able to filter logs based on this field and its available values.
2. Regex: Using Regex, you can assign the value of your new field. Upon ingesting a new log, Lumigo will check your logs against this Regex.
3. Log field type: The specified type will be used to define the format the new attribute will be stored in the Lumigo Database. For example, select String for textual fields.