Lumigo OpenTelemetry Distribution for Java
Send Data with the Lumigo Distribution for Java applications
Lumigo OpenTelemetry Distribution for Java
Overview
The Lumigo OpenTelemetry Distribution for Java is a package that integrates multiple upstream OpenTelemetry components with additional automated quality assurance and custom optimizations. It is designed for seamless, no-code instrumentation, allowing you to enable OpenTelemetry tracing, and the sending of logs to Lumigo, in your application without modifying a single line of code.
Setup
The setup of Lumigo OpenTelemetry Distribution for Java is made of the three following steps:
- Download the Lumigo OpenTelemetry Distribution for Java.
- Configure the Lumigo tracer token.
- Activate the Lumigo OpenTelemetry Distribution for Java.
1. Download
Download the latest version from the Releases page.
To load the Lumigo OpenTelemetry Distribution for Java on startup, configure your Java Virtual Machine (JVM) using one of the available options.
2. Environment-based configuration
To enable OpenTelemetry tracing with Lumigo, whether by using manual or no-code instrumentation, you must set the required environment variables.
LUMIGO_TRACER_TOKEN
: The authentication token provided by Lumigo. Found under Settings → Tracing → Manual Tracing in the Lumigo platform. see the Lumigo Tokens documentation for more details on how to retrieve your Lumigo token.
LUMIGO_TRACER_TOKEN=<token>
Replace <token>
in the command above with the token generated for you by the Lumigo platform.
OTEL_SERVICE_NAME
: The name you assign to your service for OpenTelemetry monitoring. We recommend defining theOTEL_SERVICE_NAME
environment variable to specify the service name for your application:
OTEL_SERVICE_NAME=<service name>
Replace <service name>
with the desired name of the service.
Note: Note: If you are setting environment variables for configuration, consider also defining the necessary variable for no-code tracer activation.
3. Tracer activation
The Lumigo OpenTelemetry Distribution for Java must be loaded by the Java Virtual Machine before your application is loaded.The two supported ways to achieve this are:
- Recommended: Setting the
JAVA_TOOL_OPTIONS
environment variable - Setting the
-javaagent
command-line parameter
We recommend the JAVA_TOOL_OPTIONS
method because setting environment variables is almost always easier than modifying the entrypoint of a container image, which is usually where you would set the -javaagent
property.
JAVA_TOOL_OPTIONS
Set the JAVA_TOOL_OPTIONS
environment variable on your Java Virtual Machine as follows:
export JAVA_TOOL_OPTIONS="-javaagent:<path-to-lumigo-otel-javaagent>"
This is the preferred method for containerized applications or when running your Java application as a SystemD unit. We recommend this method as in such environments, it is easier to set environment variables than to change the startup arguments of the Java Virtual Machine.
Command-line parameters
Pass the -javaagent
property to the startup command as follows:
java -javaagent:<path-to-lumigo-otel-javaagent> -jar app.jar
Advanced Configuration
OpenTelemetry configurations
The Lumigo OpenTelemetry Distribution for Java builds upon multiple upstream OpenTelemetry packages while adding custom enhancements with additional logic. Since it extends standard OpenTelemetry functionality, it fully supports the environment variables used in vanilla OpenTelemetry configurations.
The following specific configurations are supported:
The system properties are lower case versions of the environment variables, and with dots replaced by underscores.For example, to set the environment variable LUMIGO_TRACER_TOKEN
use the system property lumigo.tracer.token
.
Lumigo-specific configurations
The Lumigo OpenTelemetry Distribution for Java additionally supports the following configuration options as environment variables:
The @lumigo/opentelemetry
package additionally supports the following configuration options as environment variables:
Environment Variable | Description |
---|---|
| [Required] Token required to send data to Lumigo. You can find the value in Lumigo under |
| If |
| If |
| Masks values of keys that match the supplied list of regular expressions. Both traces and logs are filtered.
Default list is:
|
| Secret masking for environment variables, overrides |
| If |
| Avoids creating traces for empty SQS messages. See Filtering out empty SQS messages section |
| Filters client and server endpoints using a list of regular expressions. In the format of |
| Applies regex filtering exclusively to server spans. Filters according to span attributes: |
| Applies regex filtering exclusively to client spans. Filters according to span attributes: |
| If |
| A path to a local file to which spans are written for troubleshooting purposes. Should not be used in production unless directed by Lumigo support. Example value: |
| A path to a local file to dump logs, used for troubleshooting. Effective only when |
LUMIGO_REDUCED_MONGO_INSTRUMENTATION | If |
LUMIGO_REDUCED_REDIS_INSTRUMENTATION | If |
LUMIGO_TAG | Adds the tag value as an attribute to all spans; this is useful to identify the source of the telemetry in Lumigo. See here for details. |
For more configuration options, see the Upstream Agent Configuration.
Execution Tags in different spans of an invocation
Execution Tags allow you to dynamically add dimensions to your invocations so that they can be identified, searched for, and filtered in Lumigo. For example, in multi-tenanted systems, execution tags are often used to mark the identifiers of the end-users that trigger them for analysis (Such as Explore view) and for alerting purposes.
By leveraging execution tags, you can gain deeper insights into your application's runtime behavior.
Adding Execution Tags
In the Lumigo OpenTelemetry Distribution for Java, execution tags are represented as span attributes with the lumigo.execution_tags.
prefix. You can dynamically add these tags to your spans to enrich them with additional metadata. For example, you could add an execution tag as follows:
import io.opentelemetry.api.trace.Span;
Span.current().setAttribute("lumigo.execution_tags.foo","bar");
When using OpenTelemetry's trace.getActiveSpan()
API, you can dynamically retrieve the current span at any point in your program execution without needing to explicitly track it.
In OpenTelemetry, span attributes can be strings
, numbers
(double precision floating point or signed 64 bit integer), booleans
(Also known as "primitive types"), and arrays of one primitive type (Such as an array of string, and array of numbers or an array of booleans). In Lumigo, booleans and numbers are transformed to strings.
When using the Span.setAttribute
API multiple times on the same span for the same key, new values may overwrite the previous values instead of adding to them:
import io.opentelemetry.api.trace.Span;
Span.current().setAttribute("lumigo.execution_tags.foo","bar");
Span.current().setAttribute("lumigo.execution_tags.foo","baz");
In the snippet above, the foo
execution tag will only have the baz
value in Lumigo. The bar
value will have been overriden.
If you want to set multiple values for an execution tag:
import io.opentelemetry.api.trace.Span;
Span.current().setAttribute(AttributeKey.stringArrayKey("lumigo.execution_tags.foo"), Arrays.asList("bar", "baz"));
In the examples above, the resulting invocation in Lumigo will contain both bar
and baz
values for the foo
execution tag.
Another way to associate multiple values with an execution tag is by setting execution tags in different spans within the same invocation. See execution Tags in different spans of an invocation for more details.
Execution Tags in different spans of an invocation
In Lumigo, multiple spans can be grouped together into a single invocation, which represents the full context of your operation. This is the same entry you can see in the Explore view.
When spans are grouped into an invocation, all execution tags from each span are included and their values are merged:
import io.opentelemetry.api.trace.Span;
Span.current().setAttribute("lumigo.execution_tags.foo", "bar");
Tracer tracer = GlobalOpenTelemetry.getTracer("my-app");
Span nestedSpan = tracer.spanBuilder("child_span").startSpan();
try (Scope scope = nestedSpan.makeCurrent()) {
// Do something interesting
nestedSpan.setAttribute("lumigo.execution_tags.foo", "baz");
} finally {
nestedSpan.end();
}
In the examples above, the resulting invocation in Lumigo will contain both bar
and baz
values for the foo
execution tag.
The spans that are merged into a single invocation depend on their parent-child relationships. This means that child spans, such as the nestedSpan
in the example above, will be grouped under the same invocation as their parent span. For a more in-depth understanding of span relationships, refer to the Traces documentation in OpenTelemetry. In case your execution tags on different spans appear on different invocations than what you would expect, get in touch with Lumigo support.
Execution Tag Limitations
- Up to a max of 50 execution tag keys per invocation in Lumigo, irrespective of how many spans are part of the invocation or how many values each execution tag has.
- The
key
of an execution tag cannot contain the.
character. for example,lumigo.execution_tags.my.tag
is not a valid tag. The OpenTelemetrySpan.set_attribute()
API will not fail or log warnings, but that will be displayed asmy
in Lumigo. - Each execution tag key can be at most 50 characters long. The
lumigo.execution_tags.
prefix does not count against the 50 characters limit. - Each execution tag value can be at most 70 characters long.
Programmatic Errors
Programmatic Errors allow you to customize errors, on top of monitoring and troubleshooting issues that should not necessarily interfere with the service. For example, an application tries to remove a user who does not exist. These custom errors can be captured by adding just a few lines of additional code to your application.
Programmatic errors indicate that a non-fatal error occurred, such as an application error. You can also log programmatic errors, track custom error issues, and trigger Alerts.
Creating a Programmatic Error
You can create Programmatic errors by adding span events with a custom attribute being set with the key name lumigo.type
.
For example, you can add a programmatic error as follows:
Attributes eventAttributes = Attributes.of(
AttributeKey.stringKey("lumigo.type"), "<error-type>"
);
Span.current().addEvent("<error-message>", eventAttributes);
Supported runtimes
The following runtimes are supported:
- JDK: 11.x, 17.x, 21.x
Supported packages
See the latest list of updated packages supported out of the box and regularly tested by Lumigo here: here
Baseline setup
The Lumigo OpenTelemetry Distribution will automatically create the following OpenTelemetry constructs provided to a NodeTraceProvider
.
Resource attributes
SDK resource attributes
-
The attributes from the default resource:
telemetry.sdk.language
:java
telemetry.sdk.name
:opentelemetry
telemetry.sdk.version
: depends on the version of theio.opentelemetry.instrumentation:opentelemetry-instrumentation-bom
included in the dependencies
-
The
lumigo.distro.version
containing the version of the Lumigo OpenTelemetry Distribution for Java
Process resource attributes
-
The following
process.runtime.*
attributes as specified in the Process Semantic Conventions:process.runtime.description
process.runtime.name
process.runtime.version
Amazon ECS resource attributes
If the instrumented application is running on the Amazon Elastic Container Service (ECS):
cloud.provider
attribute with valueaws
cloud.platform
with valueaws_ecs
container.name
with the hostname of the ECS Task containercontainer.id
with the ID of the Docker container (based on the cgroup id)
If the ECS task uses the ECS agent v1.4.0, and therefore has access to Task metadata endpoint version 4, the following experimental attributes are added, as specified in the AWS ECS Resource Attributes specification:
aws.ecs.container.arn
aws.ecs.cluster.arn
aws.ecs.launchtype
aws.ecs.task.arn
aws.ecs.task.family
aws.ecs.task.revision
Kubernetes resource attributes
k8s.pod.uid
with the Pod identifier is supported for both cgroups v1 and v2.k8s.container.name
with the name of the container.
Span exporters
- If the
LUMIGO_TRACER_TOKEN
environment variable is set: anOTLP Exporter
is configured to push data to Lumigo. - If the
LUMIGO_DEBUG_SPANDUMP
environment variable is set to a path-like value, we define theFileLoggingSpanExporter
, to save to file the spans collected.
NOTE: Do not use this flag in production unless you need to troubleshoot trace data!
SDK configuration
The default maximum span attribute length is 1024. This can be overwritten with the OTEL_SPAN_ATTRIBUTE_VALUE_LENGTH_LIMIT
and OTEL_ATTRIBUTE_VALUE_LENGTH_LIMIT
environment variables, but when sending data to Lumigo, the span attributes will be truncated to a maximum length of 2048.
The Lumigo OpenTelemetry Java Distribution automatically configures a BatchSpanProcessor
to export tracing data to Lumigo, with the following settings:
otel.bsp.schedule.delay
:10ms
. This is how the minimum delay before theBatchSpanProcessor
will flush data to Lumigo.otel.bsp.max.export.batch.size
:100
. This is the maximum amount of spans queued before flushing. When the limit is passed, a flush will occur.otel.bsp.export.timeout
:1s
. This is the timeout for flushing data to Lumigo.
The metrics and logs exporters are disabled (otel.logs.exporter
and otel.metrics.exporter
are set to none
) as Lumigo OpenTelemetry endpoint currently does not provide support for the /v1/metrics
and /v1/logs
endpoints.
Contributing
For guidelines on contributing, please see CONTRIBUTING.md.
Updated about 3 hours ago