Logs with Go
Overview
Lumigo supports OpenTelemetry-based instrumentation for Go applications, enabling you to collect and export trace data for enhanced observability. While Lumigo doesn't provide a dedicated OpenTelemetry distribution for Go, you can configure your application to send telemetry data to Lumigo using either the OpenTelemetry Go SDK or the Direct-to-Collector workflow, and any appropriate exporters.
Scope
Logs using OpenTelemetry for Go are currently not auto-instrumented. You must use a logging library that integrates manually with OpenTelemetry. OpenTelemetry has instrumentation available for most of the most commonly used Go libraries, but Lumigo recommends using Logrus for ease of use.
Basics
Instrumentation setup
For instrumentation setup, follow the instructions in the linked article. It provides instructions on manual, vanilla OpenTelemetry instrumentation . Refer to OpenTelemetry's documentation on instrumenting for further details.
Logging with OpenTelemetry
Unlike metrics or traces, there is no user facing API provided by OpenTelemetry. Instead, tooling that enables bridging logs from pre-existing log libraries such as Logrus is provided.
To use logging with OpenTelemetry for Go applications, you can use either the experimental Direct-to-Collector workflow , or a logs SDK workflow. Both workflows cater to different requirements and needs.
Direct-to-Collector Workflow
To set up the Direct-To-Collect workflow, follow the instructions here . Lumigo provides [Collector] support, with details on setting it up here .
Logs SDK
The usual configuration of a log SDK uses an OTLP exporter alongside a batching log processor. To enable logs with Go for OpenTelemetry, you need to first set up a LoggerProvider
with a Log Bridge:
he logs SDK dictates how logs are processed when using the direct-to-Collector workflow. No log SDK is needed when using the log forwarding workflow.
The typical log SDK configuration installs a batching log record processor with an OTLP exporter.
To enable logs in your app, you’ll need to have an initialized LoggerProvider that will let you use a Log Bridge.
If a LoggerProvider is not created, the OpenTelemetry APIs for logs will use a no-op implementation and fail to generate data. Therefore, you have to modify the source code to include the SDK initialization code using the following packages:
- go.opentelemetry.io/otel
- go.opentelemetry.io/otel/sdk/log
- go.opentelemetry.io/otel/sdk/resource
- go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp
Install the necessary Go modules:
go get go.opentelemetry.io/otel \
go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp \
go.opentelemetry.io/otel/sdk \
go.opentelemetry.io/otel/sdk/log
Then complete the initialization of your LoggerProvider
:
package main
import (
"context"
"fmt"
"go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp"
"go.opentelemetry.io/otel/log/global"
"go.opentelemetry.io/otel/sdk/log"
"go.opentelemetry.io/otel/sdk/resource"
semconv "go.opentelemetry.io/otel/semconv/v1.26.0"
)
func main() {
ctx := context.Background()
// Create resource.
res, err := newResource()
if err != nil {
panic(err)
}
// Create a logger provider.
// You can pass this instance directly when creating bridges.
loggerProvider, err := newLoggerProvider(ctx, res)
if err != nil {
panic(err)
}
// Handle shutdown properly so nothing leaks.
defer func() {
if err := loggerProvider.Shutdown(ctx); err != nil {
fmt.Println(err)
}
}()
// Register as global logger provider so that it can be accessed global.LoggerProvider.
// Most log bridges use the global logger provider as default.
// If the global logger provider is not set then a no-op implementation
// is used, which fails to generate data.
global.SetLoggerProvider(loggerProvider)
}
func newResource() (*resource.Resource, error) {
return resource.Merge(resource.Default(),
resource.NewWithAttributes(semconv.SchemaURL,
semconv.ServiceName("my-service"),
semconv.ServiceVersion("0.1.0"),
))
}
func newLoggerProvider(ctx context.Context, res *resource.Resource) (*log.LoggerProvider, error) {
exporter, err := otlploghttp.New(ctx)
if err != nil {
return nil, err
}
processor := log.NewBatchProcessor(exporter)
provider := log.NewLoggerProvider(
log.WithResource(res),
log.WithProcessor(processor),
)
return provider, nil
}
After setting up the LoggerProvider
, configure it with Lumigo's recommended settings:
func newLoggerProvider(ctx context.Context, res *resource.Resource) (*log.LoggerProvider, error) {
logger.Info("Initializing Lumigo log collection...")
// Configure the log exporter with Lumigo's recommended settings
exporter, err := otlploghttp.New(ctx,
otlploghttp.WithEndpoint("ga-otlp.lumigo-tracer-edge.golumigo.com"),
otlploghttp.WithURLPath("/v1/logs"),
otlploghttp.WithHeaders(map[string]string{
"Authorization": "LumigoToken t_4ba3b526684741d3bce8a",
}),
otlploghttp.WithTimeout(10*time.Second),
otlploghttp.WithRetry(otlploghttp.RetryConfig{
Enabled: true,
InitialInterval: 1 * time.Second,
MaxInterval: 5 * time.Second,
MaxElapsedTime: 30 * time.Second,
}),
)
if err != nil {
logger.WithError(err).Error("Failed to create log exporter")
return nil, fmt.Errorf("failed to create log exporter: %w", err)
}
// Create a batch processor
processor := log.NewBatchProcessor(exporter)
provider := log.NewLoggerProvider(
log.WithResource(res),
log.WithProcessor(processor),
)
// Set the global logger provider
global.SetLoggerProvider(provider)
logger.Info("Lumigo log collection initialized successfully")
// Emit a test log via OpenTelemetry (correct API)
otelLogger := global.GetLoggerProvider().Logger("otelapp")
rec := otellog.Record{}
rec.SetBody(otellog.StringValue("This is a test log sent to Lumigo"))
rec.SetBody(otellog.StringValue("Hey Chris how are you doing?"))
otelLogger.Emit(ctx, rec)
return provider, nil
}
Then, create a new logrus
logger:
func initLogger() {
ctx := context.Background()
// Create a new logrus logger
logger = logrus.New()
// Set JSON formatter for better parsing
logger.SetFormatter(&logrus.JSONFormatter{
TimestampFormat: time.RFC3339,
FieldMap: logrus.FieldMap{
logrus.FieldKeyTime: "@timestamp",
logrus.FieldKeyLevel: "@level",
logrus.FieldKeyMsg: "@message",
},
})
logger.SetOutput(os.Stdout)
logger.SetLevel(logrus.InfoLevel)
// Add some default fields
logger = logger.WithFields(logrus.Fields{
"service": "otelapp",
"version": "1.0.0",
}).Logger
// Initialize OpenTelemetry Logger Provider
loggerProvider, err := newLoggerProvider(ctx, Resource())
if err != nil {
logger.WithError(err).Fatal("Failed to initialize logger provider")
}
defer func() {
if err := loggerProvider.Shutdown(ctx); err != nil {
logger.WithError(err).Error("Failed to shutdown logger provider")
}
}()
logger.Info("Logger initialized")
}
you can integrate logging into your application using the logrus
package. The example below shows how to log incoming HTTP requests:
func helloHandler(w http.ResponseWriter, r *http.Request) {
logger.WithFields(logrus.Fields{
"method": r.Method,
"path": r.URL.Path,
}).Info("Request received")
w.Write([]byte("Hello, World!"))
}
func main() {
// Initialize OTLP Log Exporter
initLogExporter()
// Setup HTTP server
http.HandleFunc("/", helloHandler)
port := ":8080"
logger.WithField("port", port).Info("Starting server")
if err := http.ListenAndServe(port, nil); err != nil {
logger.WithError(err).Fatal("Server failed to start")
}
}
Finally, initialize your solutions:
func main() {
ctx := context.Background()
// Initialize Logger
initLogger()
// Initialize Lumigo Tracer
shutdown, err := InitiateLumigoTracer(ctx)
if err != nil {
logger.WithError(err).Fatal("Failed to initialize tracer")
}
defer shutdown(ctx)
// Setup HTTP server with logging middleware
mux := http.NewServeMux()
mux.HandleFunc("/", helloHandler)
// Add health check endpoint
mux.HandleFunc("/health", func(w http.ResponseWriter, r *http.Request) {
logger.WithField("endpoint", "/health").Debug("Health check requested")
w.WriteHeader(http.StatusOK)
w.Write([]byte("OK"))
})
port := ":8080"
server := &http.Server{
Addr: port,
Handler: mux,
ReadTimeout: 10 * time.Second,
WriteTimeout: 10 * time.Second,
}
logger.WithField("port", port).Info("Starting server")
if err := server.ListenAndServe(); err != nil {
logger.WithError(err).Fatal("Server failed to start")
}
}
With all this done, you should be ready to go with logging for Go applications for OpenTelemetry through Lumigo.
Additional Information
Additional functionalities like using a Log Bridge can be found in OpenTelemetry's documentation.
Updated 2 days ago