Documentation

Logs with Go

Overview

Lumigo supports OpenTelemetry-based instrumentation for Go applications, enabling you to collect and export trace data for enhanced observability. While Lumigo doesn't provide a dedicated OpenTelemetry distribution for Go, you can configure your application to send telemetry data to Lumigo using either the OpenTelemetry Go SDK or the Direct-to-Collector workflow, and any appropriate exporters.​

Scope

Logs using OpenTelemetry for Go are currently not auto-instrumented. You must use a logging library that integrates manually with OpenTelemetry. OpenTelemetry has instrumentation available for most of the most commonly used Go libraries, but Lumigo recommends using Logrus for ease of use.

Basics

Instrumentation setup

For instrumentation setup, follow the instructions in the linked article. It provides instructions on manual, vanilla OpenTelemetry instrumentation . Refer to OpenTelemetry's documentation on instrumenting for further details.

Logging with OpenTelemetry

Unlike metrics or traces, there is no user facing API provided by OpenTelemetry. Instead, tooling that enables bridging logs from pre-existing log libraries such as Logrus is provided.

To use logging with OpenTelemetry for Go applications, you can use either the experimental Direct-to-Collector workflow , or a logs SDK workflow. Both workflows cater to different requirements and needs.

Direct-to-Collector Workflow

To set up the Direct-To-Collect workflow, follow the instructions here . Lumigo provides [Collector] support, with details on setting it up here .

Logs SDK

The usual configuration of a log SDK uses an OTLP exporter alongside a batching log processor. To enable logs with Go for OpenTelemetry, you need to first set up a LoggerProvider with a Log Bridge:

he logs SDK dictates how logs are processed when using the direct-to-Collector workflow. No log SDK is needed when using the log forwarding workflow.

The typical log SDK configuration installs a batching log record processor with an OTLP exporter.

To enable logs in your app, you’ll need to have an initialized LoggerProvider that will let you use a Log Bridge.

If a LoggerProvider is not created, the OpenTelemetry APIs for logs will use a no-op implementation and fail to generate data. Therefore, you have to modify the source code to include the SDK initialization code using the following packages:

Install the necessary Go modules:

go get go.opentelemetry.io/otel \
  go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp \
  go.opentelemetry.io/otel/sdk \
  go.opentelemetry.io/otel/sdk/log

Then complete the initialization of your LoggerProvider:

package main

import (
	"context"
	"fmt"

	"go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp"
	"go.opentelemetry.io/otel/log/global"
	"go.opentelemetry.io/otel/sdk/log"
	"go.opentelemetry.io/otel/sdk/resource"
	semconv "go.opentelemetry.io/otel/semconv/v1.26.0"
)

func main() {
	ctx := context.Background()

	// Create resource.
	res, err := newResource()
	if err != nil {
		panic(err)
	}

	// Create a logger provider.
	// You can pass this instance directly when creating bridges.
	loggerProvider, err := newLoggerProvider(ctx, res)
	if err != nil {
		panic(err)
	}

	// Handle shutdown properly so nothing leaks.
	defer func() {
		if err := loggerProvider.Shutdown(ctx); err != nil {
			fmt.Println(err)
		}
	}()

	// Register as global logger provider so that it can be accessed global.LoggerProvider.
	// Most log bridges use the global logger provider as default.
	// If the global logger provider is not set then a no-op implementation
	// is used, which fails to generate data.
	global.SetLoggerProvider(loggerProvider)
}

func newResource() (*resource.Resource, error) {
	return resource.Merge(resource.Default(),
		resource.NewWithAttributes(semconv.SchemaURL,
			semconv.ServiceName("my-service"),
			semconv.ServiceVersion("0.1.0"),
		))
}

func newLoggerProvider(ctx context.Context, res *resource.Resource) (*log.LoggerProvider, error) {
	exporter, err := otlploghttp.New(ctx)
	if err != nil {
		return nil, err
	}
	processor := log.NewBatchProcessor(exporter)
	provider := log.NewLoggerProvider(
		log.WithResource(res),
		log.WithProcessor(processor),
	)
	return provider, nil
}

After setting up the LoggerProvider, configure it with Lumigo's recommended settings:

func newLoggerProvider(ctx context.Context, res *resource.Resource) (*log.LoggerProvider, error) {
	logger.Info("Initializing Lumigo log collection...")

	// Configure the log exporter with Lumigo's recommended settings
	exporter, err := otlploghttp.New(ctx,
		otlploghttp.WithEndpoint("ga-otlp.lumigo-tracer-edge.golumigo.com"),
		otlploghttp.WithURLPath("/v1/logs"),
		otlploghttp.WithHeaders(map[string]string{
			"Authorization": "LumigoToken t_4ba3b526684741d3bce8a",
		}),
		otlploghttp.WithTimeout(10*time.Second),
		otlploghttp.WithRetry(otlploghttp.RetryConfig{
			Enabled:         true,
			InitialInterval: 1 * time.Second,
			MaxInterval:     5 * time.Second,
			MaxElapsedTime:  30 * time.Second,
		}),
	)
	if err != nil {
		logger.WithError(err).Error("Failed to create log exporter")
		return nil, fmt.Errorf("failed to create log exporter: %w", err)
	}

	// Create a batch processor
	processor := log.NewBatchProcessor(exporter)

	provider := log.NewLoggerProvider(
		log.WithResource(res),
		log.WithProcessor(processor),
	)

	// Set the global logger provider
	global.SetLoggerProvider(provider)
	logger.Info("Lumigo log collection initialized successfully")

	// Emit a test log via OpenTelemetry (correct API)
	otelLogger := global.GetLoggerProvider().Logger("otelapp")
	rec := otellog.Record{}
	rec.SetBody(otellog.StringValue("This is a test log sent to Lumigo"))
	rec.SetBody(otellog.StringValue("Hey Chris how are you doing?"))
	otelLogger.Emit(ctx, rec)

	return provider, nil
}

Then, create a new logrus logger:

func initLogger() {
	ctx := context.Background()

	// Create a new logrus logger
	logger = logrus.New()

	// Set JSON formatter for better parsing
	logger.SetFormatter(&logrus.JSONFormatter{
		TimestampFormat: time.RFC3339,
		FieldMap: logrus.FieldMap{
			logrus.FieldKeyTime:  "@timestamp",
			logrus.FieldKeyLevel: "@level",
			logrus.FieldKeyMsg:   "@message",
		},
	})

	logger.SetOutput(os.Stdout)
	logger.SetLevel(logrus.InfoLevel)

	// Add some default fields
	logger = logger.WithFields(logrus.Fields{
		"service": "otelapp",
		"version": "1.0.0",
	}).Logger

	// Initialize OpenTelemetry Logger Provider
	loggerProvider, err := newLoggerProvider(ctx, Resource())
	if err != nil {
		logger.WithError(err).Fatal("Failed to initialize logger provider")
	}

	defer func() {
		if err := loggerProvider.Shutdown(ctx); err != nil {
			logger.WithError(err).Error("Failed to shutdown logger provider")
		}
	}()

	logger.Info("Logger initialized")
}

you can integrate logging into your application using the logrus package. The example below shows how to log incoming HTTP requests:

func helloHandler(w http.ResponseWriter, r *http.Request) {
	logger.WithFields(logrus.Fields{
		"method": r.Method,
		"path":   r.URL.Path,
	}).Info("Request received")

	w.Write([]byte("Hello, World!"))
}

func main() {
	// Initialize OTLP Log Exporter
	initLogExporter()

	// Setup HTTP server
	http.HandleFunc("/", helloHandler)

	port := ":8080"
	logger.WithField("port", port).Info("Starting server")
	if err := http.ListenAndServe(port, nil); err != nil {
		logger.WithError(err).Fatal("Server failed to start")
	}
}

Finally, initialize your solutions:

func main() {
	ctx := context.Background()

	// Initialize Logger
	initLogger()

	// Initialize Lumigo Tracer
	shutdown, err := InitiateLumigoTracer(ctx)
	if err != nil {
		logger.WithError(err).Fatal("Failed to initialize tracer")
	}
	defer shutdown(ctx)

	// Setup HTTP server with logging middleware
	mux := http.NewServeMux()
	mux.HandleFunc("/", helloHandler)

	// Add health check endpoint
	mux.HandleFunc("/health", func(w http.ResponseWriter, r *http.Request) {
		logger.WithField("endpoint", "/health").Debug("Health check requested")
		w.WriteHeader(http.StatusOK)
		w.Write([]byte("OK"))
	})

	port := ":8080"
	server := &http.Server{
		Addr:         port,
		Handler:      mux,
		ReadTimeout:  10 * time.Second,
		WriteTimeout: 10 * time.Second,
	}

	logger.WithField("port", port).Info("Starting server")
	if err := server.ListenAndServe(); err != nil {
		logger.WithError(err).Fatal("Server failed to start")
	}
}

With all this done, you should be ready to go with logging for Go applications for OpenTelemetry through Lumigo.

Additional Information

Additional functionalities like using a Log Bridge can be found in OpenTelemetry's documentation.