Logging Best Practices in Go

Effective logging is crucial for monitoring, debugging, and maintaining Go applications. This guide covers best practices for implementing logging in your Go applications.

Understanding Logging Fundamentals

1. Logging Levels

Standard logging levels and their use cases:

1. Debug:

   • Detailed information for debugging
   • Development environment
   • Verbose output
   • Performance impact acceptable

2. Info:

   • General operational information
   • Normal operations
   • System state changes
   • Configuration settings

3. Warning:

   • Potential issues
   • Degraded service
   • Recoverable errors
   • Resource limitations

4. Error:

   • Application errors
   • Failed operations
   • System issues
   • Requires attention

5. Fatal:

   • Critical failures
   • System cannot continue
   • Immediate attention required
   • Process termination

Structured Logging

1. Using zerolog

zerolog is a fast and structured logging library:

package main

import (
    "os"
    "github.com/rs/zerolog"
    "github.com/rs/zerolog/log"
)

func init() {
    // Configure global logger
    zerolog.TimeFieldFormat = zerolog.TimeFormatUnix
    log.Logger = log.Output(zerolog.ConsoleWriter{Out: os.Stdout})
}

func main() {
    // Structured logging
    log.Info().
        Str("service", "api").
        Int("port", 8080).
        Msg("server starting")
        
    // With error
    if err := process(); err != nil {
        log.Error().
            Err(err).
            Str("service", "api").
            Msg("processing failed")
    }
}

2. Using zap

zap is another popular structured logging library:

package main

import (
    "go.uber.org/zap"
    "go.uber.org/zap/zapcore"
)

func main() {
    // Production logger
    logger, _ := zap.NewProduction()
    defer logger.Sync()
    
    // Structured logging
    logger.Info("server starting",
        zap.String("service", "api"),
        zap.Int("port", 8080))
        
    // With error
    if err := process(); err != nil {
        logger.Error("processing failed",
            zap.Error(err),
            zap.String("service", "api"))
    }
}

Best Practices

1. Log Configuration

Configure logging appropriately for each environment:

type LogConfig struct {
    Level      string
    Format     string
    Output     string
    TimeFormat string
}

func NewLogger(config LogConfig) zerolog.Logger {
    // Set log level
    level, err := zerolog.ParseLevel(config.Level)
    if err != nil {
        level = zerolog.InfoLevel
    }
    zerolog.SetGlobalLevel(level)
    
    // Configure output
    var output io.Writer = os.Stdout
    if config.Output != "" {
        file, err := os.OpenFile(config.Output, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644)
        if err == nil {
            output = file
        }
    }
    
    // Configure format
    if config.Format == "console" {
        output = zerolog.ConsoleWriter{
            Out:        output,
            TimeFormat: config.TimeFormat,
        }
    }
    
    return zerolog.New(output).With().Timestamp().Logger()
}

2. Contextual Logging

Add context to your logs:

// Request context logging
func LoggingMiddleware(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        // Start timer
        start := time.Now()
        
        // Create request ID
        requestID := uuid.New().String()
        ctx := context.WithValue(r.Context(), "request_id", requestID)
        
        // Add logger to context
        logger := log.With().
            Str("request_id", requestID).
            Str("method", r.Method).
            Str("path", r.URL.Path).
            Logger()
        ctx = logger.WithContext(ctx)
        
        // Process request
        next.ServeHTTP(w, r.WithContext(ctx))
        
        // Log request completion
        logger.Info().
            Dur("duration_ms", time.Since(start)).
            Msg("request completed")
    })
}

3. Error Logging

Proper error logging patterns:

// Bad: Logging without context
log.Error().Msg(err.Error())

// Good: Structured error logging
log.Error().
    Err(err).
    Str("user_id", user.ID).
    Str("action", "payment_process").
    Msg("failed to process payment")

// Better: With stack trace
log.Error().
    Err(err).
    Stack().
    Str("user_id", user.ID).
    Str("action", "payment_process").
    Msg("failed to process payment")

Common Patterns

1. Request Logging

Log HTTP requests effectively:

type ResponseRecorder struct {
    http.ResponseWriter
    Status int
    Size   int64
}

func RequestLogger(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        start := time.Now()
        
        // Record response
        recorder := &ResponseRecorder{
            ResponseWriter: w,
            Status:        http.StatusOK,
        }
        
        next.ServeHTTP(recorder, r)
        
        // Log request details
        log.Info().
            Str("method", r.Method).
            Str("path", r.URL.Path).
            Str("remote_addr", r.RemoteAddr).
            Int("status", recorder.Status).
            Int64("size", recorder.Size).
            Dur("duration", time.Since(start)).
            Msg("request handled")
    })
}

2. Performance Logging

Track performance metrics:

func TimeTrack(start time.Time, name string) {
    elapsed := time.Since(start)
    
    log.Info().
        Str("operation", name).
        Dur("duration_ms", elapsed).
        Msg("operation completed")
}

func ProcessOrder(order Order) error {
    defer TimeTrack(time.Now(), "process_order")
    
    // Process order...
    return nil
}

Testing with Logs

1. Log Capture

Capture and verify logs in tests:

func TestOperation(t *testing.T) {
    // Create buffer for log output
    var buf bytes.Buffer
    log.Logger = log.Output(&buf)
    
    // Perform operation
    err := Operation()
    require.NoError(t, err)
    
    // Verify log output
    logOutput := buf.String()
    assert.Contains(t, logOutput, "operation completed")
    assert.Contains(t, logOutput, "success")
}

2. Log Level Control

Control log levels in tests:

func TestWithLogLevel(t *testing.T) {
    // Save current level
    prevLevel := zerolog.GlobalLevel()
    defer func() {
        zerolog.SetGlobalLevel(prevLevel)
    }()
    
    // Set test level
    zerolog.SetGlobalLevel(zerolog.ErrorLevel)
    
    // Run test...
}

Production Considerations

1. Log Aggregation

Prepare logs for aggregation:

type LogEntry struct {
    Level     string    `json:"level"`
    Timestamp time.Time `json:"timestamp"`
    Message   string    `json:"message"`
    Service   string    `json:"service"`
    TraceID   string    `json:"trace_id"`
    SpanID    string    `json:"span_id"`
    Metadata  struct {
        Host    string `json:"host"`
        Version string `json:"version"`
    } `json:"metadata"`
}

2. Log Rotation

Implement log rotation:

func setupLogging() error {
    logFile := &lumberjack.Logger{
        Filename:   "/var/log/myapp/app.log",
        MaxSize:    100, // megabytes
        MaxBackups: 3,
        MaxAge:     28,   // days
        Compress:   true, // compress rotated files
    }
    
    log.Logger = log.Output(zerolog.MultiLevelWriter(
        zerolog.ConsoleWriter{Out: os.Stdout},
        logFile,
    ))
    
    return nil
}

Next Steps

• Learn about Error Handling
• Explore Configuration Management
• Study Performance