Logging

Use the logger attached to the execution context. Keep messages short and put useful context in structured fields so the same record works in the console, files, and Grafana. Logs describe observations; they are not a transactional audit database.

Write your first application log#

Inside a service, handler, command, or task, use the supplied operation context:

// Import github.com/wisalgroup/labena/pkg/logging.
logging.FromContext(ctx).InfoContext(ctx, "profile updated", "user_id", userID)

The app supplies the logger and request/task fields; InfoContext also makes the current trace context available. Choose fields that explain the outcome without copying the request body. Logging does not require importing application resources.

Select outputs in the environment:

LOG_CHANNELS=stderr,observability

That keeps a readable console and exports structured logs directly to the collector. Add file if you also want local daily files. Formats, levels, paths, and rotation are defined once in internal/app/config/observability.go:

The application's default selection is only stderr; full local profiles select more channels. An INFO request may appear in the console and Loki but not the WARN file. That is a channel-level choice, not a missing log.

Local log viewer#

Enable the embedded viewer when you want to inspect logs without a collector or Grafana. It shares the application HTTP listener at /admin/logs/ and has its own Basic Auth credentials, independent of Identity and the queue dashboard:

LOG_CHANNELS=stderr,file
LOG_VIEWER_ENABLED=true
LOG_VIEWER_USERNAME=operator
LOG_VIEWER_PASSWORD=replace-with-a-long-random-password

Serve an entry point with app.Processes{HTTP: true} and open http://localhost:8080/admin/logs/. The browser prompts for the credentials. Set LOG_VIEWER_ENABLED=false to omit the routes and live capture entirely. Tracing, metrics and the observability log channel can all remain disabled. Keep production access behind HTTPS and an operator-restricted ingress.

The viewer provides:

• Live refresh every three seconds, pause/resume, and manual refresh.
• Channel/file selection, severity filtering and case-insensitive text search, including structured fields and request IDs.
• Newest-first paging, expandable records, copy, and JSONL export of visible entries.
• JSON and readable Tint file previews, with light/dark appearance and keyboard controls.

Live records are copies of entries accepted by the application's named stdout, stderr and file channels. Their existing minimum levels still apply. A record sent to two selected channels appears once for each channel; filter by channel when inspecting one output. Context fields and groups are preserved. Successful viewer access logs still go to normal outputs but are excluded from live capture to avoid polling noise; failed access remains visible.

Live capture is process-local and starts with application construction. It cannot recover earlier stdout/stderr output, intercept unrelated fmt.Print or global loggers, or read another process's stdout. Worker/scheduler history is visible through configured files mounted into the web process; use separate paths for each writing process. Use centralized observability for multi-instance aggregation.

File history uses only file channels defined in config/observability.go, including their existing single-file or daily names. It does not enable a channel, change retention, create directories, or accept arbitrary paths. Missing directories have no files to list. Refresh the sources after daily rotation or removal. Text files retain their original timestamps in the message; JSON timestamps also appear in the time column. Only complete newline-terminated records are shown.

The default live buffer retains at most 2,000 entries and 8 MiB of record strings, plus bounded entry/index overhead. Oldest entries expire when either limit is reached; the view reports the number expired. LOG_VIEWER_BUFFER_ENTRIES and LOG_VIEWER_BUFFER_BYTES adjust these bounds. Each record preview is limited to 16 KiB; exports contain those same visible previews, not complete original files. Each file query reads at most 1 MiB and returns at most 500 entries. The page requests 200 at a time and displays up to 1,000; load older or narrow the filters to continue. The file directory is limited to fewer than 4,096 entries. Concurrent viewer requests are bounded; busy clients receive a retry response. File browsing is a best-effort view of changing files, not a retained snapshot or audit archive.

The UI and every data/asset route require Basic Auth. Responses are non-cacheable, cross-origin data reads are rejected, and log text is never rendered as HTML. Searches use POST bodies and stay out of page URLs and browser storage. Logs may still contain sensitive application data: the viewer does not redact arbitrary fields, so keep secrets out of logs and restrict access appropriately.

Preserve error identity#

Return errors with %w and inspect them with errors.Is/errors.As. The canonical error handling guide explains classification, owned sentinels, validation messages, and cancellation. Avoid logging the same returned failure at every layer.

Translate at the boundary#

HTTP handlers use response.Error(ctx, err) from internal/platform/api/response. The boundary hides raw operational details and records safe failure categories. See boundary translation and response bodies.

HTTP request logs#

The app automatically records each completed HTTP request using the context logger. Records include the server name, method, route template, status, and duration. Health probes and unmatched routes are included. Successful and redirect responses use INFO, client errors WARN, and server errors ERROR.

The same JSON or tint settings apply to these records. There is one built-in access logger. See Middleware for fields and customization.

Retrieve the context logger#

Application runners, commands, and HTTP requests receive a logger through their contexts:

logger := logging.FromContext(ctx)
logger.InfoContext(ctx, "operation completed",
    "user_id", userID,
    "event_id", eventID,
)

The returned value is a standard *slog.Logger. Use structured key-value fields. FromContext falls back to slog.Default() if no logger was attached, which also makes ordinary standalone contexts usable.

Choose fields appropriate to your data policy. Do not log passwords, connection URLs containing credentials, authorization headers, full configuration structs, or complete request payloads.

Add fields for one operation#

Derive a logger and pass it through the operation's context:

logger := logging.FromContext(ctx).With("component", "identity", "user_id", userID)
ctx = logging.WithLogger(ctx, logger)
logger.InfoContext(ctx, "starting operation")

Pass this derived ctx to downstream calls so they can retrieve the same fields. WithLogger does not mutate the parent context. It requires a non-nil logger.

For HTTP middleware, call next(huma.WithContext(request, ctx)), where request is the Huma context. See the middleware example.

Choose the output format#

Edit each channel's Options in internal/app/config/observability.go. stderr and stdout use tint with colors; file uses tint without colors. observability uses structured OTLP/HTTP delivery. Console and OTLP channels default to INFO; the readable file channel defaults to WARN. These settings are hardcoded per channel, not loaded from environment variables.

The Watermill adapter maps routine library info messages to DEBUG, including handler registration, routing and pub/sub cleanup. This keeps ordinary CLI help and completion free of infrastructure chatter at INFO. Watermill failures still use ERROR; application logs retain their chosen severity. Set a channel's level to slog.LevelDebug in Go when those library diagnostics are needed.

Select a console channel with LOG_CHANNELS=stderr or LOG_CHANNELS=stdout. To use JSON, change that channel's format to logging.FormatJSON. Rebuild after changing Go channel definitions and restart after changing the selected defaults. Tint affects application output, not Tilt's own build or Kubernetes event messages.

Configure named channels#

Channel definitions live in internal/app/config/observability.go. A channel is a named output with its own driver, format and minimum level. The built-in drivers are stdout, stderr, file and otlp.

Select one or more default channels with a comma-separated list:

LOG_CHANNELS=stdout,file

Every normal context log now goes to both outputs. The current file channel uses storage/logs/app-YYYY-MM-DD.log with 14-day retention, as defined in Go. Unknown or repeated channel names fail configuration validation.

Channels are defined in Go; LOG_CHANNELS selects them and OTEL_EXPORTER_OTLP_LOGS_ENDPOINT configures direct log delivery. Add another channel inside LoggingConfig.Channels:

channels["audit"] = logging.Channel{
    Driver: "file",
    Path:   "storage/logs/audit.log",
    Options: logging.Options{
        Format:  logging.FormatJSON,
        Level:   slog.LevelInfo,
        NoColor: true,
    },
}

The configuration file already imports logging and log/slog. You can set different options on each channel, including a readable console and JSON files. Keep paths, formats, levels and color settings in these channel definitions.

Single or daily files#

The file driver supports two modes, configured per channel in internal/app/config/observability.go:

channels["file"] = logging.Channel{
    Driver:   "file",
    Path:     "storage/logs/app.log",
    FileMode: logging.FileDaily,
    KeepDays: 7,
    Options:  logging.Options{Format: logging.FormatJSON, Level: slog.LevelInfo},
}

• FileSingle (also the zero-value default) appends to app.log. KeepDays must be zero; records inside a single file cannot be expired individually.
• FileDaily appends to app-YYYY-MM-DD.log, using the UTC date at write time. With no extension, app becomes app-YYYY-MM-DD. It reuses the day's file after restart and rotates on the first write of a new day.
• KeepDays: 7 keeps today and the previous six UTC dates. 0 keeps all dated files; negative values are invalid. Retention is based on filename dates, not modification times or the number of files.

Cleanup runs when the channel opens and on daily rotation. No background goroutine or scheduler is needed; idle or stopped applications do not perform cleanup. Only regular files with that channel's exact dated naming pattern are eligible; undated files, unrelated names, directories and symlinks are left alone. Use a distinct path per channel and per application instance, and do not combine this writer with external rotation of its files. Writes and rotation are serialized within a channel; there is no cross-process rotation coordination.

A failure to open today's file fails channel opening. During rotation, it is reported through OnError and the next write retries; the failed record is not redirected into yesterday's file. Retention failures are also reported; after a successful rotation the current record is still written. Cleanup is retried on the next rotation or reopen. Daily retention limits age, not total disk usage.

Log to selected channels#

Select outputs from the context logger:

logger, err := logging.Channels(ctx, "audit")
if err != nil {
    return err
}
logger.InfoContext(ctx, "order approved", "user_id", userID)

This record goes only to audit. To select several outputs:

logger, err := logging.Channels(ctx, "audit", "stdout")
if err != nil {
    return err
}
logger.InfoContext(ctx, "manual adjustment", "user_id", userID)

Selection preserves fields and groups added through With and WithGroup, including request and job fields. It does not change the default logger. To use the selection throughout an operation, pass logging.WithLogger(ctx, logger) to that operation.

During dependency assembly, use logging.Select(deps.Logger, "audit") and inject the resulting *slog.Logger into the service that needs it. No application dependency container is added to context. Selection requires a logger created by the channel manager; a standalone logging.New logger or an unrelated handler has no named channels and returns an error.

Add a logging driver#

pkg/logging accepts explicit driver functions that return a slog.Handler and an optional owned closer. Add a driver to the configuration returned by LoggingConfig.Channels:

return logging.Config{
    Default: names,
    Channels: channels,
    Drivers: map[string]logging.Driver{
        "discard": func(ch logging.Channel) (slog.Handler, io.Closer, error) {
            logger, err := logging.New(io.Discard, ch.Options)
            if err != nil {
                return nil, nil, err
            }
            return logger.Handler(), nil, nil
        },
    },
}

Import io for this example and declare a channel with Driver: "discard" to use it. A real integration can return its own concurrency-safe handler and closer. A driver must clean up partial construction before returning an error; return a nil closer for borrowed resources. Built-in driver names cannot be overridden.

Ownership and output failures#

Open channels when needed#

Successfully opened outputs remain manager-owned, including when a later channel in the same selection fails. App closes them after other dependencies finish. Table-publication commands follow the same rules: stderr/stdout need no writable log directory, but a selected file channel does.

Resolve paths and permissions#

File paths are converted to absolute paths when the manager opens, using the process working directory. Later channel selection keeps that location even if the working directory changes. There is no source-tree search or executable-path fallback.

Neither mode syncs every record. Use stdout/stderr or direct OTLP in containers, or provide a writable volume for files. See daily retention for cleanup timing and failure behavior.

Understand slow or failed outputs#

Outputs run in selection order and apply their own minimum level:

slog.Logger does not return write errors to business code. These logs are not a transactional audit store. OTLP export failures and queue drops have separate asynchronous reporting described below.

For custom synchronous failure reporting, set logging.Config.OnError. It must be concurrency-safe and must not log back through these channels, which would recurse.

Stop producers before cleanup#

Standalone callers of logging.Open own the returned manager. Stop logging before calling Close or Shutdown(ctx); the latter propagates a cleanup deadline to OTLP flushing. The package neither changes slog.Default() nor closes stdout/stderr.

Set the application working directory#

Launch from the application root. The configured base path storage/logs/app.log then has a predictable location. With the current daily mode, its output includes the UTC date:

For local development, run commands from the repository root. Set your IDE's working directory to that same directory; for VS Code's Go launch configuration use "cwd": "${workspaceFolder}". Running from a nested directory deliberately uses that directory, not the repository root.

For an installed binary:

cd /opt/myapp
./bin/app

For systemd, set both paths explicitly:

[Service]
WorkingDirectory=/opt/myapp
ExecStart=/opt/myapp/bin/app

Docker keeps WORKDIR /app. Kubernetes sets workingDir: /app on both the api and migrate containers. File output requires writable storage at /app/storage/logs; stdout/stderr need no writable log directory. The development containers use a read-only root filesystem and mount storage at /app/storage. Tilt on Docker Desktop kubeadm mounts the local storage/logs directory directly at /app/storage/logs, using your host UID/GID. Writes appear in the same local file without copying. Compose uses a named volume, which is not the local checkout. The full development profile selects stderr,file,observability; the application default remains stderr. See Local development for storage details.

Create a standalone logger#

Outside the application lifecycle, construct one explicitly. This complete example writes a readable message without changing the global logger:

package main

import (
	"context"
	"fmt"
	"log/slog"
	"os"

	"github.com/wisalgroup/labena/pkg/logging"
)

func main() {
	logger, err := logging.New(os.Stderr, logging.Options{
		Format: logging.FormatTint,
		Level:  slog.LevelInfo,
	})
	if err != nil {
		fmt.Fprintln(os.Stderr, err)
		os.Exit(1)
	}
	ctx := logging.WithLogger(context.Background(), logger)
	logging.FromContext(ctx).InfoContext(ctx, "example started", "component", "docs")
}

The logger does not own its writer. If you supply an opened file, its creator must close it after logging has finished.

Panics and cancellation#

The API boundary records safe panic diagnostics and preserves transport aborts. Cancellation is not proof that an effect never happened. Follow the error-handling guarantees, and pass the same context through dependent work so logs retain correlation.

Connect logs to traces#

When an operation has a valid OpenTelemetry span, context-aware logging calls add trace_id, span_id, and trace_sampled automatically. This works with named channels, including channels selected later. Use InfoContext(ctx, ...) rather than Info(...); the context determines the current span. See observability for examples, sampling and grouped loggers.

Send logs to Loki#

The observability channel uses the otlp driver, not a file. Select it alongside readable console and daily file output:

LOG_CHANNELS=stderr,file,observability
OTEL_EXPORTER_OTLP_LOGS_ENDPOINT=http://localhost:4318/v1/logs

For standalone use, define an OTLP channel with the same public API:

channels["telemetry"] = logging.Channel{
    Driver: "otlp",
    Options: logging.Options{Level: slog.LevelInfo},
    OTLP: logging.OTLPOptions{
        Endpoint:    "http://localhost:4318/v1/logs",
        ServiceName: "orders",
        Environment: "development",
    },
}

OTLP preserves structured attributes, groups, severity and active trace context. It does not use tint or JSON text formatting; its Options.Level still filters records. It opens only when selected, does not replace global OTel providers, and needs no log directory. The app flushes it after other dependencies close.

The supplied Alloy configuration forwards OTLP logs to Loki. Request IDs and trace IDs are queryable metadata; the message remains the log body. Grafana uses the metadata for View trace and trace-to-log navigation. No file collector or observability.log is needed. Local readable files remain independent.

Exporter failures are reported through an independent stderr logger, rate-limited to once per minute with a cumulative failed-record count. The SDK also reports queue drops to stderr. These asynchronous failures do not go through OnError, which handles synchronous handler failures. Export has bounded memory and time budgets and is best-effort, not a durable delivery guarantee. See observability for limits and deployment settings.

Troubleshoot a missing or duplicated record#

File logging is independent of OTLP export. An exporter outage does not cause local files to be replayed later. See observability for request-ID filtering and troubleshooting for operation failures beyond logging.