Logging

Structured logging

Structured logging is a practive of using a machine-readable format to write log messages. Uptrace automatically parses such messages and saves the extracted data as attributes.

logfmt

In logfmt a log message consists of key/value pairs separated with a space. Therefore you must quote values containing spaces.

request failed http.method=GET http.route=/users/:id enduser.id=123 foo="hello world"

JSON format

You can also use JSON to include structured data in your log messages:

request failed {"http.method": "GET", "http.route": "/users/:id", "enduser.id": 123, "foo": "hello world"}

Application logs

To record application logs, use span events. You must set the event name to log and use semantic attributes to record the context:

  • log.severity to record the log severity. Must be one of TRACE, DEBUG, INFO, WARN, ERROR, FATAL, and PANIC.
  • log.message to record the message.
  • code.function to record the function name.
  • code.filepath to record the file path.
  • code.lineno to record the line number.

For example:

span := trace.SpanFromContext(ctx)

span.AddEvent(ctx, "log",
    // Log severity and message.
    label.String("log.severity", "ERROR"),
    label.String("log.message", "request failed"),

    // Optional.
    label.String("code.function", "org.FetchUser"),
    label.String("code.filepath", "org/user.go"),
    label.Int("code.lineno", 123),

    // Additional details.
    label.String("foo", "hello world"),
)

Uptrace also provides instrumentations for popular logging libraries which allow recording logs using more conventional API. For example, with logrusopen in new window integration you can rewrite the code above as:

logrus.WithContext(ctx).
    WithField("foo", "hello world").
    Error("request failed")

Third-party logs

When you can't change your app to record logs as events or you want to collect existing logs (syslog, nginx log), use Vectoropen in new window. Vector collects, transforms, and routes your logs to multiple destinations including Uptrace. It is blazingly fast and memory efficient.

Vector

To configure Vector to send logs to Uptrace, use the HTTP sink and your project DSN (https://<token>@api.uptrace.dev/<project_id>). For example, to collect syslog messages you can create the following Vector config:

[sources.in]
  type = "file"
  include = ["/var/log/syslog"]

[sinks.out]
  type = "http"
  inputs = ["in"]
  encoding.codec = "ndjson"
  compression = "gzip"
  uri = "http://api.uptrace.dev/api/v1/vector-logs"
  headers.uptrace-dsn = "https://<token>@api.uptrace.dev/<project_id>"

Copy the config above to vector.toml and start Vector:

vector --config=vector.toml

Propagating trace context

When using third-party logs, trace context is not automatically propagated and logs are not linked with any spans. To propagate context and associate a log message with a span, use the following attribute keys:

  • trace_id for TraceId, hex-encoded.
  • span_id for SpanId, hex-encoded.
  • trace_flags for trace flags, formatted according to W3C traceflags format.

For example:

request failed trace_id=958180131ddde684c1dbda1aeacf51d3 span_id=0cf859e4f7510204