Collecting metrics

Introduction

There are two types of metrics you can collect:

  1. In-app metrics using Uptrace client (OpenTelemetry distribution for Uptrace), for example, Go HTTP server metrics or user-defined metrics.
  2. Infrastructure metrics using OpenTelemetry Collector, for example, Linux/Windows system metrics or PostgreSQL metrics.

To start collecting in-app metrics, see Getting started guide on how to install Uptrace client. In this guide we are going to focus on infrastructure metrics and OpenTelemetry Collector.

OpenTelemetry Collector

To start monitoring your infrastructure, you need to install OpenTelemetry Collectoropen in new window on each host that you want to monitor. Collector acts as an agent that pulls metrics from monitored systems and sends them to Uptrace using the OTLP exporter.

Installation

OpenTelemetry Collector distributes pre-compiled binariesopen in new window for Linux, MacOS, and Windows.

Linux

To install otel-contrib-collector binary with the associated systemd service:

wget https://github.com/open-telemetry/opentelemetry-collector-contrib/releases/download/v0.34.0/otel-contrib-collector_0.34.0_amd64.deb
sudo dpkg -i otel-contrib-collector_0.34.0_amd64.deb
wget https://github.com/open-telemetry/opentelemetry-collector-contrib/releases/download/v0.34.0/otel-contrib-collector-0.34.0-1.x86_64.rpm
sudo rpm -ivh otel-contrib-collector-0.34.0-1.x86_64.rpm

You can check the status of the installed service with:

sudo systemctl status otel-contrib-collector

Configuration sections

By default, you can find the config file at /etc/otel-contrib-collector/config.yaml. It has the following sections:

  • receivers configures how data gets into the Collector.
  • processors specifies what happens with the received data.
  • exporters configures how you send processed data to one or more backends.
  • service pulls the configured receivers, processors, and exporters together into processing pipelines. Don't repeat a common mistake by configuring a receiver or an exporter without adding it to a processing pipeline.

You can learn more about Collector configuration at the official documentationopen in new window.

Configuring Collector for Uptrace

Replace the contents of /etc/otel-contrib-collector/config.yaml with the following config that sends host metrics to Uptrace:

receivers:
  otlp:
    protocols:
      grpc:
      http:
  hostmetrics:
    collection_interval: 10s
    scrapers:
      cpu:
      disk:
      load:
      filesystem:
      memory:
      network:
      paging:

exporters:
  otlp:
    endpoint: otlp.uptrace.dev:4317
    headers: { 'uptrace-dsn': '<dsn>' }

processors:
  resourcedetection:
    detectors: [system]
  batch:
    timeout: 10s

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [otlp]
    metrics:
      receivers: [otlp, hostmetrics]
      processors: [batch, resourcedetection]
      exporters: [otlp]

Don't forget to restart the service:

sudo systemctl restart otel-contrib-collector

Checking OpenTelemetry Collector logs can also be a good idea:

sudo journalctl -u otel-contrib-collector -f

What's next

Next, you can use Collector configuratoropen in new window to tweak your config and create custom instruments using OpenTelemetry Metrics API.