Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Custom OTLP File Exporter + opentelemetry updates #909

Open
wants to merge 12 commits into
base: main
Choose a base branch
from

Conversation

joshwlewis
Copy link
Member

@joshwlewis joshwlewis commented Feb 6, 2025

As mentioned in #907, there was an upstream change to opentelemetry-stdout's exporter: open-telemetry/opentelemetry-rust#2040. With that change, the exporter lost the ability to send telemetry to a generic writer (like a buffer or file), and also lost jsonl serialization. This means we can't export in the correct format with newer versions of the opentelemetry libraries without a new exporter. This PR adds a custom OTLP File Exporter (spec is here) for traces and updates the opentelemetry libraries to the latest.

Based on this comment, and my upstream issue, there is interest in an OTLP File Exporter upstream, so this custom exporter shouldn't live here forever.

This exporter probably isn't quite good enough to upstream yet -- it supports traces, but not metrics or logs.

Bonus changes:

  • Now uses batch export functionality, which is generally preferred according to the spec. In opentelemetry-sdk 0.28, the batch export functionality no longer requires an async runtime (like tokio) and instead runs in a separate background thread (some details here), which lowers the dependency overhead for us.
  • Slightly better feature usage -- we're no longer compiling the metrics and logs features from opentelemetry, which we don't use anyway
  • This uses a LineWriter rather than a BufWriter now. The former is better for our use case -- we want to write complete lines to the file to prevent writing partial json to the file and corrupting it.
  • There were a truckload of random API changes in opentelemetry. Some things were moved, others renamed. Almost everything uses the builder pattern now.

libcnb/src/tracing.rs Outdated Show resolved Hide resolved
@joshwlewis joshwlewis marked this pull request as ready for review February 13, 2025 00:41
@joshwlewis joshwlewis requested a review from a team as a code owner February 13, 2025 00:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants