My six questions about using OpenTelemetry metrics

[vasigorc]

Hello everyone 👋

I read the specification 🙂 I realize that Metrics component is currently being considered as alpha and maybe this could answer some of my questions, but I will still go away asking them since we (at my recent workplace) have decided to go forward with leveraging OpenTelemetry for all components of observability.

I would like to start by sharing a diagram of my understanding of how OpenTelemetry metrics should be set-up in a project. Please disregard the directions of the arrows or exact name of the fields - I didn't try to make it 💯 accurate. Besides that I've put sometimes names of implementations as opposed to interfaces just to give an idea of what I am currently looking to use. I welcome you to spot and refute any confusion from my part.

With that here are my questions:

1. For the Resource. I've read its SDK spec, but it's scope is still a little vague. Is it fair to assume that I can have a single resource per app and it to be simply a collection of attributes that is to be shared by all metrics?
2. Kind of related to my first point. Attrbutes on the level of LongCounter, LongGauge, LongHistogram etc. are they just diffs with the global Attributes owned by the Resource?
3. Based on the Metrics guide I get the expression that there is no need to keep any of different metrics (e.g. LongGauge) in memory. Will OpenTelemetry update an already existing metric just based on its attributes?
4. With AWS Cloudwatch one would use a PutMetricDataRequest to store the metrics. For Otel's LongCounter(for instance) the interface states that just calling add suffices to record the result - is this correct?
5. Why is OtlpGrpcMetricExporter not a part of opentelemetry-sdk-metrics dependency? That seems to be the only non third party option other than the NoOp.
6. Related to point number 5. Is it a fair assumption that using the DefaultMeterProvider is not an option since it uses the afore mentioned NoOp?

[jkwatson]

For questions 5 & 6, which are pretty clearly java-specific:
5) We have all of our OTLP exporters bundled together as a separate artifact, to separate export concerns from SDK concerns. We also have a prometheus exporter available, so OTLP is not the only exporter aside from no-op. See the opentelemetry-exporter-otlp artifact for the OTLP exporters, and the opentelemetry-exporter-prometheus artifact for the alpha prometheus exporter support.
6) See point 5. :)

[vasigorc]

Hello John, thank you for your answers!

Ref 5 - for some reason I thought OTLP, even though is an implementation, would be a good candidate for future default.

In what regards question 6. For tracing, we have GlobalOpenTelemetry.getTracer, which gives us a reasonable default. I was trying to confirm that there wasn't an equivalent for metrics.

Would you recommend PeriodicMetricReader to use with OtlpGrpcMetricExporter (being of the contributors to the latter)? I couldn't find another out-of-the box MetricReader in opentelemetry-sdk-metrics, opentelemetry-exporter-otlp, or opentelemetry-exporter-otlp-metrics.

Could you please also take a try on questions 3 and 4 assuming MeterProvider set-up as in the diagram? We do use Java SDK.

[jkwatson]

Yes, the PeriodicMetricReader is definitely what is intended for push-based metrics exporters, like OTLP.

We strongly recommend against using GlobalOpenTelemetry at all, unless you are using the javaagent, or have a very specific use-case that precludes using OpenTelemetry instances directly.

Metrics made it into the stable APIs at java release 1.10.x, so you probably need to update if you're looking for the metrics APIs on the OpenTelemetry class.

I know nothing of AWS CloudWatch integration, so I'll pass on trying to answer that specifically... but the metrics SDK is stateful, so it will handle the aggregation of metrics, yes.

re: question 3... I'm confused. LongGauge is an instrument, not a metric. And, you can only use it via a registered callback, so there's nothing to keep in memory, as far as I know.

[vasigorc]

I'm confused. LongGauge is an instrument, not a metric

If I understand this correctly this is pretty neat - on the application side we don't need to deal with metrics at all - only instruments?

+------------------+
| MeterProvider    |                 +-----------------+             +--------------+
|   Meter A        | Measurements... |                 | Metrics...  |              |
|     Instrument X +-----------------> In-memory state +-------------> MetricReader |
|     Instrument Y |                 |                 |             |              |
|   Meter B        |                 +-----------------+             +--------------+
|     Instrument Z |
|     ...          |                 +-----------------+             +--------------+
|     ...          | Measurements... |                 | Metrics...  |              |
|     ...          +-----------------> In-memory state +-------------> MetricReader |
|     ...          |                 |                 |             |              |
|     ...          |                 +-----------------+             +--------------+
+------------------+

I.e. once a measurement is taken control(that is aggregation, pushing to the collector) is released to the platform.

Special thanks for the heads-up to upgrade the APIs version!

[jkwatson]

yes, you got it! You only have to worry about metric instruments when writing instrumentation. The SDK will take care of the rest for you, including being 100% no-op if you don't configure an SDK at all for metrics.

[vasigorc]

Thanks again John - this was of big help!