diff --git a/CHANGELOG.md b/CHANGELOG.md index df8598e..1937db9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,17 +5,21 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). -## [Unreleased] +## [Unreleased] TBD -## [1.1.4] +### Added +- Introduce new `formatter:` options: `:json` and `:passthrough` +- Introduce new `transform:` options: `:cloud_watch` and `:passthrough` + +## [1.1.4] 2024-05-29 ### Changed -- Updated gems in the lockfile +- Updated gems in the Gemfile.lock -## [1.1.3] +## [1.1.3] 2024-05-13 ### Changed -- Updated gems in the lockfile +- Updated gems in the Gemfile.lock ### Added - Support for Ruby 3.2 and 3.3 @@ -23,14 +27,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Dropped - Check for support for 'ubuntu-18.04' -## [1.1.2] +## [1.1.2] 2022-12-28 - Add Puma 6 compatibility -## [1.1.1] + +## [1.1.1] 2022-06-22 Public release. -## [1.1.0] +## [1.1.0] 2022-06-22 Out of beta testing, reading for usage. Following is a recap from Alpha & Beta releases. @@ -42,8 +47,7 @@ Out of beta testing, reading for usage. Following is a recap from Alpha & Beta r - `config.socket_parser` option to allow custom parser implementation as needed - Datadog widgets examples under `docs/examples.md` -## [1.1.0 Beta] - +## [1.1.0 Beta] ??? ### Added Different ways to parse `Socket::Option`. Mainly due to the fact that `#inspect` can't @@ -56,8 +60,7 @@ struct, so it should more or less stay stable. You can configure it by passing in `config.socket_parser = :inspect` or `config.socket_parser = ->(opt) { your implementation }`. -## [1.1.0 Alpha] - +## [1.1.0 Alpha] ??? ### Added Socket telemetry, and to be more precise new metric: `sockets.backlog`. If enabled it will @@ -66,32 +69,32 @@ be acknowledged by Puma). It will be exposed under `sockets-backlog` metric. You can enable and test it via `config.sockets_telemetry!` option. -## [1.0.0] - 2021-09-08 +## [1.0.0] 2021-09-08 ### Added -- Release to Github Packages -- Explicitly flush datadog metrics after publishing them +- Release to GitHub Packages +- Explicitly flush Datadog metrics after publishing them - Middleware for measuring and tracking request queue time ### Changed -- Replace `statsd.batch` with direct calls, as it aggregates metrics interally by default now. +- Replace `statsd.batch` with direct calls, as it aggregates metrics internally by default now. Also `#batch` method is deprecated and will be removed in version 6 of Datadog Statsd client. -## [0.3.1] - 2021-03-26 +## [0.3.1] 2021-03-26 ### Changed - IO target replaces dots in telemetry keys with dashes for better integration with AWS CloudWatch -## [0.3.0] - 2020-12-21 +## [0.3.0] 2020-12-21 ### Added - Datadog Target integration tests ### Fixed - Datadog Target -## [0.2.0] - 2020-12-21 +## [0.2.0] 2020-12-21 ### Fixed - Removed debugging information -## [0.1.0] - 2020-12-18 +## [0.1.0] 2020-12-18 ### Added - Core Plugin - Telemetry generation diff --git a/README.md b/README.md index d0b6f66..5180c69 100644 --- a/README.md +++ b/README.md @@ -41,22 +41,35 @@ Puma::Plugin::Telemetry.configure do |config| end ``` -### Basic +### Basic IO Target -Output telemetry as JSON to `STDOUT` +A basic I/O target will emit telemetry data to `STDOUT`, formatted in JSON. ```ruby - config.add_target :io +config.add_target :io ``` +#### Options + +This target has configurable `formatter:` and `transform:` options. +The `formatter:` options are + +* `:json` _(default)_ - Print the logs in JSON. +* `:passthrough` - A passthrough formatter which returns the telemetry `Hash` unaltered, passing it directly to the `io:` instance. + +The `transform:` options are + +* `:cloud_watch` _(default)_ - Transforms telemetry keys, replacing dots with dashes to support AWS CloudWatch Log Metrics filters. +* `:passthrough` - A passthrough transform which returns the telemetry `Hash` unaltered. + ### Datadog StatsD target -Given gem provides built in target for Datadog StatsD client, that uses batch operation to publish metrics. +A target for the Datadog StatsD client, that uses batch operation to publish metrics. -**NOTE** Be sure to have `dogstatsd` gem installed. +**NOTE** Be sure to have the `dogstatsd` gem installed. ```ruby - config.add_target :dogstatsd, client: Datadog::Statsd.new +config.add_target :dogstatsd, client: Datadog::Statsd.new ``` You can provide all the tags, namespaces, and other configuration options as always to `Datadog::Statsd.new` method. @@ -73,7 +86,7 @@ Puma::Plugin::Telemetry.configure do |config| config.puma_telemetry = %w[workers.requests_count queue.backlog queue.capacity] config.socket_telemetry! config.socket_parser = :inspect - config.add_target :io, formatter: :json, io: StringIO.new + config.add_target :io, io: StringIO.new, formatter: :json, transform: :passthrough config.add_target :dogstatsd, client: Datadog::Statsd.new(tags: { env: ENV["RAILS_ENV"] }) end ``` @@ -85,8 +98,8 @@ Target is a simple object that implements `call` methods that accepts `telemetry Just be mindful that if the API takes long to call, it will slow down frequency with which telemetry will get reported. ```ruby - # Example logfmt to stdout target - config.add_target proc { |telemetry| puts telemetry.map { |k, v| "#{k}=#{v.inspect}" }.join(" ") } + # Example key/value log to `STDOUT` target + config.add_target ->(telemetry) { puts telemetry.map { |k, v| "#{k}=#{v.inspect}" }.join(" ") } ``` ## Extra middleware diff --git a/lib/puma/plugin/telemetry/formatters/json_formatter.rb b/lib/puma/plugin/telemetry/formatters/json_formatter.rb new file mode 100644 index 0000000..2a44e88 --- /dev/null +++ b/lib/puma/plugin/telemetry/formatters/json_formatter.rb @@ -0,0 +1,18 @@ +# frozen_string_literal: true + +require 'json' + +module Puma + class Plugin + module Telemetry + module Formatters + # JSON formatter, expects `call` method accepting telemetry hash + class JSONFormatter + def self.call(telemetry) + ::JSON.dump(telemetry) + end + end + end + end + end +end diff --git a/lib/puma/plugin/telemetry/formatters/passthrough_formatter.rb b/lib/puma/plugin/telemetry/formatters/passthrough_formatter.rb new file mode 100644 index 0000000..229a707 --- /dev/null +++ b/lib/puma/plugin/telemetry/formatters/passthrough_formatter.rb @@ -0,0 +1,16 @@ +# frozen_string_literal: true + +module Puma + class Plugin + module Telemetry + module Formatters + # A passthrough formatter - it returns the telemetry Hash it was given + class PassthroughFormatter + def self.call(telemetry) + telemetry + end + end + end + end + end +end diff --git a/lib/puma/plugin/telemetry/targets/base_formatting_target.rb b/lib/puma/plugin/telemetry/targets/base_formatting_target.rb new file mode 100644 index 0000000..43946c5 --- /dev/null +++ b/lib/puma/plugin/telemetry/targets/base_formatting_target.rb @@ -0,0 +1,38 @@ +# frozen_string_literal: true + +require_relative '../formatters/json_formatter' +require_relative '../formatters/passthrough_formatter' +require_relative '../transforms/cloud_watch_transform' +require_relative '../transforms/passthrough_transform' + +module Puma + class Plugin + module Telemetry + module Targets + # A base class for other Targets concerned with formatting telemetry + class BaseFormattingTarget + def initialize(formatter: :json, transform: :cloud_watch) + @transform = case transform + when :cloud_watch then Transforms::CloudWatchTranform + when :passthrough then Transforms::PassthroughTransform + else transform + end + @formatter = case formatter + when :json then Formatters::JSONFormatter + when :passthrough then Formatters::PassthroughFormatter + else formatter + end + end + + def call(_telemetry) + raise "#{__method__} must be implemented by #{self.class.name}" + end + + private + + attr_reader :formatter, :transform + end + end + end + end +end diff --git a/lib/puma/plugin/telemetry/targets/io_target.rb b/lib/puma/plugin/telemetry/targets/io_target.rb index 3272393..595cf34 100644 --- a/lib/puma/plugin/telemetry/targets/io_target.rb +++ b/lib/puma/plugin/telemetry/targets/io_target.rb @@ -1,40 +1,25 @@ # frozen_string_literal: true -require 'json' +require_relative 'base_formatting_target' module Puma class Plugin module Telemetry module Targets # Simple IO Target, publishing metrics to STDOUT or logs - # - class IOTarget - # JSON formatter for IO, expects `call` method accepting telemetry hash - # - class JSONFormatter - # NOTE: Replace dots with dashes for better support of AWS CloudWatch - # Log Metric filters, as they don't support dots in key names. - def self.call(telemetry) - log = telemetry.transform_keys { |k| k.tr('.', '-') } - - log['name'] = 'Puma::Plugin::Telemetry' - log['message'] = 'Publish telemetry' - - ::JSON.dump(log) - end - end - - def initialize(io: $stdout, formatter: :json) + class IOTarget < BaseFormattingTarget + def initialize(io: $stdout, formatter: :json, transform: :cloud_watch) + super(formatter: formatter, transform: transform) @io = io - @formatter = case formatter - when :json then JSONFormatter - else formatter - end end def call(telemetry) - @io.puts(@formatter.call(telemetry)) + io.puts(formatter.call(transform.call(telemetry))) end + + private + + attr_reader :io end end end diff --git a/lib/puma/plugin/telemetry/transforms/cloud_watch_transform.rb b/lib/puma/plugin/telemetry/transforms/cloud_watch_transform.rb new file mode 100644 index 0000000..0c44788 --- /dev/null +++ b/lib/puma/plugin/telemetry/transforms/cloud_watch_transform.rb @@ -0,0 +1,23 @@ +# frozen_string_literal: true + +require 'json' + +module Puma + class Plugin + module Telemetry + module Transforms + # Replace dots with dashes for better support of AWS CloudWatch Log + # Metric filters, as they don't support dots in key names. + # Expects `call` method accepting telemetry Hash + class CloudWatchTranform + def self.call(telemetry) + telemetry.transform_keys { |k| String(k).tr('.', '-') }.tap do |data| + data['name'] = 'Puma::Plugin::Telemetry' + data['message'] = 'Publish telemetry' + end + end + end + end + end + end +end diff --git a/lib/puma/plugin/telemetry/transforms/passthrough_transform.rb b/lib/puma/plugin/telemetry/transforms/passthrough_transform.rb new file mode 100644 index 0000000..bce9bb9 --- /dev/null +++ b/lib/puma/plugin/telemetry/transforms/passthrough_transform.rb @@ -0,0 +1,16 @@ +# frozen_string_literal: true + +module Puma + class Plugin + module Telemetry + module Transforms + # A passthrough transform - it returns the telemetry Hash it was given + class PassthroughTransform + def self.call(telemetry) + telemetry + end + end + end + end + end +end diff --git a/spec/fixtures/sockets.rb b/spec/fixtures/sockets.rb index b6cfbf0..a9a6b3e 100644 --- a/spec/fixtures/sockets.rb +++ b/spec/fixtures/sockets.rb @@ -21,7 +21,7 @@ Puma::Plugin::Telemetry.configure do |config| # Simple `key=value` formatter - config.add_target :io, formatter: ->(t) { t.map { |r| r.join('=') }.join(' ') } + config.add_target(:io, formatter: ->(t) { t.map { |r| r.join('=') }.join(' ') }, transform: :passthrough) config.frequency = 1 config.enabled = true diff --git a/spec/puma/plugin/telemetry/formatters/json_formatter_spec.rb b/spec/puma/plugin/telemetry/formatters/json_formatter_spec.rb new file mode 100644 index 0000000..34ef629 --- /dev/null +++ b/spec/puma/plugin/telemetry/formatters/json_formatter_spec.rb @@ -0,0 +1,27 @@ +# frozen_string_literal: true + +module Puma + class Plugin + module Telemetry + module Formatters + RSpec.describe JSONFormatter do + subject(:formatter) { described_class } + + it 'formats the telemetry as a JSON string' do + string = formatter.call('foo' => 'bar') + + data = ::JSON.parse(string) + expect(data.fetch('foo')).to eq('bar') + end + + it 'handles symbol keys' do + string = formatter.call(foo: 'bar') + + data = ::JSON.parse(string) + expect(data.fetch('foo')).to eq('bar') + end + end + end + end + end +end diff --git a/spec/puma/plugin/telemetry/formatters/passthrough_formatter_spec.rb b/spec/puma/plugin/telemetry/formatters/passthrough_formatter_spec.rb new file mode 100644 index 0000000..5eff53f --- /dev/null +++ b/spec/puma/plugin/telemetry/formatters/passthrough_formatter_spec.rb @@ -0,0 +1,20 @@ +# frozen_string_literal: true + +module Puma + class Plugin + module Telemetry + module Formatters + RSpec.describe PassthroughFormatter do + subject(:formatter) { described_class } + + it 'returns the telemetry, unalterted' do + telmetry_data = { 'foo' => 'bar' } + formatted_data = formatter.call(telmetry_data) + + expect(formatted_data).to eq(telmetry_data) + end + end + end + end + end +end diff --git a/spec/puma/plugin/telemetry/targets/io_target_spec.rb b/spec/puma/plugin/telemetry/targets/io_target_spec.rb new file mode 100644 index 0000000..8f21abd --- /dev/null +++ b/spec/puma/plugin/telemetry/targets/io_target_spec.rb @@ -0,0 +1,22 @@ +# frozen_string_literal: true + +module Puma + class Plugin + module Telemetry + module Targets + RSpec.describe IOTarget do + subject(:target) { described_class.new(io: io, formatter: logfmt) } + let(:io) { StringIO.new } + let(:telemetry) { { foo: 'bar' } } + let(:logfmt) { ->(telemetry) { telemetry.map { |k, v| "#{k}=#{v}" }.join(' ') } } + + it 'puts to the io object' do + target.call(telemetry) + + expect(io.string).to include('foo=bar') + end + end + end + end + end +end diff --git a/spec/puma/plugin/telemetry/transforms/cloud_watch_transform_spec.rb b/spec/puma/plugin/telemetry/transforms/cloud_watch_transform_spec.rb new file mode 100644 index 0000000..a102e0f --- /dev/null +++ b/spec/puma/plugin/telemetry/transforms/cloud_watch_transform_spec.rb @@ -0,0 +1,25 @@ +# frozen_string_literal: true + +module Puma + class Plugin + module Telemetry + module Transforms + RSpec.describe CloudWatchTranform do + subject(:transform) { described_class } + + it 'replaces dots with dashes in keys' do + data = transform.call('the.foo' => 'the.bar') + + expect(data.fetch('the-foo')).to eq('the.bar') + end + + it 'handles symbol keys' do + data = transform.call(foo: 'bar') + + expect(data.fetch('foo')).to eq('bar') + end + end + end + end + end +end diff --git a/spec/puma/plugin/telemetry/transforms/passthrough_transform_spec.rb b/spec/puma/plugin/telemetry/transforms/passthrough_transform_spec.rb new file mode 100644 index 0000000..364f3a3 --- /dev/null +++ b/spec/puma/plugin/telemetry/transforms/passthrough_transform_spec.rb @@ -0,0 +1,20 @@ +# frozen_string_literal: true + +module Puma + class Plugin + module Telemetry + module Transforms + RSpec.describe PassthroughTransform do + subject(:transform) { described_class } + + it 'returns the telemetry, unalterted' do + telmetry_data = { 'foo' => 'bar' } + transformed_data = transform.call(telmetry_data) + + expect(transformed_data).to eq(telmetry_data) + end + end + end + end + end +end