Expose new transforms and formatters design

CHANGELOG.md

@@ -5,32 +5,37 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
-## [Unreleased]
+## [Unreleased] TBD
 
-## [1.1.4]
+### Added
+- Introduce new `formatter:` options: `:json` and `:passthrough`
+- Introduce new `transform:` options: `:cloud_watch` and `:passthrough`
+
+## [1.1.4] 2024-05-29
 
 ### Changed
-- Updated gems in the lockfile
+- Updated gems in the Gemfile.lock
 
-## [1.1.3]
+## [1.1.3] 2024-05-13
 
 ### Changed
-- Updated gems in the lockfile
+- Updated gems in the Gemfile.lock
 
 ### Added
 - Support for Ruby 3.2 and 3.3
 
 ### Dropped
 - Check for support for 'ubuntu-18.04'
 
-## [1.1.2]
+## [1.1.2] 2022-12-28
 
 - Add Puma 6 compatibility
-## [1.1.1]
+
+## [1.1.1] 2022-06-22
 
 Public release.
 
-## [1.1.0]
+## [1.1.0] 2022-06-22
 
 Out of beta testing, reading for usage. Following is a recap from Alpha & Beta releases.
 
@@ -42,8 +47,7 @@ Out of beta testing, reading for usage. Following is a recap from Alpha & Beta r
 - `config.socket_parser` option to allow custom parser implementation as needed
 - Datadog widgets examples under `docs/examples.md`
 
-## [1.1.0 Beta]
-
+## [1.1.0 Beta] ???
 ### Added
 
 Different ways to parse `Socket::Option`. Mainly due to the fact that `#inspect` can't
@@ -56,8 +60,7 @@ struct, so it should more or less stay stable.
 You can configure it by passing in `config.socket_parser = :inspect` or
 `config.socket_parser = ->(opt) { your implementation }`.
 
-## [1.1.0 Alpha]
-
+## [1.1.0 Alpha] ???
 ### Added
 
 Socket telemetry, and to be more precise new metric: `sockets.backlog`. If enabled it will
@@ -66,32 +69,32 @@ be acknowledged by Puma). It will be exposed under `sockets-backlog` metric.
 
 You can enable and test it via `config.sockets_telemetry!` option.
 
-## [1.0.0] - 2021-09-08
+## [1.0.0] 2021-09-08
 ### Added
-- Release to Github Packages
-- Explicitly flush datadog metrics after publishing them
+- Release to GitHub Packages
+- Explicitly flush Datadog metrics after publishing them
 - Middleware for measuring and tracking request queue time
 
 ### Changed
-- Replace `statsd.batch` with direct calls, as it aggregates metrics interally by default now.
+- Replace `statsd.batch` with direct calls, as it aggregates metrics internally by default now.
   Also `#batch` method is deprecated and will be removed in version 6 of Datadog Statsd client.
 
-## [0.3.1] - 2021-03-26
+## [0.3.1] 2021-03-26
 ### Changed
 - IO target replaces dots in telemetry keys with dashes for better integration with AWS CloudWatch
 
-## [0.3.0] - 2020-12-21
+## [0.3.0] 2020-12-21
 ### Added
 - Datadog Target integration tests
 
 ### Fixed
 - Datadog Target
 
-## [0.2.0] - 2020-12-21
+## [0.2.0] 2020-12-21
 ### Fixed
 - Removed debugging information
 
-## [0.1.0] - 2020-12-18
+## [0.1.0] 2020-12-18
 ### Added
 - Core Plugin
 - Telemetry generation

README.md

@@ -41,22 +41,35 @@ Puma::Plugin::Telemetry.configure do |config|
 end
 ```
 
-### Basic
+### Basic IO Target
 
-Output telemetry as JSON to `STDOUT`
+A basic I/O target will emit telemetry data to `STDOUT`, formatted in JSON.
 
 ```ruby
-  config.add_target :io
+config.add_target :io
 ```
 
+#### Options
+
+This target has configurable `formatter:` and `transform:` options.
+The `formatter:` options are
+
+* `:json` _(default)_ - Print the logs in JSON.
+* `:passthrough` - A passthrough formatter which returns the telemetry `Hash` unaltered, passing it directly to the `io:` instance.
+
+The `transform:` options are
+
+* `:cloud_watch` _(default)_ - Transforms telemetry keys, replacing dots with dashes to support AWS CloudWatch Log Metrics filters.
+* `:passthrough` -  A passthrough transform which returns the telemetry `Hash` unaltered.
+
 ### Datadog StatsD target
 
-Given gem provides built in target for Datadog StatsD client, that uses batch operation to publish metrics.
+A target for the Datadog StatsD client, that uses batch operation to publish metrics.
 
-**NOTE** Be sure to have `dogstatsd` gem installed.
+**NOTE** Be sure to have the `dogstatsd` gem installed.
 
 ```ruby
-  config.add_target :dogstatsd, client: Datadog::Statsd.new
+config.add_target :dogstatsd, client: Datadog::Statsd.new
 ```
 
 You can provide all the tags, namespaces, and other configuration options as always to `Datadog::Statsd.new` method.
@@ -73,7 +86,7 @@ Puma::Plugin::Telemetry.configure do |config|
   config.puma_telemetry = %w[workers.requests_count queue.backlog queue.capacity]
   config.socket_telemetry!
   config.socket_parser = :inspect
-  config.add_target :io, formatter: :json, io: StringIO.new
+  config.add_target :io, io: StringIO.new, formatter: :json, transform: :passthrough
   config.add_target :dogstatsd, client: Datadog::Statsd.new(tags: { env: ENV["RAILS_ENV"] })
 end
 ```
@@ -85,8 +98,8 @@ Target is a simple object that implements `call` methods that accepts `telemetry
 Just be mindful that if the API takes long to call, it will slow down frequency with which telemetry will get reported.
 
 ```ruby
-  # Example logfmt to stdout target
-  config.add_target proc { |telemetry| puts telemetry.map { |k, v| "#{k}=#{v.inspect}" }.join(" ") }
+  # Example key/value log to `STDOUT` target
+  config.add_target ->(telemetry) { puts telemetry.map { |k, v| "#{k}=#{v.inspect}" }.join(" ") }
 ```
 
 ## Extra middleware

lib/puma/plugin/telemetry/formatters/json_formatter.rb

@@ -0,0 +1,18 @@
+# frozen_string_literal: true
+
+require 'json'
+
+module Puma
+  class Plugin
+    module Telemetry
+      module Formatters
+        # JSON formatter, expects `call` method accepting telemetry hash
+        class JSONFormatter
+          def self.call(telemetry)
+            ::JSON.dump(telemetry)
+          end
+        end
+      end
+    end
+  end
+end

lib/puma/plugin/telemetry/formatters/passthrough_formatter.rb

@@ -0,0 +1,16 @@
+# frozen_string_literal: true
+
+module Puma
+  class Plugin
+    module Telemetry
+      module Formatters
+        # A passthrough formatter - it returns the telemetry Hash it was given
+        class PassthroughFormatter
+          def self.call(telemetry)
+            telemetry
+          end
+        end
+      end
+    end
+  end
+end

lib/puma/plugin/telemetry/targets/base_formatting_target.rb

@@ -0,0 +1,38 @@
+# frozen_string_literal: true
+
+require_relative '../formatters/json_formatter'
+require_relative '../formatters/passthrough_formatter'
+require_relative '../transforms/cloud_watch_transform'
+require_relative '../transforms/passthrough_transform'
+
+module Puma
+  class Plugin
+    module Telemetry
+      module Targets
+        # A base class for other Targets concerned with formatting telemetry
+        class BaseFormattingTarget
+          def initialize(formatter: :json, transform: :cloud_watch)
+            @transform = case transform
+                         when :cloud_watch then Transforms::CloudWatchTranform
+                         when :passthrough then Transforms::PassthroughTransform
+                         else transform
+                         end
+            @formatter = case formatter
+                         when :json then Formatters::JSONFormatter
+                         when :passthrough then Formatters::PassthroughFormatter
+                         else formatter
+                         end
+          end
+
+          def call(_telemetry)
+            raise "#{__method__} must be implemented by #{self.class.name}"
+          end
+
+          private
+
+          attr_reader :formatter, :transform
+        end
+      end
+    end
+  end
+end

lib/puma/plugin/telemetry/targets/io_target.rb

@@ -1,40 +1,25 @@
 # frozen_string_literal: true
 
-require 'json'
+require_relative 'base_formatting_target'
 
 module Puma
   class Plugin
     module Telemetry
       module Targets
         # Simple IO Target, publishing metrics to STDOUT or logs
-        #
-        class IOTarget
-          # JSON formatter for IO, expects `call` method accepting telemetry hash
-          #
-          class JSONFormatter
-            # NOTE: Replace dots with dashes for better support of AWS CloudWatch
-            #       Log Metric filters, as they don't support dots in key names.
-            def self.call(telemetry)
-              log = telemetry.transform_keys { |k| k.tr('.', '-') }
-
-              log['name'] = 'Puma::Plugin::Telemetry'
-              log['message'] = 'Publish telemetry'
-
-              ::JSON.dump(log)
-            end
-          end
-
-          def initialize(io: $stdout, formatter: :json)
+        class IOTarget < BaseFormattingTarget
+          def initialize(io: $stdout, formatter: :json, transform: :cloud_watch)
+            super(formatter: formatter, transform: transform)
             @io = io
-            @formatter = case formatter
-                         when :json then JSONFormatter
-                         else formatter
-                         end
           end
 
           def call(telemetry)
-            @io.puts(@formatter.call(telemetry))
+            io.puts(formatter.call(transform.call(telemetry)))
           end
+
+          private
+
+          attr_reader :io
         end
       end
     end

lib/puma/plugin/telemetry/transforms/cloud_watch_transform.rb

@@ -0,0 +1,23 @@
+# frozen_string_literal: true
+
+require 'json'
+
+module Puma
+  class Plugin
+    module Telemetry
+      module Transforms
+        # Replace dots with dashes for better support of AWS CloudWatch Log
+        # Metric filters, as they don't support dots in key names.
+        # Expects `call` method accepting telemetry Hash
+        class CloudWatchTranform
+          def self.call(telemetry)
+            telemetry.transform_keys { |k| String(k).tr('.', '-') }.tap do |data|
+              data['name'] = 'Puma::Plugin::Telemetry'
+              data['message'] = 'Publish telemetry'
+            end
+          end
+        end
+      end
+    end
+  end
+end

lib/puma/plugin/telemetry/transforms/passthrough_transform.rb

@@ -0,0 +1,16 @@
+# frozen_string_literal: true
+
+module Puma
+  class Plugin
+    module Telemetry
+      module Transforms
+        # A passthrough transform - it returns the telemetry Hash it was given
+        class PassthroughTransform
+          def self.call(telemetry)
+            telemetry
+          end
+        end
+      end
+    end
+  end
+end

spec/fixtures/sockets.rb

@@ -21,7 +21,7 @@
 
 Puma::Plugin::Telemetry.configure do |config|
   # Simple `key=value` formatter
-  config.add_target :io, formatter: ->(t) { t.map { |r| r.join('=') }.join(' ') }
+  config.add_target(:io, formatter: ->(t) { t.map { |r| r.join('=') }.join(' ') }, transform: :passthrough)
   config.frequency = 1
   config.enabled = true
 

spec/puma/plugin/telemetry/formatters/json_formatter_spec.rb

@@ -0,0 +1,27 @@
+# frozen_string_literal: true
+
+module Puma
+  class Plugin
+    module Telemetry
+      module Formatters
+        RSpec.describe JSONFormatter do
+          subject(:formatter) { described_class }
+
+          it 'formats the telemetry as a JSON string' do
+            string = formatter.call('foo' => 'bar')
+
+            data = ::JSON.parse(string)
+            expect(data.fetch('foo')).to eq('bar')
+          end
+
+          it 'handles symbol keys' do
+            string = formatter.call(foo: 'bar')
+
+            data = ::JSON.parse(string)
+            expect(data.fetch('foo')).to eq('bar')
+          end
+        end
+      end
+    end
+  end
+end

spec/puma/plugin/telemetry/formatters/passthrough_formatter_spec.rb

@@ -0,0 +1,20 @@
+# frozen_string_literal: true
+
+module Puma
+  class Plugin
+    module Telemetry
+      module Formatters
+        RSpec.describe PassthroughFormatter do
+          subject(:formatter) { described_class }
+
+          it 'returns the telemetry, unalterted' do
+            telmetry_data = { 'foo' => 'bar' }
+            formatted_data = formatter.call(telmetry_data)
+
+            expect(formatted_data).to eq(telmetry_data)
+          end
+        end
+      end
+    end
+  end
+end