Add log target and l2met transform

CHANGELOG.md

@@ -5,14 +5,19 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
-## [Unreleased]
+## [Unreleased] TBD
 
-## [1.1.4]
+### Added
+- Add new `LogTarget`
+- Introduce new `formatter:` options: `:json`, `:logfmt`, and `:noop`
+- Introduce new `transform:` options: `:cloud_watch`, `:l2met`, and `:noop`
+
+## [1.1.4] 2024-05-29
 
 ### Changed
 - Updated gems in the lockfile
 
-## [1.1.3]
+## [1.1.3] 2024-05-13
 
 ### Changed
 - Updated gems in the lockfile
@@ -23,14 +28,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ### Dropped
 - Check for support for 'ubuntu-18.04'
 
-## [1.1.2]
+## [1.1.2] 2022-12-28
 
 - Add Puma 6 compatibility
-## [1.1.1]
+
+## [1.1.1] 2022-06-22
 
 Public release.
 
-## [1.1.0]
+## [1.1.0] 2022-06-22
 
 Out of beta testing, reading for usage. Following is a recap from Alpha & Beta releases.
 
@@ -42,8 +48,7 @@ Out of beta testing, reading for usage. Following is a recap from Alpha & Beta r
 - `config.socket_parser` option to allow custom parser implementation as needed
 - Datadog widgets examples under `docs/examples.md`
 
-## [1.1.0 Beta]
-
+## [1.1.0 Beta] ???
 ### Added
 
 Different ways to parse `Socket::Option`. Mainly due to the fact that `#inspect` can't
@@ -56,8 +61,7 @@ struct, so it should more or less stay stable.
 You can configure it by passing in `config.socket_parser = :inspect` or
 `config.socket_parser = ->(opt) { your implementation }`.
 
-## [1.1.0 Alpha]
-
+## [1.1.0 Alpha] ???
 ### Added
 
 Socket telemetry, and to be more precise new metric: `sockets.backlog`. If enabled it will
@@ -66,32 +70,32 @@ be acknowledged by Puma). It will be exposed under `sockets-backlog` metric.
 
 You can enable and test it via `config.sockets_telemetry!` option.
 
-## [1.0.0] - 2021-09-08
+## [1.0.0] 2021-09-08
 ### Added
-- Release to Github Packages
-- Explicitly flush datadog metrics after publishing them
+- Release to GitHub Packages
+- Explicitly flush Datadog metrics after publishing them
 - Middleware for measuring and tracking request queue time
 
 ### Changed
-- Replace `statsd.batch` with direct calls, as it aggregates metrics interally by default now.
+- Replace `statsd.batch` with direct calls, as it aggregates metrics internally by default now.
   Also `#batch` method is deprecated and will be removed in version 6 of Datadog Statsd client.
 
-## [0.3.1] - 2021-03-26
+## [0.3.1] 2021-03-26
 ### Changed
 - IO target replaces dots in telemetry keys with dashes for better integration with AWS CloudWatch
 
-## [0.3.0] - 2020-12-21
+## [0.3.0] 2020-12-21
 ### Added
 - Datadog Target integration tests
 
 ### Fixed
 - Datadog Target
 
-## [0.2.0] - 2020-12-21
+## [0.2.0] 2020-12-21
 ### Fixed
 - Removed debugging information
 
-## [0.1.0] - 2020-12-18
+## [0.1.0] 2020-12-18
 ### Added
 - Core Plugin
 - Telemetry generation

README.md

@@ -41,22 +41,61 @@ Puma::Plugin::Telemetry.configure do |config|
 end
 ```
 
-### Basic
+### Basic IO Target
 
-Output telemetry as JSON to `STDOUT`
+A basic I/O target will emit telemetry data to `STDOUT`, formatted in JSON.
 
 ```ruby
-  config.add_target :io
+config.add_target :io
 ```
 
+#### Options
+
+This target has configurable `formatter:` and `transform:` options.
+The `formatter:` options are
+
+* `:json` _(default)_ - Print the logs in JSON.
+* `:logfmt` - Print the logs in key/value pairs, as per `logfmt`.
+* `:noop` - A NOOP formatter which returns the telemetry `Hash` unaltered, passing it directly to the `io:` instance.
+
+The `transform:` options are
+
+* `:cloud_watch` _(default)_ - Transforms telemetry keys, replacing dots with dashes to support AWS CloudWatch Log Metrics filters.
+* `:logfmt` - Transforms telemetry keys, prepending `sample#` for [L2Met][l2met] consumption.
+* `:noop` -  A NOOP transform which returns the telemetry `Hash` unaltered.
+
+### Log target
+
+Emitting to `STDOUT` via the basic `IOTarget` can work for getting telemetry into logs, we also provide an explicit `LogTarget`.
+This target will defaults to emitting telemetry at the `INFO` log level via a [standard library `::Logger`][logger] instance.
+That default logger will print to `STDOUT` in [the `logfmt` format][logfmt].
+
+```ruby
+config.add_target :log
+```
+
+You can pass an explicit `logger:` option if you wanted to, for example, use the same logger as Rails.
+
+```ruby
+config.add_target :log, logger: Rails.logger
+```
+
+This target also has configurable `formatter:` and `transform:` options.
+The [possible options are the same as for the `IOTarget`](#options), but the defaults are different.
+The `LogTarget` defaults to `formatter: :logfmt`, and `transform: :noop`
+
+[l2met]: https://github.com/ryandotsmith/l2met?tab=readme-ov-file#l2met "l2met - Logs to Metrics"
+[logfmt]: https://brandur.org/logfmt "logfmt - Structured log format"
+[logger]: https://rubyapi.org/o/logger "Ruby's Logger, from the stdlib"
+
 ### Datadog StatsD target
 
-Given gem provides built in target for Datadog StatsD client, that uses batch operation to publish metrics.
+A target for the Datadog StatsD client, that uses batch operation to publish metrics.
 
-**NOTE** Be sure to have `dogstatsd` gem installed.
+**NOTE** Be sure to have the `dogstatsd` gem installed.
 
 ```ruby
-  config.add_target :dogstatsd, client: Datadog::Statsd.new
+config.add_target :dogstatsd, client: Datadog::Statsd.new
 ```
 
 You can provide all the tags, namespaces, and other configuration options as always to `Datadog::Statsd.new` method.
@@ -75,6 +114,7 @@ Puma::Plugin::Telemetry.configure do |config|
   config.socket_parser = :inspect
   config.add_target :io, formatter: :json, io: StringIO.new
   config.add_target :dogstatsd, client: Datadog::Statsd.new(tags: { env: ENV["RAILS_ENV"] })
+  config.add_target :log, logger: Rails.logger, formatter: :logfmt, transform: :l2met)
 end
 ```
 
@@ -85,8 +125,8 @@ Target is a simple object that implements `call` methods that accepts `telemetry
 Just be mindful that if the API takes long to call, it will slow down frequency with which telemetry will get reported.
 
 ```ruby
-  # Example logfmt to stdout target
-  config.add_target proc { |telemetry| puts telemetry.map { |k, v| "#{k}=#{v.inspect}" }.join(" ") }
+  # Example key/value log to `STDOUT` target
+  config.add_target ->(telemetry) { puts telemetry.map { |k, v| "#{k}=#{v.inspect}" }.join(" ") }
 ```
 
 ## Extra middleware

lib/puma/plugin/telemetry.rb

@@ -7,6 +7,7 @@
 require 'puma/plugin/telemetry/data'
 require 'puma/plugin/telemetry/targets/datadog_statsd_target'
 require 'puma/plugin/telemetry/targets/io_target'
+require 'puma/plugin/telemetry/targets/log_target'
 require 'puma/plugin/telemetry/config'
 
 module Puma

lib/puma/plugin/telemetry/config.rb

@@ -32,7 +32,8 @@ class Config
 
         TARGETS = {
           dogstatsd: Telemetry::Targets::DatadogStatsdTarget,
-          io: Telemetry::Targets::IOTarget
+          io: Telemetry::Targets::IOTarget,
+          log: Telemetry::Targets::LogTarget
         }.freeze
 
         # Whenever telemetry should run with puma

lib/puma/plugin/telemetry/formatters/json_formatter.rb

@@ -0,0 +1,19 @@
+# frozen_string_literal: true
+
+require 'json'
+
+module Puma
+  class Plugin
+    module Telemetry
+      module Formatters
+        # JSON formatter, expects `call` method accepting telemetry hash
+        #
+        class JSONFormatter
+          def self.call(telemetry)
+            ::JSON.dump(telemetry)
+          end
+        end
+      end
+    end
+  end
+end

lib/puma/plugin/telemetry/formatters/logfmt_formatter.rb

@@ -0,0 +1,17 @@
+# frozen_string_literal: true
+
+module Puma
+  class Plugin
+    module Telemetry
+      module Formatters
+        # Logfmt formatter, expects `call` method accepting telemetry hash
+        #
+        class LogfmtFormatter
+          def self.call(telemetry)
+            telemetry.map { |k, v| "#{String(k)}=#{v.inspect}" }.join(' ')
+          end
+        end
+      end
+    end
+  end
+end

lib/puma/plugin/telemetry/formatters/noop_formatter.rb

@@ -0,0 +1,16 @@
+# frozen_string_literal: true
+
+module Puma
+  class Plugin
+    module Telemetry
+      module Formatters
+        # A NOOP formatter - it returns the telemetry Hash it was given
+        class NoopFormatter
+          def self.call(telemetry)
+            telemetry
+          end
+        end
+      end
+    end
+  end
+end

lib/puma/plugin/telemetry/targets/base_formatting_target.rb

@@ -0,0 +1,47 @@
+# frozen_string_literal: true
+
+require_relative '../formatters/json_formatter'
+require_relative '../formatters/logfmt_formatter'
+require_relative '../formatters/noop_formatter'
+require_relative '../transforms/cloud_watch_transform'
+require_relative '../transforms/l2met_transform'
+require_relative '../transforms/noop_transform'
+
+module Puma
+  class Plugin
+    module Telemetry
+      module Targets
+        # A base class for other Targets concerned with formatting telemetry
+        #
+        class BaseFormattingTarget
+          def initialize(formatter: :json, transform: :cloud_watch)
+            @formatter = FORMATTERS.fetch(formatter) { formatter }
+            @transform = TRANSFORMS.fetch(transform) { transform }
+          end
+
+          def call(_telemetry)
+            raise "#{__method__} must be implemented by #{self.class.name}"
+          end
+
+          private
+
+          attr_reader :formatter, :transform
+
+          FORMATTERS = {
+            json: Formatters::JSONFormatter,
+            logfmt: Formatters::LogfmtFormatter,
+            noop: Formatters::NoopFormatter
+          }.freeze
+          private_constant :FORMATTERS
+
+          TRANSFORMS = {
+            cloud_watch: Transforms::CloudWatchTranform,
+            l2met: Transforms::L2metTransform,
+            noop: Transforms::NoopTransform
+          }.freeze
+          private_constant :TRANSFORMS
+        end
+      end
+    end
+  end
+end

lib/puma/plugin/telemetry/targets/io_target.rb

@@ -1,40 +1,26 @@
 # frozen_string_literal: true
 
-require 'json'
+require_relative 'base_formatting_target'
 
 module Puma
   class Plugin
     module Telemetry
       module Targets
         # Simple IO Target, publishing metrics to STDOUT or logs
         #
-        class IOTarget
-          # JSON formatter for IO, expects `call` method accepting telemetry hash
-          #
-          class JSONFormatter
-            # NOTE: Replace dots with dashes for better support of AWS CloudWatch
-            #       Log Metric filters, as they don't support dots in key names.
-            def self.call(telemetry)
-              log = telemetry.transform_keys { |k| k.tr('.', '-') }
-
-              log['name'] = 'Puma::Plugin::Telemetry'
-              log['message'] = 'Publish telemetry'
-
-              ::JSON.dump(log)
-            end
-          end
-
-          def initialize(io: $stdout, formatter: :json)
+        class IOTarget < BaseFormattingTarget
+          def initialize(io: $stdout, formatter: :json, transform: :cloud_watch)
+            super(formatter: formatter, transform: transform)
             @io = io
-            @formatter = case formatter
-                         when :json then JSONFormatter
-                         else formatter
-                         end
           end
 
           def call(telemetry)
-            @io.puts(@formatter.call(telemetry))
+            io.puts(formatter.call(transform.call(telemetry)))
           end
+
+          private
+
+          attr_reader :io
         end
       end
     end

lib/puma/plugin/telemetry/targets/log_target.rb

@@ -0,0 +1,30 @@
+# frozen_string_literal: true
+
+require 'logger'
+require_relative 'base_formatting_target'
+
+module Puma
+  class Plugin
+    module Telemetry
+      module Targets
+        # Simple Log Target, publishing metrics to a Ruby ::Logger at stdout
+        # at the INFO log level
+        #
+        class LogTarget < BaseFormattingTarget
+          def initialize(logger: ::Logger.new($stdout), formatter: :logfmt, transform: :noop)
+            super(formatter: formatter, transform: transform)
+            @logger = logger
+          end
+
+          def call(telemetry)
+            logger.info(formatter.call(transform.call(telemetry)))
+          end
+
+          private
+
+          attr_reader :logger
+        end
+      end
+    end
+  end
+end