Add structured metadata support for Loki output plugin

[error9098x]

Signed-off-by: ProCodec 43810146+error9098x@users.noreply.github.com

What this PR does / why we need it:

Adds support for structured metadata in the Loki output plugin. This allows attaching high-cardinality data (like pod names, trace IDs, thread IDs) to log lines without using them as labels, following Loki's recommended best practices for Loki 3.0+.

The implementation adds:

• structuredMetadata field to set key-value pairs as structured metadata
• structuredMetadataKeys field to reference record keys for structured metadata
• Processing logic in the Params() method to generate the proper Fluent Bit configuration
• Test cases to verify the functionality

Which issue(s) this PR fixes:

Fixes #1551

Does this PR introduced a user-facing change?

Added support for structured metadata in Loki output plugin. Users can now configure high-cardinality data to be sent as structured metadata instead of labels.

Note: This feature requires Loki 3.0+ with schema v13 or higher, TSDB storage, and allow_structured_metadata: true in Loki configuration.

Additional documentation, usage docs, etc.:

Example usage:

apiVersion: fluentbit.fluent.io/v1alpha2
kind: ClusterOutput
metadata:
name: loki-output
spec:
match: kube.*
loki:
host: loki-gateway
port: 3100
labels:
- job=fluentbit
- environment=production
structuredMetadata:
pod: "${record['kubernetes']['pod_name']}"
      container: "${record['kubernetes']['container_name']}"
trace_id: "\${record['trace_id']}"
structuredMetadataKeys:
- level
- caller

For more info on structured metadata: https://grafana.com/docs/loki/latest/get-started/labels/structured-metadata/

[wenchajun]

This one lacks the necessary crd

[wenchajun]

like this https://github.com/fluent/fluent-operator/pull/1567/files#diff-c5f9558fe8a96922357adef343c180e7f93bb8d66f112e4040e3638d0bca4792

[cw-Guo]

to resolve this issue, please run make generate, make manifests, and make docs-update

[error9098x]

to resolve this issue, please run make generate, make manifests, and make docs-update

did that, is it ok now?

[cw-Guo]

LGTM

[cw-Guo]

@error9098x , there is a failed test, can you please fix it?

[error9098x]

@error9098x , there is a failed test, can you please fix it?

it appears that it was failing because of the label order difference in loki_type.go output and the test. I have added sorting mechanism to fix that.