Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(translator): JsonPath in PatchPolicy #3757

Merged
merged 1 commit into from
Aug 13, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion api/v1alpha1/envoypatchpolicy_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,12 @@ type JSONPatchOperation struct {
Op JSONPatchOperationType `json:"op"`
// Path is the location of the target document/field where the operation will be performed
// Refer to https://datatracker.ietf.org/doc/html/rfc6901 for more details.
Path string `json:"path"`
// +optional
Path *string `json:"path,omitempty"`
// JSONPath specifies the locations of the target document/field where the operation will be performed
// Refer to https://datatracker.ietf.org/doc/rfc9535/ for more details.
// +optional
JSONPath *string `json:"jsonPath,omitempty"`
// From is the source location of the value to be copied or moved. Only valid
// for move or copy operations
// Refer to https://datatracker.ietf.org/doc/html/rfc6901 for more details.
Expand Down
10 changes: 10 additions & 0 deletions api/v1alpha1/zz_generated.deepcopy.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,11 @@ spec:
for move or copy operations
Refer to https://datatracker.ietf.org/doc/html/rfc6901 for more details.
type: string
jsonPath:
description: |-
JSONPath specifies the locations of the target document/field where the operation will be performed
Refer to https://datatracker.ietf.org/doc/rfc9535/ for more details.
type: string
op:
description: Op is the type of operation to perform
enum:
Expand All @@ -93,7 +98,6 @@ spec:
x-kubernetes-preserve-unknown-fields: true
required:
- op
- path
type: object
type:
description: Type is the typed URL of the Envoy xDS Resource
Expand Down
1 change: 1 addition & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ require (
github.com/grafana/tempo v1.5.0
github.com/hashicorp/go-multierror v1.1.1
github.com/miekg/dns v1.1.61
github.com/ohler55/ojg v1.22.1
github.com/prometheus/client_golang v1.19.1
github.com/prometheus/common v0.55.0
github.com/spf13/cobra v1.8.1
Expand Down
2 changes: 2 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -899,6 +899,8 @@ github.com/mxk/go-flowrate v0.0.0-20140419014527-cca7078d478f/go.mod h1:ZdcZmHo+
github.com/nxadm/tail v1.4.4/go.mod h1:kenIhsEOeOJmVchQTgglprH7qJGnHDVpk1VPCcaMI8A=
github.com/nxadm/tail v1.4.8 h1:nPr65rt6Y5JFSKQO7qToXr7pePgD6Gwiw05lkbyAQTE=
github.com/nxadm/tail v1.4.8/go.mod h1:+ncqLTQzXmGhMZNUePPaPqPvBxHAIsmXswZKocGu+AU=
github.com/ohler55/ojg v1.22.1 h1:MvUieaWTwksoYk47GYyP9kzXIAkxHYX6rxeLjUEeq/8=
github.com/ohler55/ojg v1.22.1/go.mod h1:gQhDVpQLqrmnd2eqGAvJtn+NfKoYJbe/A4Sj3/Vro4o=
github.com/oklog/ulid v1.3.1/go.mod h1:CirwcVhetQ6Lv90oh/F+FBtV6XMibvdAFo93nm5qn4U=
github.com/olekukonko/tablewriter v0.0.0-20170122224234-a0225b3f23b5/go.mod h1:vsDQFd/mU46D+Z4whnwzcISnGGzXWMclvtLoiIKAKIo=
github.com/onsi/ginkgo v0.0.0-20170829012221-11459a886d9c/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
Expand Down
1 change: 1 addition & 0 deletions internal/gatewayapi/envoypatchpolicy.go
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,7 @@ func (t *Translator) ProcessEnvoyPatchPolicies(envoyPatchPolicies []*egv1a1.Envo
irPatch.Name = patch.Name
irPatch.Operation.Op = string(patch.Operation.Op)
irPatch.Operation.Path = patch.Operation.Path
irPatch.Operation.JSONPath = patch.Operation.JSONPath
irPatch.Operation.From = patch.Operation.From
irPatch.Operation.Value = patch.Operation.Value

Expand Down
19 changes: 18 additions & 1 deletion internal/ir/xds.go
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,10 @@ import (
egv1a1validation "github.com/envoyproxy/gateway/api/v1alpha1/validation"
)

const (
EmptyPath = ""
)

var (
ErrListenerNameEmpty = errors.New("field Name must be specified")
ErrListenerAddressInvalid = errors.New("field Address must be a valid IP address")
Expand Down Expand Up @@ -1720,7 +1724,12 @@ type JSONPatchOperation struct {
Op string `json:"op" yaml:"op"`
// Path is the location of the target document/field where the operation will be performed
// Refer to https://datatracker.ietf.org/doc/html/rfc6901 for more details.
Path string `json:"path" yaml:"path"`
// +optional
Path *string `json:"path,omitempty" yaml:"path,omitempty"`
// JSONPath specifies the locations of the target document/field where the operation will be performed
// Refer to https://datatracker.ietf.org/doc/rfc9535/ for more details.
// +optional
JSONPath *string `json:"jsonPath,omitempty" yaml:"jsonPath,omitempty"`
// From is the source location of the value to be copied or moved. Only valid
// for move or copy operations
// Refer to https://datatracker.ietf.org/doc/html/rfc6901 for more details.
Expand All @@ -1730,6 +1739,14 @@ type JSONPatchOperation struct {
Value *apiextensionsv1.JSON `json:"value,omitempty" yaml:"value,omitempty"`
}

func (o *JSONPatchOperation) IsPathNilOrEmpty() bool {
return o.Path == nil || *o.Path == EmptyPath
}

func (o *JSONPatchOperation) IsJSONPathNilOrEmpty() bool {
return o.JSONPath == nil || *o.JSONPath == EmptyPath
}

// Tracing defines the configuration for tracing a Envoy xDS Resource
// +k8s:deepcopy-gen=true
type Tracing struct {
Expand Down
10 changes: 10 additions & 0 deletions internal/ir/zz_generated.deepcopy.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

233 changes: 129 additions & 104 deletions internal/xds/translator/jsonpatch.go
Original file line number Diff line number Diff line change
Expand Up @@ -84,9 +84,9 @@
}
}

// If Path is "" and op is "add", unmarshal and add the patch as a complete
// If Path and JSONPath is "" and op is "add", unmarshal and add the patch as a complete
// resource
if p.Operation.Op == AddOperation && p.Operation.Path == EmptyPath {
if p.Operation.Op == AddOperation && p.Operation.IsPathNilOrEmpty() && p.Operation.IsJSONPathNilOrEmpty() {
// Convert patch to JSON
// The patch library expects an array so convert it into one
y, err := yaml.Marshal(p.Operation.Value)
Expand Down Expand Up @@ -240,125 +240,150 @@
}
}

// Convert patch to JSON
// The patch library expects an array so convert it into one
y, err := yaml.Marshal([]ir.JSONPatchOperation{p.Operation})
if err != nil {
tErr := fmt.Errorf("unable to marshal patch %+v, err: %s", p.Operation, err.Error())
tErrs = errors.Join(tErrs, tErr)
continue
}
jsonBytes, err := yaml.YAMLToJSON(y)
if err != nil {
tErr := fmt.Errorf("unable to convert patch to json %s, err: %s", string(y), err.Error())
tErrs = errors.Join(tErrs, tErr)
continue
}
patchObj, err := jsonpatchv5.DecodePatch(jsonBytes)
if err != nil {
tErr := fmt.Errorf("unable to decode patch %s, err: %s", string(jsonBytes), err.Error())
tErrs = errors.Join(tErrs, tErr)
continue
}

// Apply patch
opts := jsonpatchv5.NewApplyOptions()
opts.EnsurePathExistsOnAdd = true
modifiedJSON, err := patchObj.ApplyWithOptions(resourceJSON, opts)
if err != nil {
tErr := fmt.Errorf("unable to apply patch:\n%s on resource:\n%s, err: %s", string(jsonBytes), string(resourceJSON), err.Error())
tErrs = errors.Join(tErrs, tErr)
continue
}

// Unmarshal back to typed resource
// Use a temp staging variable that can be marshalled
// into and validated before saving it into the xds output resource
switch p.Type {
case resourcev3.ListenerType:
temp := &listenerv3.Listener{}
if err = protojson.Unmarshal(modifiedJSON, temp); err != nil {
tErr := fmt.Errorf(unmarshalErrorMessage(err, string(modifiedJSON)))
tErrs = errors.Join(tErrs, tErr)
continue
var jsonPointers []string
if p.Operation.JSONPath != nil {
arkodg marked this conversation as resolved.
Show resolved Hide resolved
path := ""
if p.Operation.Path != nil {
path = *p.Operation.Path
}
if err = temp.Validate(); err != nil {
tErr := fmt.Errorf("validation failed for xds resource %s, err:%s", string(modifiedJSON), err.Error())
tErrs = errors.Join(tErrs, tErr)
continue
}
if err = deepCopyPtr(temp, listener); err != nil {
tErr := fmt.Errorf("unable to copy xds resource %s, err: %w", string(modifiedJSON), err)
tErrs = errors.Join(tErrs, tErr)
continue
}
case resourcev3.RouteType:
temp := &routev3.RouteConfiguration{}
if err = protojson.Unmarshal(modifiedJSON, temp); err != nil {
tErr := fmt.Errorf(unmarshalErrorMessage(err, string(modifiedJSON)))
tErrs = errors.Join(tErrs, tErr)
continue
}
if err = temp.Validate(); err != nil {
tErr := fmt.Errorf("validation failed for xds resource %s, err:%s", string(modifiedJSON), err.Error())
tErrs = errors.Join(tErrs, tErr)
continue
}
if err = deepCopyPtr(temp, routeConfig); err != nil {
tErr := fmt.Errorf("unable to copy xds resource %s, err: %w", string(modifiedJSON), err)
tErrs = errors.Join(tErrs, tErr)
continue
}
case resourcev3.ClusterType:
temp := &clusterv3.Cluster{}
if err = protojson.Unmarshal(modifiedJSON, temp); err != nil {
tErr := fmt.Errorf(unmarshalErrorMessage(err, string(modifiedJSON)))
tErrs = errors.Join(tErrs, tErr)
continue
}
if err = temp.Validate(); err != nil {
tErr := fmt.Errorf("validation failed for xds resource %s, err:%s", string(modifiedJSON), err.Error())
tErrs = errors.Join(tErrs, tErr)
continue
}
if err = deepCopyPtr(temp, cluster); err != nil {
tErr := fmt.Errorf("unable to copy xds resource %s, err: %w", string(modifiedJSON), err)
jsonPointers, err = ConvertPathToPointers(resourceJSON, *p.Operation.JSONPath, path)
if err != nil {
tErr := fmt.Errorf("unable to convert jsonPath: '%s' into jsonPointers, err: %s", *p.Operation.JSONPath, err.Error())

Check warning on line 251 in internal/xds/translator/jsonpatch.go

View check run for this annotation

Codecov / codecov/patch

internal/xds/translator/jsonpatch.go#L251

Added line #L251 was not covered by tests
tErrs = errors.Join(tErrs, tErr)
continue
}
case resourcev3.EndpointType:
temp := &endpointv3.ClusterLoadAssignment{}
if err = protojson.Unmarshal(modifiedJSON, temp); err != nil {
tErr := fmt.Errorf(unmarshalErrorMessage(err, string(modifiedJSON)))
tErrs = errors.Join(tErrs, tErr)
continue
} else {
jsonPointers = []string{*p.Operation.Path}
}

for _, path := range jsonPointers {
op := ir.JSONPatchOperation{
Path: &path,
Op: p.Operation.Op,
Value: p.Operation.Value,
From: p.Operation.From,
}
if err = temp.Validate(); err != nil {
tErr := fmt.Errorf("validation failed for xds resource %s, err:%s", string(modifiedJSON), err.Error())

// Convert patch to JSON
// The patch library expects an array so convert it into one
y, err := yaml.Marshal([]ir.JSONPatchOperation{op})
if err != nil {
tErr := fmt.Errorf("unable to marshal patch %+v, err: %s", op, err.Error())

Check warning on line 271 in internal/xds/translator/jsonpatch.go

View check run for this annotation

Codecov / codecov/patch

internal/xds/translator/jsonpatch.go#L271

Added line #L271 was not covered by tests
tErrs = errors.Join(tErrs, tErr)
continue
}
if err = deepCopyPtr(temp, endpoint); err != nil {
tErr := fmt.Errorf("unable to copy xds resource %s, err: %w", string(modifiedJSON), err)
jsonBytes, err := yaml.YAMLToJSON(y)
if err != nil {
tErr := fmt.Errorf("unable to convert patch to json %s, err: %s", string(y), err.Error())

Check warning on line 277 in internal/xds/translator/jsonpatch.go

View check run for this annotation

Codecov / codecov/patch

internal/xds/translator/jsonpatch.go#L277

Added line #L277 was not covered by tests
tErrs = errors.Join(tErrs, tErr)
continue
}
case resourcev3.SecretType:
temp := &tlsv3.Secret{}
if err = protojson.Unmarshal(modifiedJSON, temp); err != nil {
tErr := fmt.Errorf(unmarshalErrorMessage(err, string(modifiedJSON)))
patchObj, err := jsonpatchv5.DecodePatch(jsonBytes)
if err != nil {
tErr := fmt.Errorf("unable to decode patch %s, err: %s", string(jsonBytes), err.Error())

Check warning on line 283 in internal/xds/translator/jsonpatch.go

View check run for this annotation

Codecov / codecov/patch

internal/xds/translator/jsonpatch.go#L283

Added line #L283 was not covered by tests
tErrs = errors.Join(tErrs, tErr)
continue
}
if err = temp.Validate(); err != nil {
tErr := fmt.Errorf("validation failed for xds resource %s, err:%s", string(modifiedJSON), err.Error())

// Apply patch
opts := jsonpatchv5.NewApplyOptions()
opts.EnsurePathExistsOnAdd = true
modifiedJSON, err := patchObj.ApplyWithOptions(resourceJSON, opts)
if err != nil {
tErr := fmt.Errorf("unable to apply patch:\n%s on resource:\n%s, err: %s", string(jsonBytes), string(resourceJSON), err.Error())

Check warning on line 293 in internal/xds/translator/jsonpatch.go

View check run for this annotation

Codecov / codecov/patch

internal/xds/translator/jsonpatch.go#L293

Added line #L293 was not covered by tests
tErrs = errors.Join(tErrs, tErr)
continue
}
if err = deepCopyPtr(temp, secret); err != nil {
tErr := fmt.Errorf("unable to copy xds resource %s, err: %w", string(modifiedJSON), err)
tErrs = errors.Join(tErrs, tErr)
continue

// Unmarshal back to typed resource
// Use a temp staging variable that can be marshalled
// into and validated before saving it into the xds output resource
switch p.Type {
case resourcev3.ListenerType:
temp := &listenerv3.Listener{}
if err = protojson.Unmarshal(modifiedJSON, temp); err != nil {
tErr := fmt.Errorf(unmarshalErrorMessage(err, string(modifiedJSON)))
tErrs = errors.Join(tErrs, tErr)
continue
}
if err = temp.Validate(); err != nil {
tErr := fmt.Errorf("validation failed for xds resource %s, err:%s", string(modifiedJSON), err.Error())
tErrs = errors.Join(tErrs, tErr)
continue

Check warning on line 312 in internal/xds/translator/jsonpatch.go

View check run for this annotation

Codecov / codecov/patch

internal/xds/translator/jsonpatch.go#L310-L312

Added lines #L310 - L312 were not covered by tests
}
if err = deepCopyPtr(temp, listener); err != nil {
tErr := fmt.Errorf("unable to copy xds resource %s, err: %w", string(modifiedJSON), err)
tErrs = errors.Join(tErrs, tErr)
continue

Check warning on line 317 in internal/xds/translator/jsonpatch.go

View check run for this annotation

Codecov / codecov/patch

internal/xds/translator/jsonpatch.go#L315-L317

Added lines #L315 - L317 were not covered by tests
}
case resourcev3.RouteType:
temp := &routev3.RouteConfiguration{}
if err = protojson.Unmarshal(modifiedJSON, temp); err != nil {
tErr := fmt.Errorf(unmarshalErrorMessage(err, string(modifiedJSON)))
tErrs = errors.Join(tErrs, tErr)
continue

Check warning on line 324 in internal/xds/translator/jsonpatch.go

View check run for this annotation

Codecov / codecov/patch

internal/xds/translator/jsonpatch.go#L322-L324

Added lines #L322 - L324 were not covered by tests
}
if err = temp.Validate(); err != nil {
tErr := fmt.Errorf("validation failed for xds resource %s, err:%s", string(modifiedJSON), err.Error())
tErrs = errors.Join(tErrs, tErr)
continue

Check warning on line 329 in internal/xds/translator/jsonpatch.go

View check run for this annotation

Codecov / codecov/patch

internal/xds/translator/jsonpatch.go#L327-L329

Added lines #L327 - L329 were not covered by tests
}
if err = deepCopyPtr(temp, routeConfig); err != nil {
tErr := fmt.Errorf("unable to copy xds resource %s, err: %w", string(modifiedJSON), err)
tErrs = errors.Join(tErrs, tErr)
continue

Check warning on line 334 in internal/xds/translator/jsonpatch.go

View check run for this annotation

Codecov / codecov/patch

internal/xds/translator/jsonpatch.go#L332-L334

Added lines #L332 - L334 were not covered by tests
}
case resourcev3.ClusterType:
temp := &clusterv3.Cluster{}
if err = protojson.Unmarshal(modifiedJSON, temp); err != nil {
tErr := fmt.Errorf(unmarshalErrorMessage(err, string(modifiedJSON)))
tErrs = errors.Join(tErrs, tErr)
continue

Check warning on line 341 in internal/xds/translator/jsonpatch.go

View check run for this annotation

Codecov / codecov/patch

internal/xds/translator/jsonpatch.go#L336-L341

Added lines #L336 - L341 were not covered by tests
}
if err = temp.Validate(); err != nil {
tErr := fmt.Errorf("validation failed for xds resource %s, err:%s", string(modifiedJSON), err.Error())
tErrs = errors.Join(tErrs, tErr)
continue

Check warning on line 346 in internal/xds/translator/jsonpatch.go

View check run for this annotation

Codecov / codecov/patch

internal/xds/translator/jsonpatch.go#L343-L346

Added lines #L343 - L346 were not covered by tests
}
if err = deepCopyPtr(temp, cluster); err != nil {
tErr := fmt.Errorf("unable to copy xds resource %s, err: %w", string(modifiedJSON), err)
tErrs = errors.Join(tErrs, tErr)
continue

Check warning on line 351 in internal/xds/translator/jsonpatch.go

View check run for this annotation

Codecov / codecov/patch

internal/xds/translator/jsonpatch.go#L348-L351

Added lines #L348 - L351 were not covered by tests
}
case resourcev3.EndpointType:
temp := &endpointv3.ClusterLoadAssignment{}
if err = protojson.Unmarshal(modifiedJSON, temp); err != nil {
tErr := fmt.Errorf(unmarshalErrorMessage(err, string(modifiedJSON)))
tErrs = errors.Join(tErrs, tErr)
continue

Check warning on line 358 in internal/xds/translator/jsonpatch.go

View check run for this annotation

Codecov / codecov/patch

internal/xds/translator/jsonpatch.go#L356-L358

Added lines #L356 - L358 were not covered by tests
}
if err = temp.Validate(); err != nil {
tErr := fmt.Errorf("validation failed for xds resource %s, err:%s", string(modifiedJSON), err.Error())
tErrs = errors.Join(tErrs, tErr)
continue

Check warning on line 363 in internal/xds/translator/jsonpatch.go

View check run for this annotation

Codecov / codecov/patch

internal/xds/translator/jsonpatch.go#L361-L363

Added lines #L361 - L363 were not covered by tests
}
if err = deepCopyPtr(temp, endpoint); err != nil {
tErr := fmt.Errorf("unable to copy xds resource %s, err: %w", string(modifiedJSON), err)
tErrs = errors.Join(tErrs, tErr)
continue

Check warning on line 368 in internal/xds/translator/jsonpatch.go

View check run for this annotation

Codecov / codecov/patch

internal/xds/translator/jsonpatch.go#L366-L368

Added lines #L366 - L368 were not covered by tests
}
case resourcev3.SecretType:
temp := &tlsv3.Secret{}
if err = protojson.Unmarshal(modifiedJSON, temp); err != nil {
tErr := fmt.Errorf(unmarshalErrorMessage(err, string(modifiedJSON)))
tErrs = errors.Join(tErrs, tErr)
continue

Check warning on line 375 in internal/xds/translator/jsonpatch.go

View check run for this annotation

Codecov / codecov/patch

internal/xds/translator/jsonpatch.go#L373-L375

Added lines #L373 - L375 were not covered by tests
}
if err = temp.Validate(); err != nil {
tErr := fmt.Errorf("validation failed for xds resource %s, err:%s", string(modifiedJSON), err.Error())
tErrs = errors.Join(tErrs, tErr)
continue

Check warning on line 380 in internal/xds/translator/jsonpatch.go

View check run for this annotation

Codecov / codecov/patch

internal/xds/translator/jsonpatch.go#L378-L380

Added lines #L378 - L380 were not covered by tests
}
if err = deepCopyPtr(temp, secret); err != nil {
tErr := fmt.Errorf("unable to copy xds resource %s, err: %w", string(modifiedJSON), err)
tErrs = errors.Join(tErrs, tErr)
continue

Check warning on line 385 in internal/xds/translator/jsonpatch.go

View check run for this annotation

Codecov / codecov/patch

internal/xds/translator/jsonpatch.go#L383-L385

Added lines #L383 - L385 were not covered by tests
}
}
}
}
Expand Down
Loading