Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CCM6126 respond with a json:api format for 413 errors #752

Merged
merged 32 commits into from
Oct 7, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
5dc7da1
CCM6126: initial commit for 413 error
smaldon-bjss Sep 27, 2024
4a475a0
CCM-6126: revert mistaken edit
smaldon-bjss Sep 30, 2024
d0fb2f9
CCM-6126: initial attempt at limiting number of messages per request
smaldon-bjss Sep 30, 2024
34c75d1
CCM-6126: adjust status code for too many items error
smaldon-bjss Sep 30, 2024
cc5a139
CCM-6126: attempt reverting change to validation logic
smaldon-bjss Oct 1, 2024
c81e814
Revert "CCM-6126: attempt reverting change to validation logic"
smaldon-bjss Oct 1, 2024
e932318
CCM-6126: effort at creating a generic raisefault function
smaldon-bjss Oct 1, 2024
b603f1c
CCM-6126: amend typo
smaldon-bjss Oct 1, 2024
18fe637
CCM-6126: effort to make reference to status code work directly
smaldon-bjss Oct 1, 2024
d97a153
CCM-6126: attempt using another context variable
smaldon-bjss Oct 1, 2024
426a825
CCM-6126: leave out reasonphrase
smaldon-bjss Oct 1, 2024
622527f
CCM-6126: update pre-existing tests
smaldon-bjss Oct 1, 2024
0196b1e
CCM-6126: change these expected status codes back now that we're just…
smaldon-bjss Oct 1, 2024
91ae115
CCM-6126: development test for 413 errors
smaldon-bjss Oct 1, 2024
45377a2
CCM-6126: tests for 413 errors
smaldon-bjss Oct 1, 2024
2005de4
CCM-6126: documentation updates
smaldon-bjss Oct 1, 2024
65cdbb2
CCM-6126: linter fixes
smaldon-bjss Oct 1, 2024
fd0cad9
CCM-6126: remove backend test for sandbox environment - not applicable
smaldon-bjss Oct 1, 2024
a979f26
CCM-6126: update comment in raisefault policy
smaldon-bjss Oct 1, 2024
569884f
CCM-6126: update tests and documentation
smaldon-bjss Oct 2, 2024
2b998a9
CCM-6126: don't override apigee default for response title
smaldon-bjss Oct 2, 2024
a958fab
CCM-6126: update development tests to check content of errors
smaldon-bjss Oct 2, 2024
d874d72
CCM-6126: remove pointer from 413 for payload too large
smaldon-bjss Oct 3, 2024
5fc3431
CCM-6126: tests to check returned errors
smaldon-bjss Oct 3, 2024
78e4c9d
CCM-6126: phase out non-generic policy
smaldon-bjss Oct 3, 2024
51ffe1a
CCM-6126: change response format again
smaldon-bjss Oct 4, 2024
b5072aa
CCM-6126: openapi spec for 413 errors
smaldon-bjss Oct 4, 2024
524047e
CCM-6126: fix pointer to error enum
smaldon-bjss Oct 4, 2024
109f805
CCM-6126: swap order things appear in for formatting reasons
smaldon-bjss Oct 4, 2024
1b6396e
CCM-6126: remove 413 errors from other endpoints besides batch
smaldon-bjss Oct 4, 2024
18f87f6
CCM-6126: change example
smaldon-bjss Oct 4, 2024
5520e76
CCM-6126: remove enums from error pointer
smaldon-bjss Oct 4, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions docs/tests/development/post_v1_message-batches/performance.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
# Performance Tests


## Scenario: An API consumer submitting a request with a request body containing 50,000 messages receives a 201 response
## Scenario: An API consumer submitting a request with a request body containing 40,000 messages receives a 201 response

**Given** the API consumer provides a message body of around 50k messages
**Given** the API consumer provides a message body of around 40k messages
<br/>
**When** the request is submitted
<br/>
Expand All @@ -16,9 +16,9 @@
- Response returns a 201 status code


## Scenario: An API consumer submitting a request with a large request body containing 50,000 duplicate messages receives a 400 response
## Scenario: An API consumer submitting a request with a large request body containing 40,000 duplicate messages receives a 400 response

**Given** the API consumer provides a message body of 50,000 duplicate messages
**Given** the API consumer provides a message body of 40,000 duplicate messages
<br/>
**When** the request is submitted
<br/>
Expand Down
8 changes: 4 additions & 4 deletions docs/tests/integration/post_v1_message-batches/performance.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
# Performance Tests


## Scenario: An API consumer submitting a request with a request body containing 50,000 messages receives a 201 response
## Scenario: An API consumer submitting a request with a request body containing 40,000 messages receives a 201 response

**Given** the API consumer provides a message body of around 50k messages
**Given** the API consumer provides a message body of around 40k messages
<br/>
**When** the request is submitted
<br/>
Expand All @@ -16,9 +16,9 @@
- Response returns a 201 status code


## Scenario: An API consumer submitting a request with a large request body containing 50,000 duplicate messages receives a 400 response
## Scenario: An API consumer submitting a request with a large request body containing 40,000 duplicate messages receives a 400 response

**Given** the API consumer provides a message body of 50,000 duplicate messages
**Given** the API consumer provides a message body of 40,000 duplicate messages
<br/>
**When** the request is submitted
<br/>
Expand Down
8 changes: 4 additions & 4 deletions docs/tests/production/post_v1_message-batches/performance.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
# Performance Tests


## Scenario: An API consumer submitting a request with a request body containing 50,000 messages receives a 201 response
## Scenario: An API consumer submitting a request with a request body containing 40,000 messages receives a 201 response

**Given** the API consumer provides a message body of around 50k messages
**Given** the API consumer provides a message body of around 40k messages
<br/>
**When** the request is submitted
<br/>
Expand All @@ -16,9 +16,9 @@
- Response returns a 201 status code


## Scenario: An API consumer submitting a request with a large request body containing 50,000 duplicate messages receives a 400 response
## Scenario: An API consumer submitting a request with a large request body containing 40,000 duplicate messages receives a 400 response

**Given** the API consumer provides a message body of 50,000 duplicate messages
**Given** the API consumer provides a message body of 40,000 duplicate messages
<br/>
**When** the request is submitted
<br/>
Expand Down
12 changes: 6 additions & 6 deletions docs/tests/sandbox/post_v1_message-batches/performance.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
# Performance Tests


## Scenario: An API consumer submitting a request with a request body containing 50,000 messages receives a 201 response
## Scenario: An API consumer submitting a request with a request body containing 40,000 messages receives a 201 response

**Given** the API consumer provides a message body of around 50k messages
**Given** the API consumer provides a message body of around 40k messages
<br/>
**When** the request is submitted
<br/>
Expand All @@ -16,9 +16,9 @@
- Response returns a 201 status code


## Scenario: An API consumer submitting a request with a large request body containing 50,000 duplicate messages receives a 400 response
## Scenario: An API consumer submitting a request with a large request body containing 40,000 duplicate messages receives a 400 response

**Given** the API consumer provides a message body of 50,000 duplicate messages
**Given** the API consumer provides a message body of 40,000 duplicate messages
<br/>
**When** the request is submitted
<br/>
Expand All @@ -34,9 +34,9 @@
- Response returns 100 error message blocks


## Scenario: An API consumer submitting a request with a request body containing 50,000 messages receives a 201 response
## Scenario: An API consumer submitting a request with a request body containing 40,000 messages receives a 201 response

**Given** the API consumer provides a message body of around 50k messages
**Given** the API consumer provides a message body of around 40k messages
<br/>
**When** the request is submitted
<br/>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<Name>JavaScript.MessageBatches.Create.Validate</Name>
</Step>
<Step>
<Name>RaiseFault.400BadRequest</Name>
<Name>RaiseFault.4xxGeneric</Name>
adadigital marked this conversation as resolved.
Show resolved Hide resolved
<Condition>errors != null</Condition>
</Step>
<Step>
Expand All @@ -31,4 +31,4 @@
<Condition>
(proxy.pathsuffix MatchesPath "/v1/message-batches") and (request.verb = "POST")
</Condition>
</Flow>
</Flow>
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<Name>JavaScript.Messages.Create.Validate</Name>
</Step>
<Step>
<Name>RaiseFault.400BadRequest</Name>
<Name>RaiseFault.4xxGeneric</Name>
<Condition>errors != null</Condition>
</Step>
<Step>
Expand All @@ -31,4 +31,4 @@
<Condition>
(proxy.pathsuffix MatchesPath "/v1/messages") and (request.verb = "POST")
</Condition>
</Flow>
</Flow>
6 changes: 6 additions & 0 deletions proxies/shared/partials/Partial.Target.FaultRules.xml
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,12 @@
<Name>RaiseFault.404NotFound</Name>
<Condition>response.status.code = 404</Condition>
</Step>
<Step>
<Name>RaiseFault.413RequestTooLarge</Name>
<Condition>
response.status.code = 413 and response.content Like "*Request Too Long*"
</Condition>
</Step>
<Step>
<Name>RaiseFault.400BackendException.OdsCodeRequired</Name>
<Condition>response.status.code = 400 and response.content Like "*odsCode must be provided*"</Condition>
Expand Down
42 changes: 42 additions & 0 deletions proxies/shared/policies/RaiseFault.413RequestTooLarge.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<!--
This policy raises a 413 error response for a request that was too large

Valid values are:
* `*/*`
* `application/json`
* `application/vnd.api+json`

Raisefault policies stop the execution of the current flow and move to the error flow, which returns the error response defined here to the requesting application.

For more information on RaiseFault policies within Apigee see the following resource:
* https://docs.apigee.com/api-platform/reference/policies/raise-fault-policy
-->
<RaiseFault async="false" continueOnError="false" enabled="true" name="RaiseFault.413RequestTooLarge">
<DisplayName>RaiseFault.413RequestTooLarge</DisplayName>
<Properties/>
<FaultResponse>
<Set>
<Headers/>
<StatusCode>413</StatusCode>
<Payload variablePrefix="@" variableSuffix="#">
{
"errors" : [
{
"id" : "@messageid#.0",
"code" : "CM_TOO_LARGE",
"links" : {
"about" : "{{ ERROR_ABOUT_LINK }}"
},
"status" : "413",
"title" : "Request too large",
"detail" : "Request message was larger than the service limit",
"source": {"pointer": "/"}
}
]
}
</Payload>
</Set>
</FaultResponse>
<IgnoreUnresolvedVariables>true</IgnoreUnresolvedVariables>
</RaiseFault>
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<!--
This policy raises a 400 error response containing an array of errors populated by the MessageBatches.Create.Validate.js script.
This policy raises a 4xx error response containing an array of errors populated by the MessageBatches.Create.Validate.js script, the exact status code depends on context variables.

Since the payload is JSON we can't use the standard {variable.name} syntax for Apigee flow variables, this is where we use the variablePrefix and variableSuffix attributes.

Expand All @@ -9,8 +9,8 @@
For more information on RaiseFault policies within Apigee see the following resource:
* https://docs.apigee.com/api-platform/reference/policies/raise-fault-policy
-->
<RaiseFault async="false" continueOnError="false" enabled="true" name="RaiseFault.400BadRequest">
<DisplayName>RaiseFault.400BadRequest</DisplayName>
<RaiseFault async="false" continueOnError="false" enabled="true" name="RaiseFault.4xxGeneric">
<DisplayName>RaiseFault.4xxGeneric</DisplayName>
<Properties/>
<FaultResponse>
<Set>
Expand All @@ -20,8 +20,7 @@
"errors" : %errors#
}
</Payload>
<StatusCode>400</StatusCode>
<ReasonPhrase>Bad request</ReasonPhrase>
<StatusCode>{generic_status_code}</StatusCode>
</Set>
</FaultResponse>
<IgnoreUnresolvedVariables>true</IgnoreUnresolvedVariables>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,10 @@ const validate = () => {
// $.data.attributes.messages
const validArray = validateArray(errors, data.attributes.messages, "/data/attributes/messages", 1)
if (validArray) {
if (data.attributes.messages.length > 45000) {
errors.push(tooManyItemsError("/data/attributes/messages"));
return null;
}
// $.data.attributes.messages.x
data.attributes.messages.forEach((message, index) => {
var pointer = "/data/attributes/messages/" + index;
Expand Down Expand Up @@ -106,7 +110,9 @@ const validate = () => {
validate();

if (errors.length > 0) {
context.setVariable("generic_status_code", errors[0].status);
context.setVariable("errors", JSON.stringify(errors));
} else {
context.setVariable("generic_status_code", null);
context.setVariable("errors", null);
}
2 changes: 2 additions & 0 deletions proxies/shared/resources/jsc/Messages.Create.Validate.js
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,9 @@ const validate = () => {
validate();

if (errors.length > 0) {
context.setVariable("generic_status_code", errors[0].status);
context.setVariable("errors", JSON.stringify(errors));
} else {
context.setVariable("generic_status_code", null);
context.setVariable("errors", null);
}
22 changes: 19 additions & 3 deletions proxies/shared/resources/jsc/helpers/validationErrors.js
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
function createErrorObject(code, title, detail, pointer, links) {
function createErrorObject(code, status_code, title, detail, pointer, links) {
return {
"id": messageId + "." + errors.length,
"code": code,
"links": Object.assign({}, { "about": "https://digital.nhs.uk/developer/api-catalogue/nhs-notify" }, links), // NOSONAR
"status": "400",
"status": status_code,
"title": title,
"detail": detail,
"source": {
Expand All @@ -15,6 +15,7 @@ function createErrorObject(code, title, detail, pointer, links) {
function missingError(pointer) {
return createErrorObject(
"CM_MISSING_VALUE",
"400",
"Missing property",
"The property at the specified location is required, but was not present in the request.",
pointer,
Expand All @@ -25,6 +26,7 @@ function missingError(pointer) {
function nullError(pointer) {
return createErrorObject(
"CM_NULL_VALUE",
"400",
"Property cannot be null",
"The property at the specified location is required, but a null value was passed in the request.",
pointer,
Expand All @@ -35,6 +37,7 @@ function nullError(pointer) {
function invalidError(pointer) {
return createErrorObject(
"CM_INVALID_VALUE",
"400",
"Invalid value",
"The property at the specified location does not allow this value.",
pointer,
Expand All @@ -45,6 +48,7 @@ function invalidError(pointer) {
function duplicateError(pointer) {
return createErrorObject(
"CM_DUPLICATE_VALUE",
"400",
"Duplicate value",
"The property at the specified location is a duplicate, duplicated values are not allowed.",
pointer,
Expand All @@ -55,20 +59,32 @@ function duplicateError(pointer) {
function tooFewItemsError(pointer) {
return createErrorObject(
"CM_TOO_FEW_ITEMS",
"400",
"Too few items",
"The property at the specified location contains too few items.",
pointer,
{}
)
}

function tooManyItemsError(pointer) {
return createErrorObject(
"CM_TOO_MANY_ITEMS",
"413",
"Too many items",
"The property at the specified location contains too many items.",
pointer,
{}
)
}

function invalidNhsNumberError(pointer) {
return createErrorObject(
"CM_INVALID_NHS_NUMBER",
"400",
"Invalid nhs number",
"The value provided in this nhsNumber field is not a valid NHS number.",
pointer,
{ "nhsNumbers": "https://www.datadictionary.nhs.uk/attributes/nhs_number.html" }
);
}

2 changes: 2 additions & 0 deletions specification/endpoints/create_message_batch.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,8 @@ responses:
$ref: ../responses/4xx/406_NotAcceptable.yaml
'408':
$ref: ../responses/4xx/408_RequestTimeout.yaml
'413':
$ref: ../responses/4xx/413_RequestEntityTooLarge.yaml
'415':
$ref: ../responses/4xx/415_UnsupportedMedia.yaml
'425':
Expand Down
12 changes: 12 additions & 0 deletions specification/responses/4xx/413_RequestEntityTooLarge.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
description: |+
Either the whole of or some part of the request was too large for the host side to handle and was rejected.

content:
application/vnd.api+json:
schema:
$ref: ../../schemas/responses/errors/RequestEntityTooLarge.yaml
application/json:
schema:
$ref: ../../schemas/responses/errors/RequestEntityTooLarge.yaml
headers:
$ref: ../../snippets/StandardResponseHeaders.yaml
6 changes: 6 additions & 0 deletions specification/schemas/enums/ErrorRequestEntityTooLarge.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
title: Enum_Error_RequestEntityTooLarge
type: string
enum:
- CM_TOO_LARGE
- CM_TOO_MANY_ITEMS
example: CM_TOO_LARGE
Loading
Loading