Skip to content

Commit

Permalink
remove functions wrapper
Browse files Browse the repository at this point in the history
  • Loading branch information
idanasulin2706 committed Nov 23, 2023
1 parent e1faf38 commit 5abef07
Show file tree
Hide file tree
Showing 2 changed files with 1 addition and 208 deletions.
101 changes: 1 addition & 100 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -951,103 +951,4 @@ consumer.destroy()

```python
memphis.is_connected()
```

### Creating a Memphis function
Memphis provides a create_function utility for more easily creatin Memphis Functions.

The user created `event_handler` will be called for every message in the given batch of events. The user's `event_handler` will take in a `msg_payload` as bytes, `msg_headers` as a dict and `inputs` as a dict, and should return a modified version of the payload and headers in the same data types.

The user function should raise an exception if the message processing has failed. If any exception is raised (deliberately or by a failed operation) the message will be sent to the dead letter station.

If the returned modified version of the `msg_payload` or `msg_headers` are returned as `None`, then the message will be skipped and will not be sent to the station or dead letter station.

> Make sure to encode the modified `msg_payload` bytes object with utf-8 encoding!
This example function takes the bytes object `msg_payload` and encodes it into a string so that it may be parsed as JSON.

```python
import json
import base64
from memphis.functions import create_function

def handler(event, context): # The name of this file and this function should match the handler field in the memphis.yaml file in the following format <file name>.<function name>
return create_function(event, event_handler = event_handler)

def event_handler(msg_payload, msg_headers, inputs):
payload = str(msg_payload, 'utf-8')
as_json = json.loads(payload)
as_json['modified'] = True

return bytes(json.dumps(as_json), encoding='utf-8'), msg_headers
```

If the user would want to have a message that they would want to validate and send to the dead letter station if the validation fails then the user can raise an exception. In the following example, the field `check` is simply a boolean. The following function will send any messages which fail the `check` to the dead letter station.

```python
import json
import base64
from memphis.functions import create_function

def handler(event, context): # The name of this file and this function should match the handler field in the memphis.yaml file in the following format <file name>.<function name>
return create_function(event, event_handler = event_handler)

def event_handler(msg_payload, msg_headers, inputs):
payload = str(msg_payload, 'utf-8')
as_json = json.loads(payload)
if as_json['check'] == False:
raise Exception("Validation Failed!")

return bytes(json.dumps(as_json), encoding='utf-8'), msg_headers
```

If a user would rather just skip the message and not have it be sent to the station or dead letter station, the cuser could instead return `None`, `None`:

```python
import json
import base64
from memphis.functions import create_function

def handler(event, context): # The name of this file and this function should match the handler field in the memphis.yaml file in the following format <file name>.<function name>
return create_function(event, event_handler = event_handler)

def event_handler(msg_payload, msg_headers, inputs):
payload = str(msg_payload, 'utf-8')
as_json = json.loads(payload)
if as_json['check'] == False:
return None, None

return bytes(json.dumps(as_json), encoding='utf-8'), msg_headers
```

Lastly, if the user is using another data format like Protocol Buffers, the user may simply decode the `msg_payload` into that format instead of JSON. Assuming we have a .proto definition like this:
```proto
syntax = "proto3";
package protobuf_example;
message Message{
string data_field = 1;
}
```

We can decode this and get the data_field out like this:

```python
import json
import base64
from memphis.functions import create_function
import message_pb2

def handler(event, context): # The name of this file and this function should match the handler field in the memphis.yaml file in the following format <file name>.<function name>
return create_function(event, event_handler = event_handler)

def event_handler(msg_payload, msg_headers, inputs):
message = message_pb2.Message()
message.ParseFromString(base64.b64decode(encoded_str))

# Arbitrarily changing the data_field
message.data_field = "my new data"

# SerializeToString returns bytes, which is the type we want
return message.SerializeToString(), msg_headers
```
```
108 changes: 0 additions & 108 deletions memphis/functions.py

This file was deleted.

0 comments on commit 5abef07

Please sign in to comment.