Dynamic Machine IP for clustered deployments on Fargate

[krainboltgreene]

So at my company we deploy elixir applications to Fargate, a subset of AWS's Elastic Container Service platform.

tl;dr on ecs: "container image + machine details" as the total interface for deploying to EC2 (the virtual server).
tl;dr on fargate: "ecs automated network + scaling mechanisms" as the total interface

Cloud Run is pretty much the same concept, but with more of the network/scaling details automated away based on the context of HTTP requests.

In both of these services one of the key points is that as requests come in the definition of a instance can be scaled up to meet demand. This creates a scenario where a machine, which elixir is running on, cannot know it's own IP address for clustering inherently. AWS's solution to this (and probably GCP's as well) is to have an HTTP endpoint defined as an environment variable that describes in detail the metadata of the instance running.

At my company this is how we get the machine ip for clustering:

# env.sh.eex
IP=`curl --silent ${ECS_CONTAINER_METADATA_URI} | jq -r .Networks[0].IPv4Addresses[0]`

Currently livebook doesn't seem to have a way to support this LIVEBOOK_NODE value as you need to know up front the machine IP according to the documentation. My gut instinct is to modify the dockerfile provided by the github repository to ship a custom env.sh.eex, but I figured I'd ask here first in case there's something I'm missing.

[josevalim]

If you can detect when you running inside Fargate and finding new nodes can be done with DNS queries, then we could add one entry here:

https://github.com/livebook-dev/livebook/blob/main/rel/server/env.sh.eex#L1

[krainboltgreene]

That's fair. Should I also assume a similar environment variable to FLY_APP_NAME for these environments? And if yes, do you have a preference on the name of this variable?

Final question, the way we do this at my company is piping the metadata response (json) into jq to get the data. Do you find this a sufficiently acceptable process?

[josevalim]

You would check for AWS_EXECUTION_ENV or ECS_CONTAINER_METADATA_URI. Could we use this instead to avoid jq? https://medium.com/@radhagayathripatel/retrieving-aws-ec2-instance-metadata-using-metadata-in-scripts-251bf18dbabf

[krainboltgreene]

So to be clear I do that, but in the fly chunk the project uses the FLY_APP_NAME as the basis for the DNS cluster as well as the LIVEBOOK_NODE name. There is no such thing for AWS platforms however we could require one (and default to livebook). For example if you deploy with LIVEBOOK_PROJECT_SLUG=lovelace then we would default to LIVEBOOK_CLUSTER="dns:lovelace.local and LIVEBOOK_NODE=lovelace-${image_tag}@${machine_ip}.

[krainboltgreene]

Regarding the link: Unfortunately as I found out that API style is deprecated, they're now on version v4:

• https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-metadata-endpoint-v4.html
• https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-metadata-endpoint-v4-response.html

(I guess AWS found simple GET requests without json to be too demanding)

[josevalim]

I would go with LIVEBOOK_CLUSTER="dns:livebook.local" and LIVEBOOK_NODE=livebook-${image_tag}@${machine_ip}. And then we tell people to set LIVEBOOK_CLUSTER if they want to.

Let's see what @jonatanklosko thinks about adding jq as a dependency. But last time I checked, it was 900kb.

[krainboltgreene]

There’s some alternatives to jq that are faster, but I’m not sure about smaller. I’ll start looking for some alternatives.

[krainboltgreene]

Moving discussion to #2797