-
Notifications
You must be signed in to change notification settings - Fork 159
RFC: Use RunTask API for attached runs
This is a proposal for moving attached runs (emp run without the -d flag) to use the ECS RunTask API, instead of using the Docker Daemon API directly. The benefits of this are that:
- It means containers started via an attached run can use ECS Task Roles.
- It normalizes how attached/detached containers are ran.
- Paves the way for the possibility of adding an
emp attachcommand.
Currently, "attached runs" are NOT implemented in the CloudFormation Scheduler, but rather in the Docker Scheduler.
When you run a command like emp run bash, Empire uses the Docker scheduler, connects to the specified Docker daemon, and uses the CreateContainer, StartContainer, and AttachToContainer API's.
I'm proposing that we instead just use the RunTask API (which detached runs already use), and use the Docker Daemon CreateExec/StartExec API to attach to the container on the given host, after ECS has started it. When using the RunTask API, it will start the container using a dummy command that runs forever. According to http://unix.stackexchange.com/questions/42901/how-to-do-nothing-forever-in-an-elegant-way, this is the best POSIX compliant way to block forever:
tail -f /dev/null
When a container is started via ECS, Empire will need to connect to the Docker Daemon on the host that ECS started the container on. Empire will open a new docker connection for each attached run.
Since ECS doesn't expose any information about the Docker container ID. Empire will use ListContainers to find containers where the com.amazonaws.ecs.task-arn label matches the started task, then create a new exec instance within this container. I believe that the RunTask API will return before the actual Docker container is running, so Empire will need to wait for the task to enter the RUNNING state before connecting.
When the exec instance is stopped, the StopTask API will be called.
At the moment, attached runs are ran by the Docker daemon specified at start up (--docker.socket/DOCKER_HOST). This is also the daemon that Empire uses to pull images, when extracting a Procfile.
A new flag would be added to allow you to specify some container instance constraints for where attached runs would be ran, so that you can limit attached runs to a specific host or set of hosts. By default, the zero value would let the ECS scheduler handle what instance to schedule the container on.
After this change, the DOCKER_HOST flag would only be used to tell Empire what docker daemon to use.
Empire should assume that that Docker Daemon is exposed over TLS when connecting to the container instance.
No container instance matching constraints
Since the previous implementation would just blindly start a container on the Docker host, resource constraints of the container instance were not taken into consideration. After this change, it's possible that starting an attached run could fail if there is no container instance matching the constraints of the task.
docker pull output won't appear
The docker pull will be done by the ECS agent, so no way to grab this.