diff --git a/README.md b/README.md index 523d86c..9cbf21f 100644 --- a/README.md +++ b/README.md @@ -9,9 +9,9 @@ If [available in Hex](https://hex.pm/docs/publish), the package can be installed by adding `divo` to your list of dependencies in `mix.exs`: ```elixir -def deps do +def deps() do [ - {:divo, "~> 1.0.2", only: [:dev, :integration], organization: "smartcolumbus_os"} + {:divo, "~> 1.1.0", only: [:dev, :integration], organization: "smartcolumbus_os"} ] end ``` @@ -23,9 +23,34 @@ be found at [https://hexdocs.pm/divo](https://hexdocs.pm/divo). ## Usage Define services in your mix configuration file (typically under a `config/integration.exs` file) -to define the dockerized service(s) you want to run as a dependency of your Elixir app. +to define the dockerized service(s) you want to run as a dependency of your Elixir app. Define divo config in one of three ways" +### Method 1 - Pre-existing definition from a behaviour-derived module stack +In your mix file, include the additional dependency +```elixir +def deps() do + [ + {:divo, "~> 1.1.0", only: [:dev, :integration], organization: "smartcolumbus_os"}, + {:divo_redis, "~> 0.1.0", only: [:dev, :integration], organization: "smartcolumbus_os"} + ] ``` +And in your environment config, include the imported dependency module(s) as a list of tuples along with any environment variables the stack takes as a keyword list +```elixir +config :myapp, + divo: [ + {DivoRedis, [initial_key: "myapp:secret"]} + ] +``` + +### Method 2 - Pre-existing definition from a supplied compose file on the file system +In your environment config, include the path to the yaml- or json-formatted compose file +```elixir +config :myapp, + divo: "test/support/docker-compose.yaml +``` + +### Method 3 - Define the custom compose file as an elixir map directly in your config +```elixir config :myapp, divo: [ kafka: %{ @@ -51,47 +76,19 @@ config :myapp, ] ``` -## Options - -These options can be added to a service definition's config to customize it. - -* `image` - the name of the docker image to be started. This is the only required key. - -* `env` - a keyword list of environment variables to be set in the started container. Each element is translated to a `--env=VARIABLE=VALUE` option in the run command. - -* `ports` - a list of ports to be exposed to the system. Each element is translated to a `--publish=LOCAL:REMOTE` option in the run command. - -* `volumes` - a list of tuples of the format `{local_volume, remote_volume}`. Each element is translated to a `--volume=LOCAL:REMOTE` option in the run command +## Compose File Definition -* `command` - a command to be run in the created container. Does not support piping `ls | grep logs` or additional commands `ls && cd ..` +Docker Compose instructions are passed to the `docker-compose` binary on your Docker engine host as either a yaml- or json-formatted document with maps defining the container services, networks, and volumes needed to run the services as an interconnected stack of components (as well as a compose file version). The keys in the underlying map structure generally have a one-to-one relationship to the various arguments available to the `docker run` command. -* `net` - a different service defined by Divo that this container needs to be linked to. Translated to `--network=container:APP_NAME-SERVICE_NAME` +For more details, see the full [docker compose documentation](https://docs.docker.com/compose/compose-file/) -* `additional_opts` - a list of strings representing extra options to be passed to `docker run`. This allows for options not explicitly supported by Divo to be used if needed. Any option defined in the (Docker Run)[https://docs.docker.com/engine/reference/commandline/run/] docs can be used. - -* `wait_for` - see [Wait For]() - - -## Wait For +## Divo Wait Sometimes services take a moment to start up and Elixir apps tend to start (and attempt to run their tests) -too quickly for their dependencies to be ready. For those situations, add the key `:wait_for` to the map -that defines each services that will need to be fully initialized before accepting interactions from your -service-under-test. The value of that key should be a map containing a log message to expect from the service -indicating it is ready to accept requests, a interval in seconds to wait between log parsing attempts, and a -number of retries to make the attempt. +too quickly for their dependencies to be ready. For those situations, add the key `:divo_wait` to the app config that defines a wait period in milliseconds and a maximum number of tries to check for the containerized services to be healthy before aborting. In order for the wait to hold execution for the containers to register as healthy with the Docker engine, a [healthcheck](https://docs.docker.com/compose/compose-file/#healthcheck) must be built into the Dockerfile for the image or defined in the compose file. -``` -... - kafka: %{ - image: ..., - env: [ - ... - ], - wait_for: %{ - log: "home", - dwell: 400, - max_retries: 10 - } - } +```elixir +config :myapp, + divo: "test/support/docker-compose.yaml", + divo_wait: [dwell: 700, max_tries: 50] ``` diff --git a/lib/divo/compose.ex b/lib/divo/compose.ex index 74d1809..ee2b502 100644 --- a/lib/divo/compose.ex +++ b/lib/divo/compose.ex @@ -12,6 +12,12 @@ defmodule Divo.Compose do require Logger alias Divo.{File, Helper, Validate} + @doc """ + Builds and/or validates the compose file and executes the `docker-compose up` + call to start the entirety of the defined stack or a subset of the services + defined in the stack based on supplying an optional list of service keys. + """ + @spec run(keyword()) :: none() def run(opts \\ []) do services = get_services(opts) @@ -21,10 +27,22 @@ defmodule Divo.Compose do await() end + @doc """ + Builds and/or validates the compose file and executes the `docker-compose stop` + call to stop the containerized services without removing the resources created + by the compose file. + """ + @spec stop() :: none() def stop() do execute("stop") end + @doc """ + Builds and/or validates the compose file and executes the `docker-compose down` + call to stop the containerized services and removes all resources created by + the compose file such as containers, networks, and volumes. + """ + @spec kill() :: none() def kill() do execute("down") end diff --git a/lib/divo/file.ex b/lib/divo/file.ex index f65cb0d..bd70c07 100644 --- a/lib/divo/file.ex +++ b/lib/divo/file.ex @@ -7,6 +7,12 @@ defmodule Divo.File do require Logger alias Divo.Helper + @doc """ + Returns the name of the compose file to run, either as a + pass-through from an existing compose file or the path of + the file dynamically created by divo. + """ + @spec file_name() :: String.t() def file_name() do app = Helper.fetch_name() @@ -16,16 +22,36 @@ defmodule Divo.File do end end + @doc """ + Passes through the file name when the compose file is + pre-existing and supplied via file system path or builds + and writes a dynamic compose file to a temp directory before + returning the path to that temp file. + """ + @spec ensure_file(String.t() | [tuple()] | map()) :: String.t() def ensure_file(app_config) when is_binary(app_config) do Logger.info("Using : #{app_config}") app_config end + def ensure_file(app_config) when is_list(app_config) do + file = file_name() + + Logger.info("Generating : #{file} from stack module") + + app_config + |> Divo.Stack.concat_compose() + |> Jason.encode!() + |> write(file) + + file + end + def ensure_file(app_config) when is_map(app_config) do file = file_name() - Logger.info("Generating : #{file}") + Logger.info("Generating : #{file} from map") app_config |> Jason.encode!() diff --git a/lib/divo/helper.ex b/lib/divo/helper.ex index b3d740a..68fb282 100644 --- a/lib/divo/helper.ex +++ b/lib/divo/helper.ex @@ -5,10 +5,20 @@ defmodule Divo.Helper do commands. """ + @doc """ + Returns the name of the calling app from the mix config. + """ + @spec fetch_name() :: atom() def fetch_name() do Mix.Project.config()[:app] end + @doc """ + Returns the configuration for divo from the environment config + exs file that defines the container services to run or the path + to the config given an existing compose file. + """ + @spec fetch_config() :: map() | String.t() | [tuple()] def fetch_config() do with {:ok, config} <- Application.fetch_env(fetch_name(), :divo) do config diff --git a/lib/divo/stack.ex b/lib/divo/stack.ex new file mode 100644 index 0000000..1828e51 --- /dev/null +++ b/lib/divo/stack.ex @@ -0,0 +1,39 @@ +defmodule Divo.Stack do + @moduledoc """ + Implements a behaviour for importing + predefined compose stacks from external + library complementary to divo for quickly + standing up well-defined services with + little variation in configuration. + """ + + @doc """ + Defines the behaviour that must be implemented to + supply configs from external modules to divo. The + configuration values are expected as a keyword list + of attributes specific to each module adopting the + behaviour. + """ + @callback gen_stack(keyword()) :: {atom(), map()} + + @doc """ + Iterates over modules supplied in the app :divo + config, calling the `gen_stack` function on each + module's implementation of the behavior, collecting + the resulting maps into a single map and passing + the accumulated result out for writing out the file. + """ + @spec concat_compose([tuple()]) :: map() + def concat_compose(config) do + compose_file = %{version: "3.4"} + + services = + config + |> Enum.map(fn {module, envars} -> + apply(module, :gen_stack, [envars]) + end) + |> Enum.reduce(%{}, fn service, acc -> Map.merge(service, acc) end) + + Map.put(compose_file, :services, services) + end +end diff --git a/lib/divo/validate.ex b/lib/divo/validate.ex index ac732f2..2e70f5f 100644 --- a/lib/divo/validate.ex +++ b/lib/divo/validate.ex @@ -6,6 +6,11 @@ defmodule Divo.Validate do """ require Logger + @doc """ + Wraps the function of `docker-compose` to validate the structure of + the compose file for correct format/syntax and required keys are supplied. + """ + @spec validate(binary()) :: none() def validate(file) do System.cmd("docker-compose", ["--file", file, "config"], stderr_to_stdout: true) |> case do diff --git a/mix.exs b/mix.exs index de4b8dd..98f3228 100644 --- a/mix.exs +++ b/mix.exs @@ -4,7 +4,7 @@ defmodule Divo.MixProject do def project do [ app: :divo, - version: "1.0.2", + version: "1.1.0", elixir: "~> 1.8", start_permanent: Mix.env() == :prod, deps: deps(), @@ -26,10 +26,10 @@ defmodule Divo.MixProject do [ {:credo, "~> 1.0", only: [:dev, :test], runtime: false}, {:jason, "~> 1.1"}, - {:placebo, "~> 1.2.1", only: [:dev, :test]}, + {:placebo, "~> 1.2", only: [:dev, :test]}, {:ex_doc, "~> 0.19", only: :dev}, - {:patiently, "~> 0.2.0"}, - {:temporary_env, "~> 2.0.1", only: [:dev, :test]} + {:patiently, "~> 0.2"}, + {:temporary_env, "~> 2.0", only: [:dev, :test]} ] end diff --git a/mix.lock b/mix.lock index 1d5d5db..e7c0560 100644 --- a/mix.lock +++ b/mix.lock @@ -1,7 +1,7 @@ %{ "bunt": {:hex, :bunt, "0.2.0", "951c6e801e8b1d2cbe58ebbd3e616a869061ddadcc4863d0a2182541acae9a38", [:mix], [], "hexpm"}, - "credo": {:hex, :credo, "1.0.2", "88bc918f215168bf6ce7070610a6173c45c82f32baa08bdfc80bf58df2d103b6", [:mix], [{:bunt, "~> 0.2.0", [hex: :bunt, repo: "hexpm", optional: false]}, {:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: false]}], "hexpm"}, - "earmark": {:hex, :earmark, "1.3.1", "73812f447f7a42358d3ba79283cfa3075a7580a3a2ed457616d6517ac3738cb9", [:mix], [], "hexpm"}, + "credo": {:hex, :credo, "1.0.4", "d2214d4cc88c07f54004ffd5a2a27408208841be5eca9f5a72ce9e8e835f7ede", [:mix], [{:bunt, "~> 0.2.0", [hex: :bunt, repo: "hexpm", optional: false]}, {:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: false]}], "hexpm"}, + "earmark": {:hex, :earmark, "1.3.2", "b840562ea3d67795ffbb5bd88940b1bed0ed9fa32834915125ea7d02e35888a5", [:mix], [], "hexpm"}, "ex_doc": {:hex, :ex_doc, "0.19.3", "3c7b0f02851f5fc13b040e8e925051452e41248f685e40250d7e40b07b9f8c10", [:mix], [{:earmark, "~> 1.2", [hex: :earmark, repo: "hexpm", optional: false]}, {:makeup_elixir, "~> 0.10", [hex: :makeup_elixir, repo: "hexpm", optional: false]}], "hexpm"}, "jason": {:hex, :jason, "1.1.2", "b03dedea67a99223a2eaf9f1264ce37154564de899fd3d8b9a21b1a6fd64afe7", [:mix], [{:decimal, "~> 1.0", [hex: :decimal, repo: "hexpm", optional: true]}], "hexpm"}, "makeup": {:hex, :makeup, "0.8.0", "9cf32aea71c7fe0a4b2e9246c2c4978f9070257e5c9ce6d4a28ec450a839b55f", [:mix], [{:nimble_parsec, "~> 0.5.0", [hex: :nimble_parsec, repo: "hexpm", optional: false]}], "hexpm"}, diff --git a/test/file_test.exs b/test/file_test.exs index d0fcb73..c895676 100644 --- a/test/file_test.exs +++ b/test/file_test.exs @@ -26,8 +26,38 @@ defmodule Divo.FileTest do test "generates compose file from app config" do allow(File.write!(any(), any()), return: :ok) allow(System.get_env("TMPDIR"), return: "/var/tmp/foo") - file = Divo.Helper.fetch_config() + config = Divo.Helper.fetch_config() - assert Divo.File.ensure_file(file) == "/var/tmp/foo/divo.compose" + assert Divo.File.ensure_file(config) == "/var/tmp/foo/divo.compose" + end + + test "generates compose file from a behaviour implementation" do + allow(File.write!(any(), any()), return: :ok) + allow(System.get_env("TMPDIR"), return: "/var/tmp/bar") + + services = [{DivoBarbaz, []}] + + TemporaryEnv.put :divo, :divo, services do + config = Divo.Helper.fetch_config() + + assert Divo.File.ensure_file(config) == "/var/tmp/bar/divo.compose" + end + end +end + +defmodule DivoBarbaz do + @behaviour Divo.Stack + + @impl Divo.Stack + def gen_stack(_envars) do + %{ + barbaz: %{ + image: "library/barbaz", + healthcheck: %{ + test: ["CMD-SHELL", "/bin/true || exit 1"] + }, + ports: ["2345:2345", "7777:7777"] + } + } end end diff --git a/test/stack_test.exs b/test/stack_test.exs new file mode 100644 index 0000000..2df7132 --- /dev/null +++ b/test/stack_test.exs @@ -0,0 +1,57 @@ +defmodule Divo.StackTest do + use ExUnit.Case + require TemporaryEnv + + test "behaviour returns the service definition" do + services = [ + {DivoFoobar, [db_password: "we-are-divo", db_name: "foobar-db", something: "else"]} + ] + + expected = %{ + version: "3.4", + services: %{ + foobar: %{ + image: "foobar:latest", + ports: ["8080:8080"], + command: ["/bin/server", "foreground"] + }, + db: %{ + image: "cooldb:5.0.9", + environment: [ + "PASSWORD=we-are-divo", + "DB=foobar-db" + ] + } + } + } + + actual = Divo.Stack.concat_compose(services) + + assert expected == actual + end +end + +defmodule DivoFoobar do + @behaviour Divo.Stack + + @impl Divo.Stack + def gen_stack(envars) do + password = envars[:db_password] + db_name = envars[:db_name] + + %{ + foobar: %{ + image: "foobar:latest", + ports: ["8080:8080"], + command: ["/bin/server", "foreground"] + }, + db: %{ + image: "cooldb:5.0.9", + environment: [ + "PASSWORD=#{password}", + "DB=#{db_name}" + ] + } + } + end +end