README: rearrange sections to go: pitch, install, usage
Inline examples as we can't use imports due to https://github.com/github/markup/issues/1095
This commit is contained in:
Domen Kožar
parent
fbf57b4489
commit
f4948f2fde
5 changed files with 274 additions and 211 deletions
274
README.asciidoc
Normal file
274
README.asciidoc
Normal file
|
|
@ -0,0 +1,274 @@
|
||||||
|
== Introduction
|
||||||
|
|
||||||
|
Arion is a tool for building and running applications that
|
||||||
|
consist of multiple docker containers using NixOS modules.
|
||||||
|
It has special support for docker images that are built with Nix,
|
||||||
|
for a smooth development experience and improved performance.
|
||||||
|
|
||||||
|
It is built on top of https://docs.docker.com/compose/overview/[Docker
|
||||||
|
Compose], which implements the container orchestration functionality.
|
||||||
|
|
||||||
|
Instead of configuring the compositions in YAML files like
|
||||||
|
`docker-compose.yaml`, Arion uses the https://nixos.org/nix/[Nix]
|
||||||
|
language to declare the compositions. Because of this, Arion gives you
|
||||||
|
the ability to declare your deployments, configuration and packaging
|
||||||
|
in the same language. By replacing multiple tools with a single
|
||||||
|
language, you decrease your mental load and you can more easily
|
||||||
|
refactor and maintain your configurations.
|
||||||
|
|
||||||
|
Although Arion can be used as a Docker Compose with an improved
|
||||||
|
configuration front end, there is more to be gained from integrating
|
||||||
|
with Nix. In particular, the more structured approach of Nix compared
|
||||||
|
to Dockerfiles allows the following:
|
||||||
|
|
||||||
|
* Build components of your image in *parallel, automatically*
|
||||||
|
* *Share packages between images*, regardless of the order they were
|
||||||
|
added
|
||||||
|
* Improve performance by *skipping container
|
||||||
|
image creation*
|
||||||
|
* Work with *structured data instead of strings*,
|
||||||
|
templates and a multitude of expression languages
|
||||||
|
* Refactor across deployments, configuration and packaging
|
||||||
|
|
||||||
|
Arion allows to compose containers with different granuality:
|
||||||
|
|
||||||
|
* <<Minimal: Plain command using nixpkgs>>
|
||||||
|
* <<NixOS: run only one systemd service>>
|
||||||
|
* <<NixOS: run full OS>>
|
||||||
|
* <<Docker image from DockerHub>>
|
||||||
|
|
||||||
|
== Installation
|
||||||
|
|
||||||
|
=== Nix
|
||||||
|
|
||||||
|
```bash
|
||||||
|
$ nix-env -iA arion -f https://github.com/hercules-ci/arion/tarball/master
|
||||||
|
```
|
||||||
|
|
||||||
|
=== NixOS
|
||||||
|
|
||||||
|
Add this module to your NixOS configuration:
|
||||||
|
|
||||||
|
```nix
|
||||||
|
{ ... }: {
|
||||||
|
environment.systemPackages = [ (import (builtins.fetchTarball https://github.com/hercules-ci/arion/tarball/master) {}) ];
|
||||||
|
virtualisation.docker.enable = true;
|
||||||
|
users.extraUsers.myuser.extraGroups = ["docker"];
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
////
|
||||||
|
|
||||||
|
== Not installing: use it in a project
|
||||||
|
|
||||||
|
TODO: describe: using nix-shell or in a script, building images as
|
||||||
|
part of nix-build, pinning, see also todomvc-nix.
|
||||||
|
|
||||||
|
TODO: exposed Nix functions: arion.build, arion.eval (a bit of IFD)
|
||||||
|
|
||||||
|
|
||||||
|
////
|
||||||
|
|
||||||
|
|
||||||
|
== Usage
|
||||||
|
|
||||||
|
Arion is configured declaratively with two files:
|
||||||
|
|
||||||
|
=== arion-pkgs.nix
|
||||||
|
|
||||||
|
Arion needs `arion-pkgs.nix` to import nixpkgs, it's contents can be as simple as:
|
||||||
|
|
||||||
|
```nix
|
||||||
|
import <nixpkgs> {}
|
||||||
|
```
|
||||||
|
|
||||||
|
or more sophisticated (recommended) setup with https://github.com/nmattia/niv[Niv].
|
||||||
|
|
||||||
|
=== arion-compose.nix
|
||||||
|
|
||||||
|
Describe containers using NixOS modules. There are a few options:
|
||||||
|
|
||||||
|
==== Minimal: Plain command using nixpkgs
|
||||||
|
|
||||||
|
`examples/minimal/arion-compose.nix`:
|
||||||
|
|
||||||
|
```nix
|
||||||
|
{ pkgs, ... }:
|
||||||
|
{
|
||||||
|
config.docker-compose.services = {
|
||||||
|
|
||||||
|
webserver = {
|
||||||
|
service.useHostStore = true;
|
||||||
|
service.command = [ "sh" "-c" ''
|
||||||
|
cd "$$WEB_ROOT"
|
||||||
|
${pkgs.python3}/bin/python -m http.server
|
||||||
|
'' ];
|
||||||
|
service.ports = [
|
||||||
|
"8000:8000" # host:container
|
||||||
|
];
|
||||||
|
service.environment.WEB_ROOT = "${pkgs.nix.doc}/share/doc/nix/manual";
|
||||||
|
};
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
```
|
||||||
|
|
||||||
|
==== NixOS: run only one systemd service
|
||||||
|
|
||||||
|
`examples/nixos-unit/arion-compose.nix`:
|
||||||
|
|
||||||
|
```nix
|
||||||
|
|
||||||
|
{
|
||||||
|
docker-compose.services.webserver = { config, pkgs, ... }: {
|
||||||
|
|
||||||
|
nixos.configuration = {config, pkgs, ...}: {
|
||||||
|
boot.isContainer = true;
|
||||||
|
services.nginx.enable = true;
|
||||||
|
services.nginx.virtualHosts.localhost.root = "${pkgs.nix.doc}/share/doc/nix/manual";
|
||||||
|
system.build.run-nginx = pkgs.writeScript "run-nginx" ''
|
||||||
|
#!${pkgs.bash}/bin/bash
|
||||||
|
PATH='${config.systemd.services.nginx.environment.PATH}'
|
||||||
|
echo nginx:x:${toString config.users.users.nginx.uid}:${toString config.users.groups.nginx.gid}:nginx web server user:/var/empty:/bin/sh >>/etc/passwd
|
||||||
|
echo nginx:x:${toString config.users.groups.nginx.gid}:nginx >>/etc/group
|
||||||
|
${config.systemd.services.nginx.runner}
|
||||||
|
'';
|
||||||
|
};
|
||||||
|
service.command = [ config.nixos.build.run-nginx ];
|
||||||
|
service.useHostStore = true;
|
||||||
|
service.ports = [
|
||||||
|
"8000:80" # host:container
|
||||||
|
];
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
```
|
||||||
|
|
||||||
|
==== NixOS: run full OS
|
||||||
|
|
||||||
|
`examples/full-nixos/arion-compose.nix`:
|
||||||
|
|
||||||
|
```nix
|
||||||
|
{
|
||||||
|
docker-compose.services.webserver = { pkgs, ... }: {
|
||||||
|
nixos.useSystemd = true;
|
||||||
|
nixos.configuration.boot.tmpOnTmpfs = true;
|
||||||
|
nixos.configuration.services.nginx.enable = true;
|
||||||
|
nixos.configuration.services.nginx.virtualHosts.localhost.root = "${pkgs.nix.doc}/share/doc/nix/manual";
|
||||||
|
service.useHostStore = true;
|
||||||
|
service.ports = [
|
||||||
|
"8000:80" # host:container
|
||||||
|
];
|
||||||
|
};
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
==== Docker image from DockerHub
|
||||||
|
|
||||||
|
```nix
|
||||||
|
{
|
||||||
|
docker-compose.services.postgres = {
|
||||||
|
service.image = "postgres:10";
|
||||||
|
service.volumes = [ "${toString ./.}/postgres-data:/var/lib/postgresql/data" ];
|
||||||
|
service.environment.POSTGRES_PASSWORD = "mydefaultpass";
|
||||||
|
};
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
=== Run
|
||||||
|
|
||||||
|
Start containers and watch their logs:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
$ arion up -d
|
||||||
|
$ arion logs -f
|
||||||
|
```
|
||||||
|
|
||||||
|
You can go to `examples/*/` and run these commands to give it a quick try.
|
||||||
|
|
||||||
|
== A full featured Nix command example
|
||||||
|
|
||||||
|
To see how Arion can be used in a project, have a look at
|
||||||
|
https://github.com/nix-community/todomvc-nix/tree/master/deploy/arion[todomvc-nix].
|
||||||
|
|
||||||
|
```bash
|
||||||
|
$ git clone https://github.com/nix-community/todomvc-nix
|
||||||
|
$ cd todomvc-nix/deploy/arion
|
||||||
|
$ arion up
|
||||||
|
```
|
||||||
|
|
||||||
|
== Project Status
|
||||||
|
|
||||||
|
This project was born out of a process supervision need for local
|
||||||
|
development environments while working on
|
||||||
|
https://www.hercules-ci.com[Hercules CI]. (It was also born out of
|
||||||
|
ancient Greek deities disguised as horses. More on that later.)
|
||||||
|
|
||||||
|
If you do want to use Arion for production environments, you’ll probably
|
||||||
|
want to either build normal container images or manage garbage
|
||||||
|
collection roots if you control the deployment host. Neither scenario is
|
||||||
|
made easier by arion at this time.
|
||||||
|
|
||||||
|
Support for other Linux than NixOS is untested.
|
||||||
|
|
||||||
|
|
||||||
|
== How it works
|
||||||
|
|
||||||
|
Arion is essentially a thin wrapper around Nix and docker-compose. When
|
||||||
|
it runs, it does the following:
|
||||||
|
|
||||||
|
* Evaluate the configuration using Nix, producing a
|
||||||
|
`docker-compose.yaml` and a garbage collection root
|
||||||
|
* Invoke `docker-compose`
|
||||||
|
* Clean up the garbage collection root
|
||||||
|
|
||||||
|
Most of the interesting stuff happens in Arion’s Nix expressions, where
|
||||||
|
it runs the module system (known from NixOS) and provides the
|
||||||
|
configuration that makes the Docker Compose file do the things it needs
|
||||||
|
to do.
|
||||||
|
|
||||||
|
One of the more interesting built-in modules is the
|
||||||
|
link:src/nix/modules/service/host-store.nix[host-store.nix module] which
|
||||||
|
performs the bind mounts to make the host Nix store available in the
|
||||||
|
container.
|
||||||
|
|
||||||
|
== FAQ
|
||||||
|
|
||||||
|
=== Do I need to use Hercules CI?
|
||||||
|
|
||||||
|
Nope, it’s just Nix and Docker Compose under the hood.
|
||||||
|
|
||||||
|
=== What about garbage collection?
|
||||||
|
|
||||||
|
Arion removes the need for garbage collecting docker images, delegating
|
||||||
|
this task to Nix.
|
||||||
|
|
||||||
|
Arion creates a garbage collection root and cleans it up after
|
||||||
|
completing the command. This means that `arion up` without `-d` is safe
|
||||||
|
with respect to garbage collection. A deployment that is more serious
|
||||||
|
than local development must leave a GC root on the deployment host. This
|
||||||
|
use case is not supported as of now.
|
||||||
|
|
||||||
|
=== Why is my container not running latest code?
|
||||||
|
|
||||||
|
Restart it with `arion restart <name>` or if you've changed the image rebuild
|
||||||
|
them using `arion up -d --always-recreate-deps <name>`.
|
||||||
|
|
||||||
|
=== What is messing with my environment variables?
|
||||||
|
|
||||||
|
Docker Compose performs its own environment variable substitution. This
|
||||||
|
can be a little annoying in `services.command` for example. Either
|
||||||
|
reference a script from `pkgs.writeScript` or escape the dollar sign as
|
||||||
|
`$$`.
|
||||||
|
|
||||||
|
=== Why name it ``Arion``?
|
||||||
|
|
||||||
|
Arion comes from Greek mythology. Poseidon, the god of ~Docker~ the seas
|
||||||
|
had his eye on Demeter. Demeter tried to trick him by disguising as a
|
||||||
|
horse, but Poseidon saw through the deception and they had Arion.
|
||||||
|
|
||||||
|
So Arion is a super fast divine horse; the result of some weird mixing.
|
||||||
|
Also it talks.
|
||||||
|
|
||||||
|
(And we feel morally obliged to name our stuff after Greek mythology)
|
||||||
135
README.md
135
README.md
|
|
@ -1,135 +0,0 @@
|
||||||
|
|
||||||
# Run docker-compose without images with Nix
|
|
||||||
|
|
||||||
*Wait, what?*
|
|
||||||
|
|
||||||
With Arion you can fire up containers without creating images for each
|
|
||||||
service. It can use a mostly empty image, and makes the host's Nix
|
|
||||||
store available in the container, allowing the container to run
|
|
||||||
programs without having to re-package them into a docker image.
|
|
||||||
|
|
||||||
Arion is configured using Nix with modules, like those in
|
|
||||||
NixOS. Similar to `docker-compose` it can therefore combine
|
|
||||||
configuration from multiple files. For managing the network and
|
|
||||||
containers it delegates to the `docker-compose` command.
|
|
||||||
|
|
||||||
# Project Status
|
|
||||||
|
|
||||||
This project was born out of a process supervision need for local
|
|
||||||
development environments while working
|
|
||||||
on [Hercules CI](https://www.hercules-ci.com). (It was also born out
|
|
||||||
of ancient Greek deities disguised as horses. More on that later.)
|
|
||||||
|
|
||||||
If you do want to use Arion for production environments, you'll
|
|
||||||
probably want to either build normal container images or manage
|
|
||||||
garbage collection roots if you control the deployment host. Neither
|
|
||||||
scenario is made easier by arion at this time.
|
|
||||||
|
|
||||||
Support for other Linux than NixOS is untested.
|
|
||||||
|
|
||||||
# Install
|
|
||||||
|
|
||||||
Have [Nix](https://nixos.org/nix/) and Docker installed. Ubuntu users:
|
|
||||||
make sure you've installed via `apt-get install docker.io`. The Docker snap
|
|
||||||
is broken and apparently unsupported by Canonical.
|
|
||||||
|
|
||||||
git clone git@github.com:hercules-ci/arion.git
|
|
||||||
cd arion
|
|
||||||
nix-env -iA arion -f .
|
|
||||||
|
|
||||||
# Example `arion-compose.nix`
|
|
||||||
|
|
||||||
This Nix expression serves the Nix manual at host port 8000 when launched with `arion up`. It is a function from a package set (`pkgs`) to a configuration.
|
|
||||||
|
|
||||||
```nix
|
|
||||||
{ pkgs, ... }:
|
|
||||||
{
|
|
||||||
config.docker-compose.services = {
|
|
||||||
|
|
||||||
webserver = {
|
|
||||||
service.useHostStore = true;
|
|
||||||
# service.depends_on = [ "backend" ];
|
|
||||||
service.command = [ "sh" "-c" ''
|
|
||||||
cd "$$WEB_ROOT"
|
|
||||||
${pkgs.python3}/bin/python -m http.server
|
|
||||||
'' ];
|
|
||||||
service.ports = [
|
|
||||||
"8000:8000" # host:container
|
|
||||||
];
|
|
||||||
service.environment.WEB_ROOT = "${pkgs.nix.doc}/share/doc/nix/manual";
|
|
||||||
};
|
|
||||||
|
|
||||||
# backend = { ... }
|
|
||||||
};
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
The `pkgs` argument comes from a file called `arion-pkgs.nix`. It can be as simple as `import <nixpkgs> {}` to use the Nixpkgs from your `$NIX_PATH`, or you can use it to pin a specific Nixpkgs version.
|
|
||||||
|
|
||||||
# A full featured example
|
|
||||||
|
|
||||||
To see how Arion can be used in a project, have a look at [todomvc-nix](https://github.com/nix-community/todomvc-nix/tree/master/deploy/arion).
|
|
||||||
|
|
||||||
git clone git@github.com:nix-community/todomvc-nix.git
|
|
||||||
cd todomvc-nix/deploy/arion
|
|
||||||
arion up
|
|
||||||
|
|
||||||
# How it works
|
|
||||||
|
|
||||||
Arion is essentially a thin wrapper around Nix and docker-compose.
|
|
||||||
When it runs, it does the following:
|
|
||||||
|
|
||||||
- Evaluate the configuration using Nix, producing a `docker-compose.yaml` and a garbage collection root
|
|
||||||
- Invoke `docker-compose`
|
|
||||||
- Clean up the garbage collection root
|
|
||||||
|
|
||||||
Most of the interesting stuff happens in Arion's Nix expressions,
|
|
||||||
where it runs the module system (known from NixOS) and provides the configuration that makes the Docker Compose file do the things it needs to do.
|
|
||||||
|
|
||||||
One of the more interesting built-in modules is the [host-store.nix module](src/nix/modules/service/host-store.nix) which performs the bind mounts to make the host Nix store available in the container.
|
|
||||||
|
|
||||||
# FAQ
|
|
||||||
|
|
||||||
### Do I need to use Hercules CI?
|
|
||||||
|
|
||||||
Nope, it's just Nix and Docker Compose under the hood.
|
|
||||||
|
|
||||||
|
|
||||||
### Does Arion support Docker images?
|
|
||||||
|
|
||||||
Yes, you can also specify a normal Docker image. For example:
|
|
||||||
|
|
||||||
postgres = {
|
|
||||||
service.image = "postgres:10";
|
|
||||||
service.volumes = [ "${toString ./.}/postgres-data:/var/lib/postgresql/data" ];
|
|
||||||
service.environment.POSTGRES_PASSWORD = "mydefaultpass";
|
|
||||||
};
|
|
||||||
|
|
||||||
### What about garbage collection?
|
|
||||||
|
|
||||||
Arion removes the need for garbage collecting docker images,
|
|
||||||
delegating this task to Nix.
|
|
||||||
|
|
||||||
Arion creates a garbage collection root and cleans it up after
|
|
||||||
completing the command. This means that `arion up` without `-d` is
|
|
||||||
safe with respect to garbage collection. A deployment that is more
|
|
||||||
serious than local development must leave a GC root on the deployment
|
|
||||||
host. This use case is not supported as of now.
|
|
||||||
|
|
||||||
### What is messing with my environment variables?
|
|
||||||
|
|
||||||
Docker Compose performs its own environment variable
|
|
||||||
substitution. This can be a little annoying in `services.command` for
|
|
||||||
example. Either reference a script from `pkgs.writeScript` or escape
|
|
||||||
the dollar sign as `$$`.
|
|
||||||
|
|
||||||
### Why "Arion"?
|
|
||||||
|
|
||||||
Arion comes from Greek mythology. Poseidon, the god of ~Docker~ the
|
|
||||||
seas had his eye on Demeter. Demeter tried to trick him by disguising
|
|
||||||
as a horse, but Poseidon saw through the deception and they had Arion.
|
|
||||||
|
|
||||||
So Arion is a super fast divine horse; the result of some weird
|
|
||||||
mixing. Also it talks.
|
|
||||||
|
|
||||||
(And we feel morally obliged to name our stuff after Greek mythology)
|
|
||||||
|
|
@ -1,39 +0,0 @@
|
||||||
|
|
||||||
= Installation
|
|
||||||
|
|
||||||
== Imperative installation, bleeding edge
|
|
||||||
|
|
||||||
```
|
|
||||||
git clone git@github.com:hercules-ci/arion.git
|
|
||||||
cd arion
|
|
||||||
nix-env -iA arion -f .
|
|
||||||
```
|
|
||||||
|
|
||||||
// TODO: replace the above with something like below
|
|
||||||
////
|
|
||||||
|
|
||||||
== Not installing: use it in a project
|
|
||||||
|
|
||||||
TODO: describe: using nix-shell or in a script, building images as
|
|
||||||
part of nix-build, pinning, see also todomvc-nix.
|
|
||||||
|
|
||||||
TODO: exposed Nix functions: arion.build, arion.eval (a bit of IFD)
|
|
||||||
|
|
||||||
== Mac or traditional Linux
|
|
||||||
|
|
||||||
```
|
|
||||||
nix-env -iA nixpkgs.arion
|
|
||||||
```
|
|
||||||
|
|
||||||
== NixOS
|
|
||||||
|
|
||||||
Add this module to your NixOS configuration:
|
|
||||||
|
|
||||||
```
|
|
||||||
{ pkgs, ... } {
|
|
||||||
virtualisation.docker.enable = true;
|
|
||||||
environment.systemPackages = [ pkgs.arion ];
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
////
|
|
||||||
|
|
@ -1,33 +0,0 @@
|
||||||
|
|
||||||
= Introduction
|
|
||||||
|
|
||||||
Arion is a tool for building and running applications that
|
|
||||||
consist of multiple docker containers. It has special support
|
|
||||||
for docker images that are built with Nix, for a smooth
|
|
||||||
development experience and improved performance.
|
|
||||||
|
|
||||||
It is built on top of https://docs.docker.com/compose/overview/[Docker
|
|
||||||
Compose], which implements the container orchestration functionality.
|
|
||||||
|
|
||||||
Instead of configuring the compositions in YAML files like
|
|
||||||
`docker-compose.yaml`, Arion uses the https://nixos.org/nix/[Nix]
|
|
||||||
language to declare the compositions. Because of this, Arion gives you
|
|
||||||
the ability to declare your deployments, configuration and packaging
|
|
||||||
in the same language. By replacing multiple tools with a single
|
|
||||||
language, you decrease your mental load and you can more easily
|
|
||||||
refactor and maintain your configurations.
|
|
||||||
|
|
||||||
Although Arion can be used as a Docker Compose with an improved
|
|
||||||
configuration front end, there is more to be gained from integrating
|
|
||||||
with Nix. In particular, the more structured approach of Nix compared
|
|
||||||
to Dockerfiles allows the following:
|
|
||||||
|
|
||||||
* Build components of your image in _parallel, automatically_.
|
|
||||||
* Share packages between images, regardless of the order they were
|
|
||||||
added.
|
|
||||||
* Improve performance by _skipping_ container
|
|
||||||
image creation.
|
|
||||||
* Work with _structured data_ instead of strings,
|
|
||||||
templates and a multitude of expression languages.
|
|
||||||
* Refactor across deployments, configuration and
|
|
||||||
packaging.
|
|
||||||
|
|
@ -4,7 +4,6 @@
|
||||||
|
|
||||||
webserver = {
|
webserver = {
|
||||||
service.useHostStore = true;
|
service.useHostStore = true;
|
||||||
# service.depends_on = [ "backend" ];
|
|
||||||
service.command = [ "sh" "-c" ''
|
service.command = [ "sh" "-c" ''
|
||||||
cd "$$WEB_ROOT"
|
cd "$$WEB_ROOT"
|
||||||
${pkgs.python3}/bin/python -m http.server
|
${pkgs.python3}/bin/python -m http.server
|
||||||
|
|
@ -14,8 +13,5 @@
|
||||||
];
|
];
|
||||||
service.environment.WEB_ROOT = "${pkgs.nix.doc}/share/doc/nix/manual";
|
service.environment.WEB_ROOT = "${pkgs.nix.doc}/share/doc/nix/manual";
|
||||||
};
|
};
|
||||||
|
|
||||||
# backend = { ... }
|
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue