Merge pull request #38 from hercules-ci/README-asciidoc
README: rearrange sections to go: pitch, install, usage
This commit is contained in:
Robert Hensing
GitHub
commit
871db29671
8 changed files with 275 additions and 217 deletions
274
README.asciidoc
Normal file
274
README.asciidoc
Normal file
|
|
@ -0,0 +1,274 @@
|
|||
== Introduction
|
||||
|
||||
Arion is a tool for building and running applications that
|
||||
consist of multiple docker containers using NixOS modules.
|
||||
It has special support for docker images that are built with Nix,
|
||||
for a smooth development experience and improved performance.
|
||||
|
||||
It is built on top of https://docs.docker.com/compose/overview/[Docker
|
||||
Compose], which implements the container orchestration functionality.
|
||||
|
||||
Instead of configuring the compositions in YAML files like
|
||||
`docker-compose.yaml`, Arion uses the https://nixos.org/nix/[Nix]
|
||||
language to declare the compositions. Because of this, Arion gives you
|
||||
the ability to declare your deployments, configuration and packaging
|
||||
in the same language. By replacing multiple tools with a single
|
||||
language, you decrease your mental load and you can more easily
|
||||
refactor and maintain your configurations.
|
||||
|
||||
Although Arion can be used as a Docker Compose with an improved
|
||||
configuration front end, there is more to be gained from integrating
|
||||
with Nix. In particular, the more structured approach of Nix compared
|
||||
to Dockerfiles allows the following:
|
||||
|
||||
* Build components of your image in *parallel, automatically*
|
||||
* *Share packages between images*, regardless of the order they were
|
||||
added
|
||||
* Improve performance by *skipping container
|
||||
image creation*
|
||||
* Work with *structured data instead of strings*,
|
||||
templates and a multitude of expression languages
|
||||
* Refactor across deployments, configuration and packaging
|
||||
|
||||
Arion allows to compose containers with different granuality:
|
||||
|
||||
* <<Minimal: Plain command using nixpkgs>>
|
||||
* <<NixOS: run only one systemd service>>
|
||||
* <<NixOS: run full OS>>
|
||||
* <<Docker image from DockerHub>>
|
||||
|
||||
== Installation
|
||||
|
||||
=== Nix
|
||||
|
||||
```bash
|
||||
$ nix-env -iA arion -f https://github.com/hercules-ci/arion/tarball/master
|
||||
```
|
||||
|
||||
=== NixOS
|
||||
|
||||
Add this module to your NixOS configuration:
|
||||
|
||||
```nix
|
||||
{ ... }: {
|
||||
environment.systemPackages = [ (import (builtins.fetchTarball https://github.com/hercules-ci/arion/tarball/master) {}) ];
|
||||
virtualisation.docker.enable = true;
|
||||
users.extraUsers.myuser.extraGroups = ["docker"];
|
||||
}
|
||||
```
|
||||
|
||||
////
|
||||
|
||||
== Not installing: use it in a project
|
||||
|
||||
TODO: describe: using nix-shell or in a script, building images as
|
||||
part of nix-build, pinning, see also todomvc-nix.
|
||||
|
||||
TODO: exposed Nix functions: arion.build, arion.eval (a bit of IFD)
|
||||
|
||||
|
||||
////
|
||||
|
||||
|
||||
== Usage
|
||||
|
||||
Arion is configured declaratively with two files:
|
||||
|
||||
=== arion-pkgs.nix
|
||||
|
||||
Arion needs `arion-pkgs.nix` to import nixpkgs, it's contents can be as simple as:
|
||||
|
||||
```nix
|
||||
import <nixpkgs> {}
|
||||
```
|
||||
|
||||
or more sophisticated (recommended) setup with https://github.com/nmattia/niv[Niv].
|
||||
|
||||
=== arion-compose.nix
|
||||
|
||||
Describe containers using NixOS-style modules. There are a few options:
|
||||
|
||||
==== Minimal: Plain command using nixpkgs
|
||||
|
||||
`examples/minimal/arion-compose.nix`:
|
||||
|
||||
```nix
|
||||
{ pkgs, ... }:
|
||||
{
|
||||
config.docker-compose.services = {
|
||||
|
||||
webserver = {
|
||||
service.useHostStore = true;
|
||||
service.command = [ "sh" "-c" ''
|
||||
cd "$$WEB_ROOT"
|
||||
${pkgs.python3}/bin/python -m http.server
|
||||
'' ];
|
||||
service.ports = [
|
||||
"8000:8000" # host:container
|
||||
];
|
||||
service.environment.WEB_ROOT = "${pkgs.nix.doc}/share/doc/nix/manual";
|
||||
};
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
```
|
||||
|
||||
==== NixOS: run only one systemd service
|
||||
|
||||
`examples/nixos-unit/arion-compose.nix`:
|
||||
|
||||
```nix
|
||||
|
||||
{
|
||||
docker-compose.services.webserver = { config, pkgs, ... }: {
|
||||
|
||||
nixos.configuration = {config, pkgs, ...}: {
|
||||
boot.isContainer = true;
|
||||
services.nginx.enable = true;
|
||||
services.nginx.virtualHosts.localhost.root = "${pkgs.nix.doc}/share/doc/nix/manual";
|
||||
system.build.run-nginx = pkgs.writeScript "run-nginx" ''
|
||||
#!${pkgs.bash}/bin/bash
|
||||
PATH='${config.systemd.services.nginx.environment.PATH}'
|
||||
echo nginx:x:${toString config.users.users.nginx.uid}:${toString config.users.groups.nginx.gid}:nginx web server user:/var/empty:/bin/sh >>/etc/passwd
|
||||
echo nginx:x:${toString config.users.groups.nginx.gid}:nginx >>/etc/group
|
||||
${config.systemd.services.nginx.runner}
|
||||
'';
|
||||
};
|
||||
service.command = [ config.nixos.build.run-nginx ];
|
||||
service.useHostStore = true;
|
||||
service.ports = [
|
||||
"8000:80" # host:container
|
||||
];
|
||||
};
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
==== NixOS: run full OS
|
||||
|
||||
`examples/full-nixos/arion-compose.nix`:
|
||||
|
||||
```nix
|
||||
{
|
||||
docker-compose.services.webserver = { pkgs, ... }: {
|
||||
nixos.useSystemd = true;
|
||||
nixos.configuration.boot.tmpOnTmpfs = true;
|
||||
nixos.configuration.services.nginx.enable = true;
|
||||
nixos.configuration.services.nginx.virtualHosts.localhost.root = "${pkgs.nix.doc}/share/doc/nix/manual";
|
||||
service.useHostStore = true;
|
||||
service.ports = [
|
||||
"8000:80" # host:container
|
||||
];
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
==== Docker image from DockerHub
|
||||
|
||||
```nix
|
||||
{
|
||||
docker-compose.services.postgres = {
|
||||
service.image = "postgres:10";
|
||||
service.volumes = [ "${toString ./.}/postgres-data:/var/lib/postgresql/data" ];
|
||||
service.environment.POSTGRES_PASSWORD = "mydefaultpass";
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
=== Run
|
||||
|
||||
Start containers and watch their logs:
|
||||
|
||||
```bash
|
||||
$ arion up -d
|
||||
$ arion logs -f
|
||||
```
|
||||
|
||||
You can go to `examples/*/` and run these commands to give it a quick try.
|
||||
|
||||
== A full featured Nix command example
|
||||
|
||||
To see how Arion can be used in a project, have a look at
|
||||
https://github.com/nix-community/todomvc-nix/tree/master/deploy/arion[todomvc-nix].
|
||||
|
||||
```bash
|
||||
$ git clone https://github.com/nix-community/todomvc-nix
|
||||
$ cd todomvc-nix/deploy/arion
|
||||
$ arion up
|
||||
```
|
||||
|
||||
== Project Status
|
||||
|
||||
This project was born out of a process supervision need for local
|
||||
development environments while working on
|
||||
https://www.hercules-ci.com[Hercules CI]. (It was also born out of
|
||||
ancient Greek deities disguised as horses. More on that later.)
|
||||
|
||||
If you do want to use Arion for production environments, you’ll probably
|
||||
want to either build normal container images or manage garbage
|
||||
collection roots if you control the deployment host. Neither scenario is
|
||||
made easier by arion at this time.
|
||||
|
||||
Arion has run successfully on Linux distributions other than NixOS, but we only perform CI for Arion on NixOS.
|
||||
|
||||
|
||||
== How it works
|
||||
|
||||
Arion is essentially a thin wrapper around Nix and docker-compose. When
|
||||
it runs, it does the following:
|
||||
|
||||
* Evaluate the configuration using Nix, producing a
|
||||
`docker-compose.yaml` and a garbage collection root
|
||||
* Invoke `docker-compose`
|
||||
* Clean up the garbage collection root
|
||||
|
||||
Most of the interesting stuff happens in Arion’s Nix expressions, where
|
||||
it runs the module system (known from NixOS) and provides the
|
||||
configuration that makes the Docker Compose file do the things it needs
|
||||
to do.
|
||||
|
||||
One of the more interesting built-in modules is the
|
||||
link:src/nix/modules/service/host-store.nix[host-store.nix module] which
|
||||
performs the bind mounts to make the host Nix store available in the
|
||||
container.
|
||||
|
||||
== FAQ
|
||||
|
||||
=== Do I need to use Hercules CI?
|
||||
|
||||
Nope, it’s just Nix and Docker Compose under the hood.
|
||||
|
||||
=== What about garbage collection?
|
||||
|
||||
Arion removes the need for garbage collecting docker images, delegating
|
||||
this task to Nix.
|
||||
|
||||
Arion creates a garbage collection root and cleans it up after
|
||||
completing the command. This means that `arion up` without `-d` is safe
|
||||
with respect to garbage collection. A deployment that is more serious
|
||||
than local development must leave a GC root on the deployment host. This
|
||||
use case is not supported as of now.
|
||||
|
||||
=== Why is my container not running latest code?
|
||||
|
||||
Restart it with `arion restart <name>` or if you've changed the image rebuild
|
||||
them using `arion up -d --always-recreate-deps <name>`.
|
||||
|
||||
=== What is messing with my environment variables?
|
||||
|
||||
Docker Compose performs its own environment variable substitution. This
|
||||
can be a little annoying in `services.command` for example. Either
|
||||
reference a script from `pkgs.writeScript` or escape the dollar sign as
|
||||
`$$`.
|
||||
|
||||
=== Why name it ``Arion``?
|
||||
|
||||
Arion comes from Greek mythology. Poseidon, the god of ~Docker~ the seas
|
||||
had his eye on Demeter. Demeter tried to trick him by disguising as a
|
||||
horse, but Poseidon saw through the deception and they had Arion.
|
||||
|
||||
So Arion is a super fast divine horse; the result of some weird mixing.
|
||||
Also it talks.
|
||||
|
||||
(And we feel morally obliged to name our stuff after Greek mythology)
|
||||
135
README.md
135
README.md
|
|
@ -1,135 +0,0 @@
|
|||
|
||||
# Run docker-compose without images with Nix
|
||||
|
||||
*Wait, what?*
|
||||
|
||||
With Arion you can fire up containers without creating images for each
|
||||
service. It can use a mostly empty image, and makes the host's Nix
|
||||
store available in the container, allowing the container to run
|
||||
programs without having to re-package them into a docker image.
|
||||
|
||||
Arion is configured using Nix with modules, like those in
|
||||
NixOS. Similar to `docker-compose` it can therefore combine
|
||||
configuration from multiple files. For managing the network and
|
||||
containers it delegates to the `docker-compose` command.
|
||||
|
||||
# Project Status
|
||||
|
||||
This project was born out of a process supervision need for local
|
||||
development environments while working
|
||||
on [Hercules CI](https://www.hercules-ci.com). (It was also born out
|
||||
of ancient Greek deities disguised as horses. More on that later.)
|
||||
|
||||
If you do want to use Arion for production environments, you'll
|
||||
probably want to either build normal container images or manage
|
||||
garbage collection roots if you control the deployment host. Neither
|
||||
scenario is made easier by arion at this time.
|
||||
|
||||
Support for other Linux than NixOS is untested.
|
||||
|
||||
# Install
|
||||
|
||||
Have [Nix](https://nixos.org/nix/) and Docker installed. Ubuntu users:
|
||||
make sure you've installed via `apt-get install docker.io`. The Docker snap
|
||||
is broken and apparently unsupported by Canonical.
|
||||
|
||||
git clone git@github.com:hercules-ci/arion.git
|
||||
cd arion
|
||||
nix-env -iA arion -f .
|
||||
|
||||
# Example `arion-compose.nix`
|
||||
|
||||
This Nix expression serves the Nix manual at host port 8000 when launched with `arion up`. It is a function from a package set (`pkgs`) to a configuration.
|
||||
|
||||
```nix
|
||||
{ pkgs, ... }:
|
||||
{
|
||||
config.docker-compose.services = {
|
||||
|
||||
webserver = {
|
||||
service.useHostStore = true;
|
||||
# service.depends_on = [ "backend" ];
|
||||
service.command = [ "sh" "-c" ''
|
||||
cd "$$WEB_ROOT"
|
||||
${pkgs.python3}/bin/python -m http.server
|
||||
'' ];
|
||||
service.ports = [
|
||||
"8000:8000" # host:container
|
||||
];
|
||||
service.environment.WEB_ROOT = "${pkgs.nix.doc}/share/doc/nix/manual";
|
||||
};
|
||||
|
||||
# backend = { ... }
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
The `pkgs` argument comes from a file called `arion-pkgs.nix`. It can be as simple as `import <nixpkgs> {}` to use the Nixpkgs from your `$NIX_PATH`, or you can use it to pin a specific Nixpkgs version.
|
||||
|
||||
# A full featured example
|
||||
|
||||
To see how Arion can be used in a project, have a look at [todomvc-nix](https://github.com/nix-community/todomvc-nix/tree/master/deploy/arion).
|
||||
|
||||
git clone git@github.com:nix-community/todomvc-nix.git
|
||||
cd todomvc-nix/deploy/arion
|
||||
arion up
|
||||
|
||||
# How it works
|
||||
|
||||
Arion is essentially a thin wrapper around Nix and docker-compose.
|
||||
When it runs, it does the following:
|
||||
|
||||
- Evaluate the configuration using Nix, producing a `docker-compose.yaml` and a garbage collection root
|
||||
- Invoke `docker-compose`
|
||||
- Clean up the garbage collection root
|
||||
|
||||
Most of the interesting stuff happens in Arion's Nix expressions,
|
||||
where it runs the module system (known from NixOS) and provides the configuration that makes the Docker Compose file do the things it needs to do.
|
||||
|
||||
One of the more interesting built-in modules is the [host-store.nix module](src/nix/modules/service/host-store.nix) which performs the bind mounts to make the host Nix store available in the container.
|
||||
|
||||
# FAQ
|
||||
|
||||
### Do I need to use Hercules CI?
|
||||
|
||||
Nope, it's just Nix and Docker Compose under the hood.
|
||||
|
||||
|
||||
### Does Arion support Docker images?
|
||||
|
||||
Yes, you can also specify a normal Docker image. For example:
|
||||
|
||||
postgres = {
|
||||
service.image = "postgres:10";
|
||||
service.volumes = [ "${toString ./.}/postgres-data:/var/lib/postgresql/data" ];
|
||||
service.environment.POSTGRES_PASSWORD = "mydefaultpass";
|
||||
};
|
||||
|
||||
### What about garbage collection?
|
||||
|
||||
Arion removes the need for garbage collecting docker images,
|
||||
delegating this task to Nix.
|
||||
|
||||
Arion creates a garbage collection root and cleans it up after
|
||||
completing the command. This means that `arion up` without `-d` is
|
||||
safe with respect to garbage collection. A deployment that is more
|
||||
serious than local development must leave a GC root on the deployment
|
||||
host. This use case is not supported as of now.
|
||||
|
||||
### What is messing with my environment variables?
|
||||
|
||||
Docker Compose performs its own environment variable
|
||||
substitution. This can be a little annoying in `services.command` for
|
||||
example. Either reference a script from `pkgs.writeScript` or escape
|
||||
the dollar sign as `$$`.
|
||||
|
||||
### Why "Arion"?
|
||||
|
||||
Arion comes from Greek mythology. Poseidon, the god of ~Docker~ the
|
||||
seas had his eye on Demeter. Demeter tried to trick him by disguising
|
||||
as a horse, but Poseidon saw through the deception and they had Arion.
|
||||
|
||||
So Arion is a super fast divine horse; the result of some weird
|
||||
mixing. Also it talks.
|
||||
|
||||
(And we feel morally obliged to name our stuff after Greek mythology)
|
||||
2
doc/manual/.gitignore
vendored
2
doc/manual/.gitignore
vendored
|
|
@ -1,5 +1,3 @@
|
|||
introduction.xml
|
||||
installation.xml
|
||||
manual.html
|
||||
options-composition.xml
|
||||
options-service.xml
|
||||
|
|
|
|||
|
|
@ -17,7 +17,7 @@ docbookxsl = http://docbook.sourceforge.net/release/xsl/current
|
|||
|
||||
all: manual.html
|
||||
|
||||
manual.html: manual.xml introduction.xml installation.xml options-composition.xml options-service.xml
|
||||
manual.html: manual.xml options-composition.xml options-service.xml
|
||||
$(xsltproc) --xinclude --stringparam profile.condition manual \
|
||||
$(docbookxsl)/profiling/profile.xsl manual.xml | \
|
||||
$(xsltproc) --output manual.html $(docbookxsl)/xhtml/docbook.xsl -
|
||||
|
|
|
|||
|
|
@ -1,39 +0,0 @@
|
|||
|
||||
= Installation
|
||||
|
||||
== Imperative installation, bleeding edge
|
||||
|
||||
```
|
||||
git clone git@github.com:hercules-ci/arion.git
|
||||
cd arion
|
||||
nix-env -iA arion -f .
|
||||
```
|
||||
|
||||
// TODO: replace the above with something like below
|
||||
////
|
||||
|
||||
== Not installing: use it in a project
|
||||
|
||||
TODO: describe: using nix-shell or in a script, building images as
|
||||
part of nix-build, pinning, see also todomvc-nix.
|
||||
|
||||
TODO: exposed Nix functions: arion.build, arion.eval (a bit of IFD)
|
||||
|
||||
== Mac or traditional Linux
|
||||
|
||||
```
|
||||
nix-env -iA nixpkgs.arion
|
||||
```
|
||||
|
||||
== NixOS
|
||||
|
||||
Add this module to your NixOS configuration:
|
||||
|
||||
```
|
||||
{ pkgs, ... } {
|
||||
virtualisation.docker.enable = true;
|
||||
environment.systemPackages = [ pkgs.arion ];
|
||||
}
|
||||
```
|
||||
|
||||
////
|
||||
|
|
@ -1,33 +0,0 @@
|
|||
|
||||
= Introduction
|
||||
|
||||
Arion is a tool for building and running applications that
|
||||
consist of multiple docker containers. It has special support
|
||||
for docker images that are built with Nix, for a smooth
|
||||
development experience and improved performance.
|
||||
|
||||
It is built on top of https://docs.docker.com/compose/overview/[Docker
|
||||
Compose], which implements the container orchestration functionality.
|
||||
|
||||
Instead of configuring the compositions in YAML files like
|
||||
`docker-compose.yaml`, Arion uses the https://nixos.org/nix/[Nix]
|
||||
language to declare the compositions. Because of this, Arion gives you
|
||||
the ability to declare your deployments, configuration and packaging
|
||||
in the same language. By replacing multiple tools with a single
|
||||
language, you decrease your mental load and you can more easily
|
||||
refactor and maintain your configurations.
|
||||
|
||||
Although Arion can be used as a Docker Compose with an improved
|
||||
configuration front end, there is more to be gained from integrating
|
||||
with Nix. In particular, the more structured approach of Nix compared
|
||||
to Dockerfiles allows the following:
|
||||
|
||||
* Build components of your image in _parallel, automatically_.
|
||||
* Share packages between images, regardless of the order they were
|
||||
added.
|
||||
* Improve performance by _skipping_ container
|
||||
image creation.
|
||||
* Work with _structured data_ instead of strings,
|
||||
templates and a multitude of expression languages.
|
||||
* Refactor across deployments, configuration and
|
||||
packaging.
|
||||
|
|
@ -16,9 +16,6 @@
|
|||
|
||||
</info>
|
||||
|
||||
<xi:include href="introduction.xml" xpointer="xpointer(/article)" />
|
||||
<xi:include href="installation.xml" xpointer="xpointer(/article)" />
|
||||
|
||||
<xi:include href="options-composition.xml" />
|
||||
<xi:include href="options-service.xml" />
|
||||
|
||||
|
|
|
|||
|
|
@ -4,7 +4,6 @@
|
|||
|
||||
webserver = {
|
||||
service.useHostStore = true;
|
||||
# service.depends_on = [ "backend" ];
|
||||
service.command = [ "sh" "-c" ''
|
||||
cd "$$WEB_ROOT"
|
||||
${pkgs.python3}/bin/python -m http.server
|
||||
|
|
@ -14,8 +13,5 @@
|
|||
];
|
||||
service.environment.WEB_ROOT = "${pkgs.nix.doc}/share/doc/nix/manual";
|
||||
};
|
||||
|
||||
# backend = { ... }
|
||||
};
|
||||
}
|
||||
|
||||
|
|
|
|||
Loading…
Reference in a new issue