Compare commits

..

1 commit

Author SHA1 Message Date
Domen Kožar
17d8091b6b
Add ngrok service 2019-06-07 18:06:21 +02:00
121 changed files with 1537 additions and 3781 deletions

7
.envrc
View file

@ -1,7 +0,0 @@
HOST_XDG_DATA_DIRS="${XDG_DATA_DIRS:-}"
eval "$(lorri direnv)"
export XDG_DATA_DIRS="${XDG_DATA_DIRS}:${HOST_XDG_DATA_DIRS}"
# Use system PKI
unset SSL_CERT_FILE
unset NIX_SSL_CERT_FILE

7
.gitignore vendored
View file

@ -1,9 +1,2 @@
result result
result-* result-*
dist/
dist-newstyle/
cabal.project.local
*.swp

View file

@ -1,100 +0,0 @@
# Revision history for Arion
## 0.2.1.0 -- 2023-07-26
### Added
* `service.networks` now supports attribute set values with various options, thanks to @pedorich-n.
* `docker-compose.volumes` can now be specified in multiple modules, thanks to @qaifshaikh.
* `image.fakeRootCommands` for making modifications to the image that aren't "add a link farm".
### Fixed
* Regular maintenance fixes, including one by olebedev
## 0.2.0.0 -- 2022-12-02
### BREAKING
* The `project.name` option is now mandatory for projects that aren't deployed with the NixOS module.
* The NixOS module now sets the default network name to the project name (commonly referred to as `<name>` in the option path).
If this is not desired, for instance if you need the projects to be on the same network, set `networks.default.name` in each of them.
* The NixOS module now sets the default project name. You can still set your own value with the `project.name` option.
If you did not set one, docker compose heuristically determined the name to be `store`, so you may want to set `project.name = "store"` or prepare to rename the network manually.
### Removed
- NixOS 20.09 support. Its docker-compose does not support the
`networks.<name>.name` option, which is important in later versions.
A newer, bundled docker compose may work there, but for now the decision
is to drop this legacy version.
### Changed
* Healthcheck-based dependencies in `service.depends_on`.
### Added
* Support `service.healthcheck` for defining custom healthchecks.
* Arion now declares a `networks.default` by default, with `name` set to
`project.name`. This improves compatibility with container runtimes by
copying pre-existing behavior. Most users will want to keep using this
behavior, but it can be disabled with `enableDefaultNetwork`.
## 0.1.3.0 -- 2020-05-03
### Changed
* `useHostStore` now uses an image derived from the `image.*` options. You may
need to enable `enableRecommendedContents` because with this change, files
like `/bin/sh` aren't added by default anymore.
* Drop obsolete NixOS 19.03, 19.09 and 20.03 from CI.
### Added
* NixOS-based containers can now run on Podman when it is configured to provide a docker socket. See the [installation docs](https://docs.hercules-ci.com/arion/#_nixos).
* Support `service.dns`, for overriding the DNS servers used by containers.
* Support `service.labels`, which is useful for autodiscovery among other things.
* Add a tested example for Traefik with label-based routing.
* Add a `flake.nix` and an experimental flake example
* Add a warning when systemd `DynamicUser` is used but not available to the
container.
* CI with NixOS 21.05
## 0.1.2.0 -- 2020-03-05
* Support use of prebuilt `docker-compose.yaml`.
Separates build and execution without duplicating evaluation.
* Avoid storing tarballs (wasting store space) by using
`dockerTools.streamLayeredImage` if available.
* Project name is now configurable via the `project.name` option
* Support --no-ansi, --compatibility, --log-level options
## 0.1.1.1 -- 2020-03-20
* Fix ambiguous import of `lines`
* Improve base version constraint
* Fix warnings
## 0.1.1.0 -- 2020-03-19
* Support Nixpkgs 20.03
* Fixes for macOS
## 0.1.0.0 -- 2019-10-04
* First released version. Released on an unsuspecting world.

View file

@ -1,31 +0,0 @@
# Hacking on the modules
## Easiest option
The module system does not distinguish between modules and configurations.
This mean you can prototype any feature by factoring out functionality in a real-world project.
## Changing the built-in modules
If your change is not just an addition or if it's better implemented by refactoring, you'll want to fork and edit arion sources directly.
For a fast iteration cycle (but possibly outdated arion command logic):
~/src/arion/run-arion-quick up -d
To update the arion command logic on the next run
rm ~/src/arion/result-run-arion-quick
# Hacking on the arion command
The arion command is written in Haskell. Anyone can make small changes to the code.
Experience with Haskell tooling is not required. You can use the nixified scripts in the root of the repo for common tasks.
- `build` or `live-check` for typechecking
- `live-unit-tests` (only the test suite is "live" though)
- `repl` for a Haskell REPL
- `run-arion` to run an incrementally built arion
- `run-arion-via-nix` to run a nix-built arion
- ~~`run-arion-quick`~~ *not for command hacking;* use stale command. See previous section.

View file

@ -187,7 +187,7 @@
same "printed page" as the copyright notice for easier same "printed page" as the copyright notice for easier
identification within third-party archives. identification within third-party archives.
Copyright 2019 Hercules Labs OÜ Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License"); Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License. you may not use this file except in compliance with the License.

View file

@ -1,7 +1,274 @@
== Introduction
Arion is a tool for building and running applications that Arion is a tool for building and running applications that
consist of multiple docker containers using NixOS modules. consist of multiple docker containers using NixOS modules.
It has special support for docker images that are built with Nix, It has special support for docker images that are built with Nix,
for a smooth development experience and improved performance. for a smooth development experience and improved performance.
It is built on top of https://docs.docker.com/compose/overview/[Docker
Compose], which implements the container orchestration functionality.
# https://docs.hercules-ci.com/arion/[Intro and Documentation] Instead of configuring the compositions in YAML files like
`docker-compose.yaml`, Arion uses the https://nixos.org/nix/[Nix]
language to declare the compositions. Because of this, Arion gives you
the ability to declare your deployments, configuration and packaging
in the same language. By replacing multiple tools with a single
language, you decrease your mental load and you can more easily
refactor and maintain your configurations.
Although Arion can be used as a Docker Compose with an improved
configuration front end, there is more to be gained from integrating
with Nix. In particular, the more structured approach of Nix compared
to Dockerfiles allows the following:
* Build components of your image in *parallel, automatically*
* *Share packages between images*, regardless of the order they were
added
* Improve performance by *skipping container
image creation*
* Work with *structured data instead of strings*,
templates and a multitude of expression languages
* Refactor across deployments, configuration and packaging
Arion allows to compose containers with different granularity:
* <<Minimal: Plain command using nixpkgs>>
* <<NixOS: run only one systemd service>>
* <<NixOS: run full OS>>
* <<Docker image from DockerHub>>
== Installation
=== Nix
```bash
$ nix-env -iA arion -f https://github.com/hercules-ci/arion/tarball/master
```
=== NixOS
Add this module to your NixOS configuration:
```nix
{ ... }: {
environment.systemPackages = [ (import (builtins.fetchTarball https://github.com/hercules-ci/arion/tarball/master) {}) ];
virtualisation.docker.enable = true;
users.extraUsers.myuser.extraGroups = ["docker"];
}
```
////
== Not installing: use it in a project
TODO: describe: using nix-shell or in a script, building images as
part of nix-build, pinning, see also todomvc-nix.
TODO: exposed Nix functions: arion.build, arion.eval (a bit of IFD)
////
== Usage
Arion is configured declaratively with two files:
=== arion-pkgs.nix
Arion needs `arion-pkgs.nix` to import nixpkgs, it's contents can be as simple as:
```nix
import <nixpkgs> {}
```
or more sophisticated (recommended) setup with https://github.com/nmattia/niv[Niv].
=== arion-compose.nix
Describe containers using NixOS-style modules. There are a few options:
==== Minimal: Plain command using nixpkgs
`examples/minimal/arion-compose.nix`:
```nix
{ pkgs, ... }:
{
config.docker-compose.services = {
webserver = {
service.useHostStore = true;
service.command = [ "sh" "-c" ''
cd "$$WEB_ROOT"
${pkgs.python3}/bin/python -m http.server
'' ];
service.ports = [
"8000:8000" # host:container
];
service.environment.WEB_ROOT = "${pkgs.nix.doc}/share/doc/nix/manual";
};
};
}
```
==== NixOS: run only one systemd service
`examples/nixos-unit/arion-compose.nix`:
```nix
{
docker-compose.services.webserver = { config, pkgs, ... }: {
nixos.configuration = {config, pkgs, ...}: {
boot.isContainer = true;
services.nginx.enable = true;
services.nginx.virtualHosts.localhost.root = "${pkgs.nix.doc}/share/doc/nix/manual";
system.build.run-nginx = pkgs.writeScript "run-nginx" ''
#!${pkgs.bash}/bin/bash
PATH='${config.systemd.services.nginx.environment.PATH}'
echo nginx:x:${toString config.users.users.nginx.uid}:${toString config.users.groups.nginx.gid}:nginx web server user:/var/empty:/bin/sh >>/etc/passwd
echo nginx:x:${toString config.users.groups.nginx.gid}:nginx >>/etc/group
${config.systemd.services.nginx.runner}
'';
};
service.command = [ config.nixos.build.run-nginx ];
service.useHostStore = true;
service.ports = [
"8000:80" # host:container
];
};
}
```
==== NixOS: run full OS
`examples/full-nixos/arion-compose.nix`:
```nix
{
docker-compose.services.webserver = { pkgs, ... }: {
nixos.useSystemd = true;
nixos.configuration.boot.tmpOnTmpfs = true;
nixos.configuration.services.nginx.enable = true;
nixos.configuration.services.nginx.virtualHosts.localhost.root = "${pkgs.nix.doc}/share/doc/nix/manual";
service.useHostStore = true;
service.ports = [
"8000:80" # host:container
];
};
}
```
==== Docker image from DockerHub
```nix
{
docker-compose.services.postgres = {
service.image = "postgres:10";
service.volumes = [ "${toString ./.}/postgres-data:/var/lib/postgresql/data" ];
service.environment.POSTGRES_PASSWORD = "mydefaultpass";
};
}
```
=== Run
Start containers and watch their logs:
```bash
$ arion up -d
$ arion logs -f
```
You can go to `examples/*/` and run these commands to give it a quick try.
== A full featured Nix command example
To see how Arion can be used in a project, have a look at
https://github.com/nix-community/todomvc-nix/tree/master/deploy/arion[todomvc-nix].
```bash
$ git clone https://github.com/nix-community/todomvc-nix
$ cd todomvc-nix/deploy/arion
$ arion up
```
== Project Status
This project was born out of a process supervision need for local
development environments while working on
https://www.hercules-ci.com[Hercules CI]. (It was also born out of
ancient Greek deities disguised as horses. More on that later.)
If you do want to use Arion for production environments, youll probably
want to either build normal container images or manage garbage
collection roots if you control the deployment host. Neither scenario is
made easier by arion at this time.
Arion has run successfully on Linux distributions other than NixOS, but we only perform CI for Arion on NixOS.
== How it works
Arion is essentially a thin wrapper around Nix and docker-compose. When
it runs, it does the following:
* Evaluate the configuration using Nix, producing a
`docker-compose.yaml` and a garbage collection root
* Invoke `docker-compose`
* Clean up the garbage collection root
Most of the interesting stuff happens in Arions Nix expressions, where
it runs the module system (known from NixOS) and provides the
configuration that makes the Docker Compose file do the things it needs
to do.
One of the more interesting built-in modules is the
link:src/nix/modules/service/host-store.nix[host-store.nix module] which
performs the bind mounts to make the host Nix store available in the
container.
== FAQ
=== Do I need to use Hercules CI?
Nope, its just Nix and Docker Compose under the hood.
=== What about garbage collection?
Arion removes the need for garbage collecting docker images, delegating
this task to Nix.
Arion creates a garbage collection root and cleans it up after
completing the command. This means that `arion up` without `-d` is safe
with respect to garbage collection. A deployment that is more serious
than local development must leave a GC root on the deployment host. This
use case is not supported as of now.
=== Why is my container not running latest code?
Restart it with `arion restart <name>` or if you've changed the image rebuild
them using `arion up -d --always-recreate-deps <name>`.
=== What is messing with my environment variables?
Docker Compose performs its own environment variable substitution. This
can be a little annoying in `services.command` for example. Either
reference a script from `pkgs.writeScript` or escape the dollar sign as
`$$`.
=== Why name it ``Arion``?
Arion comes from Greek mythology. Poseidon, the god of ~Docker~ the seas
had his eye on Demeter. Demeter tried to trick him by disguising as a
horse, but Poseidon saw through the deception and they had Arion.
So Arion is a super fast divine horse; the result of some weird mixing.
Also it talks.
(And we feel morally obliged to name our stuff after Greek mythology)

View file

@ -1,2 +0,0 @@
import Distribution.Simple
main = defaultMain

View file

@ -1,13 +0,0 @@
# This is not used when the docs are imported into the docs site.
# It is only here for development preview.
content:
sources:
- url: .
start_path: docs
branches: HEAD
ui:
bundle:
url: ./docs/ui-bundle.zip
output:
dir: ./public

View file

@ -1,90 +0,0 @@
cabal-version: 2.4
name: arion-compose
version: 0.2.1.0
synopsis: Run docker-compose with help from Nix/NixOS
description: Arion is a tool for building and running applications that consist of multiple docker containers using NixOS modules. It has special support for docker images that are built with Nix, for a smooth development experience and improved performance.
homepage: https://github.com/hercules-ci/arion#readme
-- bug-reports:
license: Apache-2.0
license-file: LICENSE
author: Robert Hensing
maintainer: robert@hercules-ci.com
-- copyright:
category: Distribution, Network, Cloud, Distributed Computing
extra-source-files: CHANGELOG.md, README.asciidoc,
src/haskell/testdata/**/*.nix
src/haskell/testdata/**/*.json
data-files: nix/*.nix
, nix/modules/composition/*.nix
, nix/modules/networks/*.nix
, nix/modules/nixos/*.nix
, nix/modules/service/*.nix
, nix/modules/lib/*.nix
-- all data is verbatim from some sources
data-dir: src
source-repository head
type: git
location: https://github.com/hercules-ci/arion
common common
build-depends: base >=4.12.0.0 && <4.99
, aeson >=2
, aeson-pretty
, async
, bytestring
, directory
, lens
, lens-aeson
, process
, temporary
, text
, protolude >= 0.2
, unix
ghc-options: -Wall
flag ghci
default: False
manual: True
library
import: common
exposed-modules: Arion.Nix
Arion.Aeson
Arion.DockerCompose
Arion.ExtendedInfo
Arion.Images
Arion.Services
other-modules: Paths_arion_compose
autogen-modules: Paths_arion_compose
-- other-extensions:
hs-source-dirs: src/haskell/lib
default-language: Haskell2010
executable arion
import: common
main-is: Main.hs
-- other-modules:
-- other-extensions:
build-depends: optparse-applicative
, arion-compose
hs-source-dirs: src/haskell/exe
default-language: Haskell2010
test-suite arion-unit-tests
import: common
if flag(ghci)
hs-source-dirs: src/haskell/lib
ghc-options: -Wno-missing-home-modules
type: exitcode-stdio-1.0
main-is: TestMain.hs
other-modules: Spec
, Arion.NixSpec
-- other-extensions:
build-depends: arion-compose
, hspec
, QuickCheck
hs-source-dirs: src/haskell/test
default-language: Haskell2010

38
arion.nix Normal file
View file

@ -0,0 +1,38 @@
{ stdenv, lib
, coreutils, docker_compose, jq
}:
let
arion = stdenv.mkDerivation {
name = "arion";
src = ./src;
unpackPhase = "";
buildPhase = "";
installPhase = ''
mkdir -p $out/bin $out/share/arion
cp -a nix $out/share/arion/
cp -a arion-image $out/share/arion/
tar -czf $out/share/arion/arion-image/tarball.tar.gz -C arion-image/tarball .
substitute arion $out/bin/arion \
--subst-var-by path ${lib.makeBinPath [jq coreutils docker_compose]} \
--subst-var-by nix_dir $out/share/arion/nix \
;
chmod a+x $out/bin/arion
'';
inherit passthru;
};
passthru = {
inherit eval build;
};
eval = import "${nix_dir}/eval-composition.nix";
build = args@{...}:
let composition = eval args;
in composition.config.build.dockerComposeYaml;
nix_dir = "${arion.outPath}/share/arion/nix";
in
arion

View file

@ -1,5 +0,0 @@
status = [
"ci/hercules/onPush/default",
"ci/hercules/evaluation",
]
delete_merged_branches = true

6
build
View file

@ -1,6 +0,0 @@
#!/usr/bin/env nix-shell
#!nix-shell -i bash
# Build the Haskell package via cabal, outside Nix
cabal new-build --write-ghc-environment-files=never

View file

@ -1 +0,0 @@
packages: .

View file

@ -1,11 +1,6 @@
let flake = import ./nix/compat.nix; args@{ pkgs ? import ./nix args, ... }:
in
{ pkgs ? import flake.inputs.nixpkgs { }
, haskellPackages ? pkgs.haskellPackages
}:
let
pkgsWithArion = pkgs.extend flake.overlays.default;
in
{ {
inherit (pkgsWithArion) arion; inherit (pkgs) arion tests;
doc = pkgs.recurseIntoAttrs (import ./doc { inherit pkgs; });
} }

8
doc/default.nix Normal file
View file

@ -0,0 +1,8 @@
{ pkgs ? import ../nix {} }:
let
inherit (pkgs) recurseIntoAttrs callPackage;
in
recurseIntoAttrs {
manual = callPackage ./manual {};
}

3
doc/manual/.gitignore vendored Normal file
View file

@ -0,0 +1,3 @@
manual.html
options-composition.xml
options-service.xml

35
doc/manual/Makefile Normal file
View file

@ -0,0 +1,35 @@
xsltproc = xsltproc --nonet \
--param section.autolabel 0 \
--param section.label.includes.component.label 0 \
--param chapter.autolabel 0 \
--param chapter.label.includes.component.label 0 \
--param appendix.autolabel 0 \
--param appendix.label.includes.component.label 0 \
--param generate.toc "'book toc,title chapter nop section nop sect1 nop sect2 nop sect3 nop sect4 nop sect5 nop'" \
--param html.stylesheet \'style.css\' \
--param xref.with.number.and.title 0 \
--param toc.section.depth 3 \
--param admon.style \'\' \
--param callout.graphics.extension \'.gif\' \
--param contrib.inline.enabled 0
docbookxsl = http://docbook.sourceforge.net/release/xsl/current
all: manual.html
manual.html: manual.xml options-composition.xml options-service.xml
$(xsltproc) --xinclude --stringparam profile.condition manual \
$(docbookxsl)/profiling/profile.xsl manual.xml | \
$(xsltproc) --output manual.html $(docbookxsl)/xhtml/docbook.xsl -
# -e 's_<book lang="en">__' -e 's_</book>__'
%.xml: %.asciidoc
asciidoctor --backend docbook45 --doctype article $<
sed -e 's/<!DOCTYPE.*//' -e 's/<?asciidoc-[a-z]*?>//' -i $@
options-composition.xml options-service.xml:
echo "options-composition.xml and options-service.xml should be written by the derivation. Are you running in 'nix-shell -A manual'?"; exit 1; fi
install: all
mkdir -p $(docdir)
cp manual.html style.css $(docdir)

2
doc/manual/build Executable file
View file

@ -0,0 +1,2 @@
#!/usr/bin/env bash
nix-shell -A manual --run 'patchPhase && make'

132
doc/manual/default.nix Normal file
View file

@ -0,0 +1,132 @@
{ pkgs ? import ../../nix {}
, version ? "none"
# Default sourceUrl is for local development. Works with
# nix-build -A doc.manual
# For release, use: https://github.com/hercules-ci/arion/blob/${version}
, sourceUrl ? "../../.."
}:
let
inherit (pkgs) recurseIntoAttrs callPackage runCommand lib stdenv ;
nixosManualPath = s: "${pkgs.path}/nixos/doc/manual/${s}";
# NixOS module system options in JSON format.
options = { moduleType, description, /*optionsList*/ optionsExpr }: recurseIntoAttrs rec {
optionsXML =
# builtins.toFile "options.xml" (builtins.toXML optionsList);
pkgs.runCommand "options.xml" {
buildInputs = [pkgs.nix pkgs.jq];
inherit optionsExpr;
} ''
export NIX_LOG_DIR=$PWD
export NIX_STATE_DIR=$PWD
nix-instantiate \
--option sandbox false \
--readonly-mode \
--eval \
--expr "$optionsExpr" \
--xml \
--strict \
--show-trace \
>$out
'';
optionsDocBook = runCommand "options-db.xml" {} ''
optionsXML=${optionsXML}
${pkgs.buildPackages.libxslt.bin}/bin/xsltproc \
--stringparam revision '${version}' \
--stringparam sourceUrl '${sourceUrl}' \
-o intermediate.xml ${./options-to-docbook.xsl} $optionsXML
${pkgs.buildPackages.libxslt.bin}/bin/xsltproc \
-o "$out" ${nixosManualPath "postprocess-option-descriptions.xsl"} intermediate.xml
'';
};
compositionOptions = options {
moduleType = "composition";
description = "List of Arion composition-level options in JSON format";
optionsExpr = let
src = ../../src/nix;
in ''
let pkgs = import ${pkgs.path} {};
fixPaths = opt: opt // {
declarations = map (d: "src/nix" + (lib.strings.removePrefix (toString ${src}) (toString d))) opt.declarations;
};
inherit (pkgs) lib;
composition = import ${src}/eval-composition.nix { inherit pkgs; };
in map fixPaths (lib.filter (opt: opt.visible && !opt.internal) (lib.optionAttrSetToDocList composition.options))
'';
};
serviceOptions = options {
moduleType = "service";
description = "List of Arion service-level options in JSON format";
optionsExpr = let
src = ../../src/nix;
in ''
let pkgs = import ${pkgs.path} {};
fixPaths = opt: opt // {
declarations = map (d: "src/nix" + (lib.strings.removePrefix (toString ${src}) (toString d))) opt.declarations;
};
inherit (pkgs) lib;
composition = pkgs.callPackage ${src}/eval-service.nix {} { modules = []; host = {}; name = abort "The manual's service options section must not depend on the service name."; };
in map fixPaths (lib.filter (opt: opt.visible && !opt.internal) (lib.optionAttrSetToDocList composition.options))
'';
};
generatedDocBook = runCommand "generated-docbook" {} ''
mkdir $out
ln -s ${compositionOptions.optionsDocBook} $out/options-composition.xml
ln -s ${serviceOptions.optionsDocBook} $out/options-service.xml
'';
manual = stdenv.mkDerivation {
src = lib.sourceByRegex ./. [
"Makefile$"
".*\.asciidoc$"
".*\.xsl$"
".*\.css$"
"^manual.xml$"
"^manual.xml$"
];
name = "arion-manual";
version = version;
buildInputs = [
(pkgs.libxslt.bin or pkgs.libxslt)
pkgs.asciidoctor
];
XML_CATALOG_FILES = "${pkgs.docbook_xsl}/xml/xsl/docbook/catalog.xml";
inherit generatedDocBook;
configurePhase = ''
export docdir=$out/doc
'';
postPatch = ''
substituteInPlace manual.xml --subst-var version
'';
prePatch = ''
cp ${generatedDocBook}/* .
substituteInPlace options-service.xml \
--replace 'xml:id="appendix-configuration-options"' 'xml:id="appendix-service-options"' \
--replace '<title>Configuration Options</title>' '<title>Service Options</title>' \
--replace 'xml:id="configuration-variable-list"' 'xml:id="service-variable-list"' \
;
substituteInPlace options-composition.xml \
--replace 'xml:id="appendix-configuration-options"' 'xml:id="appendix-composition-options"' \
--replace '<title>Configuration Options</title>' '<title>Composition Options</title>' \
--replace 'xml:id="configuration-variable-list"' 'xml:id="composition-variable-list"' \
;
'';
shellHook = ''
live-build() {
patchPhase
inotifywait -e MODIFY -m -r . | while read; do
make
done
}
'';
passthru = {
inherit generatedDocBook;
};
};
in
manual

3
doc/manual/live-build Executable file
View file

@ -0,0 +1,3 @@
#!/usr/bin/env nix-shell
#!nix-shell -A manual
#!nix-shell --run live-build

22
doc/manual/manual.xml Normal file
View file

@ -0,0 +1,22 @@
<book xmlns="http://docbook.org/ns/docbook"
xmlns:xlink="http://www.w3.org/1999/xlink"
xmlns:xi="http://www.w3.org/2001/XInclude">
<info>
<title>Arion Manual</title>
<subtitle>Version none</subtitle>
<author>
<affiliation>
<orgname>Hercules Labs and other Arion contributors</orgname>
</affiliation>
<contrib>Author</contrib>
</author>
</info>
<xi:include href="options-composition.xml" />
<xi:include href="options-service.xml" />
</book>

View file

@ -0,0 +1,224 @@
<?xml version="1.0"?>
<xsl:stylesheet version="1.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xmlns:str="http://exslt.org/strings"
xmlns:xlink="http://www.w3.org/1999/xlink"
xmlns:nixos="tag:nixos.org"
xmlns="http://docbook.org/ns/docbook"
extension-element-prefixes="str"
>
<xsl:output method='xml' encoding="UTF-8" />
<xsl:param name="revision" />
<xsl:param name="program" />
<xsl:param name="sourceUrl" />
<xsl:param name="nixPathKey" />
<xsl:template match="/expr/list">
<appendix xml:id="appendix-configuration-options">
<title>Configuration Options</title>
<variablelist xml:id="configuration-variable-list">
<xsl:for-each select="attrs">
<xsl:variable name="id" select="concat('opt-', str:replace(str:replace(str:replace(str:replace(attr[@name = 'name']/string/@value, '*', '_'), '&lt;', '_'), '>', '_'), '?', '_'))" />
<varlistentry>
<term xlink:href="#{$id}">
<xsl:attribute name="xml:id"><xsl:value-of select="$id"/></xsl:attribute>
<option>
<xsl:value-of select="attr[@name = 'name']/string/@value" />
</option>
</term>
<listitem>
<nixos:option-description>
<para>
<xsl:value-of disable-output-escaping="yes"
select="attr[@name = 'description']/string/@value" />
</para>
</nixos:option-description>
<xsl:if test="attr[@name = 'type']">
<para>
<emphasis>Type:</emphasis>
<xsl:text> </xsl:text>
<xsl:value-of select="attr[@name = 'type']/string/@value"/>
<xsl:if test="attr[@name = 'readOnly']/bool/@value = 'true'">
<xsl:text> </xsl:text>
<emphasis>(read only)</emphasis>
</xsl:if>
</para>
</xsl:if>
<xsl:if test="attr[@name = 'default']">
<para>
<emphasis>Default:</emphasis>
<xsl:text> </xsl:text>
<xsl:apply-templates select="attr[@name = 'default']" mode="top" />
</para>
</xsl:if>
<xsl:if test="attr[@name = 'example']">
<para>
<emphasis>Example:</emphasis>
<xsl:text> </xsl:text>
<xsl:choose>
<xsl:when test="attr[@name = 'example']/attrs[attr[@name = '_type' and string[@value = 'literalExample']]]">
<programlisting><xsl:value-of select="attr[@name = 'example']/attrs/attr[@name = 'text']/string/@value" /></programlisting>
</xsl:when>
<xsl:otherwise>
<xsl:apply-templates select="attr[@name = 'example']" mode="top" />
</xsl:otherwise>
</xsl:choose>
</para>
</xsl:if>
<xsl:if test="attr[@name = 'relatedPackages']">
<para>
<emphasis>Related packages:</emphasis>
<xsl:text> </xsl:text>
<xsl:value-of disable-output-escaping="yes"
select="attr[@name = 'relatedPackages']/string/@value" />
</para>
</xsl:if>
<xsl:if test="count(attr[@name = 'declarations']/list/*) != 0">
<para>
<emphasis>Declared by:</emphasis>
</para>
<xsl:apply-templates select="attr[@name = 'declarations']" />
</xsl:if>
<xsl:if test="count(attr[@name = 'definitions']/list/*) != 0">
<para>
<emphasis>Defined by:</emphasis>
</para>
<xsl:apply-templates select="attr[@name = 'definitions']" />
</xsl:if>
</listitem>
</varlistentry>
</xsl:for-each>
</variablelist>
</appendix>
</xsl:template>
<xsl:template match="*" mode="top">
<xsl:choose>
<xsl:when test="string[contains(@value, '&#010;')]">
<programlisting>
<xsl:text>''
</xsl:text><xsl:value-of select='str:replace(string/@value, "${", "&apos;&apos;${")' /><xsl:text>''</xsl:text></programlisting>
</xsl:when>
<xsl:otherwise>
<literal><xsl:apply-templates /></literal>
</xsl:otherwise>
</xsl:choose>
</xsl:template>
<xsl:template match="null">
<xsl:text>null</xsl:text>
</xsl:template>
<xsl:template match="string">
<xsl:choose>
<xsl:when test="(contains(@value, '&quot;') or contains(@value, '\')) and not(contains(@value, '&#010;'))">
<xsl:text>''</xsl:text><xsl:value-of select='str:replace(@value, "${", "&apos;&apos;${")' /><xsl:text>''</xsl:text>
</xsl:when>
<xsl:otherwise>
<xsl:text>"</xsl:text><xsl:value-of select="str:replace(str:replace(str:replace(str:replace(@value, '\', '\\'), '&quot;', '\&quot;'), '&#010;', '\n'), '$', '\$')" /><xsl:text>"</xsl:text>
</xsl:otherwise>
</xsl:choose>
</xsl:template>
<xsl:template match="int">
<xsl:value-of select="@value" />
</xsl:template>
<xsl:template match="bool[@value = 'true']">
<xsl:text>true</xsl:text>
</xsl:template>
<xsl:template match="bool[@value = 'false']">
<xsl:text>false</xsl:text>
</xsl:template>
<xsl:template match="list">
[
<xsl:for-each select="*">
<xsl:apply-templates select="." />
<xsl:text> </xsl:text>
</xsl:for-each>
]
</xsl:template>
<xsl:template match="attrs[attr[@name = '_type' and string[@value = 'literalExample']]]">
<xsl:value-of select="attr[@name = 'text']/string/@value" />
</xsl:template>
<xsl:template match="attrs">
{
<xsl:for-each select="attr">
<xsl:value-of select="@name" />
<xsl:text> = </xsl:text>
<xsl:apply-templates select="*" /><xsl:text>; </xsl:text>
</xsl:for-each>
}
</xsl:template>
<xsl:template match="derivation">
<replaceable>(build of <xsl:value-of select="attr[@name = 'name']/string/@value" />)</replaceable>
</xsl:template>
<xsl:template match="attr[@name = 'declarations' or @name = 'definitions']">
<simplelist>
<xsl:for-each select="list/string">
<member><filename>
<!-- Hyperlink the filename either to the NixOS Subversion
repository (if its a module and we have a revision number),
or to the local filesystem. -->
<xsl:choose>
<xsl:when test="not(starts-with(@value, '/'))">
<xsl:attribute name="xlink:href"><xsl:value-of select="$sourceUrl"/>/<xsl:value-of select="@value"/></xsl:attribute>
</xsl:when>
<xsl:otherwise>
<xsl:attribute name="xlink:href">file://<xsl:value-of select="@value"/></xsl:attribute>
</xsl:otherwise>
</xsl:choose>
<!-- Print the filename and make it user-friendly by replacing the
/nix/store/<hash> prefix by the default location of nixos
sources. -->
<xsl:choose>
<xsl:when test="$nixPathKey != ''">
&lt;<xsl:value-of select="$nixPathKey"/>/<xsl:value-of select="@value"/>&gt;
</xsl:when>
<xsl:otherwise>
<xsl:value-of select="@value" />
</xsl:otherwise>
</xsl:choose>
</filename></member>
</xsl:for-each>
</simplelist>
</xsl:template>
<xsl:template match="function">
<xsl:text>λ</xsl:text>
</xsl:template>
</xsl:stylesheet>

159
doc/manual/style.css Normal file
View file

@ -0,0 +1,159 @@
hr { color: #ddd; margin-top: 3ex; }
h1, h2, h3, h4 { font-weight: bold; }
h1 { font-size: 200%; margin-top: 5ex; }
h2 { font-size: 160%; margin-top: 4ex; }
h3,h4 { font-size: 120%; margin-top: 3ex; }
/* From Semantic UI */
body {
font-family: Lato,'Helvetica Neue',Arial,Helvetica,sans-serif;
font-size: 14px;
line-height: 1.4285em;
color: rgba(0,0,0,.87);
}
code.literal {
background-color: #f7f7f7;
}
a {
background-color:transparent;
-webkit-text-decoration-skip:objects
}
a {
color:#4183c4;
text-decoration:none
}
a:hover {
color:#1e70bf;
text-decoration:none
}
::-webkit-selection {
background-color:#cce2ff;
color:rgba(0,0,0,.87)
}
::-moz-selection {
background-color:#cce2ff;
color:rgba(0,0,0,.87)
}
::selection {
background-color:#cce2ff;
color:rgba(0,0,0,.87)
}
/* toc menu */
@media screen and (min-width: 60em) {
body {
margin-left: 16.5em;
}
div.toc {
position: fixed;
top: 0pt;
left: 1em;
height: 100%;
overflow-y: auto;
width: 15.5em;
}
}
@media screen and (min-width: 90em) {
div.toc {
left: 5em;
}
}
/* hide per chapter toc */
div.chapter div.toc {
display: none;
}
/* From Nixpkgs: */
div.book
{
text-align: center;
}
div.book > div
{
/*
* based on https://medium.com/@zkareemz/golden-ratio-62b3b6d4282a
* we do 70 characters per line to fit code listings better
* 70 * (font-size / 1.618)
* expression for emacs:
* (* 70 (/ 1 1.618))
*/
max-width: 43.2em;
text-align: left;
margin: auto;
}
/***************************************************************************
Special elements:
***************************************************************************/
.term
{
font-weight: bold;
}
div.variablelist dd p, div.glosslist dd p
{
margin-top: 0em;
}
div.variablelist dd, div.glosslist dd
{
margin-left: 1.5em;
}
div.glosslist dt
{
font-style: italic;
}
.varname
{
color: #004000;
}
span.command strong
{
font-weight: normal;
color: #004000;
}
div.calloutlist table
{
box-shadow: none;
}
table
{
border-collapse: collapse;
box-shadow: 0.4em 0.4em 0.5em #e0e0e0;
}
table.simplelist
{
text-align: left;
color: #005aa0;
border: 0;
padding: 5px;
background: #fffff5;
font-weight: normal;
font-style: italic;
box-shadow: none;
margin-bottom: 1em;
}
div.navheader table, div.navfooter table {
box-shadow: none;
}
div.affiliation
{
font-style: italic;
}

View file

@ -1,4 +0,0 @@
# Documentation
Please refer to the [**rendered documentation**](https://docs.hercules-ci.com/arion), which includes the [**options.**](https://docs.hercules-ci.com/arion/options/)

View file

@ -1,7 +0,0 @@
name: arion
title: Arion Documentation
version: 'master'
nav:
- modules/ROOT/nav.adoc
- modules/reference/nav.adoc
nix: true

View file

@ -1,31 +0,0 @@
{
perSystem = { config, pkgs, lib, ... }: {
packages.generated-option-doc-arion =
# TODO: use the render pipeline in flake-parts,
# which has support for things like {options}`foo`.
let
eval = lib.evalModules {
modules = import ../src/nix/modules.nix;
};
in
(pkgs.nixosOptionsDoc
{
options = eval.options;
}).optionsCommonMark;
packages.generated-antora-files =
pkgs.runCommand "generated-antora-files"
{
nativeBuildInputs = [ pkgs.pandoc ];
doc_arion = config.packages.generated-option-doc-arion;
}
# TODO: use the render pipeline in flake-parts,
# which has support for things like {options}`foo`.
''
mkdir -p $out/modules/ROOT/partials
pandoc --from=markdown --to=asciidoc \
< $doc_arion \
> $out/modules/ROOT/partials/arion-options.adoc
'';
};
}

View file

@ -1 +0,0 @@
../../../../../examples/full-nixos/arion-compose.nix

View file

@ -1 +0,0 @@
../../../../../examples/minimal/arion-compose.nix

View file

@ -1 +0,0 @@
../../../../../examples/nixos-unit/arion-compose.nix

View file

@ -1,3 +0,0 @@
* xref:index.adoc[Getting Started]
* xref:options.adoc[Arion Options]
* xref:deployment.adoc[Deployment]

View file

@ -1,71 +0,0 @@
= Deployment with Arion
Arion projects can be deployed in Nix-like or Docker-like ways.
== Docker images
When you disable `useHostStore`, arion will build images, which can be deployed
to any Docker host, including non-NixOS hosts.
=== Remote Docker socket
NOTE: Access to a Docker socket is equivalent to root access on the host.
Docker supports authentication via TLS client certificates.
The xref:hercules-ci-effects:ROOT:reference/nix-functions/runArion.adoc[runArion Effect] uses this technique.
Because this technique works with a single Docker host, it does not need a registry.
=== Upload to registry
You can either use `arion push` or write custom push logic using the `arion cat`
command, the `eval` function on the `arion` package, or the `lib.eval` function
on the flake to retrieve the images defined in a project.
== NixOS module
Arion projects can be deployed as part of a NixOS configuration. This ties the
project revision to the system configuration revision, which can be good or bad
thing, depending on your deployment strategy. At a low level, a benefit is that
no store paths need to be copied locally and remote NixOS deployments can use
Nix's copy-closure algorithm for efficient transfers, and transparent binary
caches rather than an inherently stateful Docker registry solution.
Extend your NixOS configuration by adding the configuration elements to an
existing configuration. You could create a new module file for it, if your
choice of `imports` allows it.
NOTE: This deployment method does NOT use an `arion-pkgs.nix` file, but reuses
the host `pkgs`.
```nix
{
imports = [
# Pick one of:
# - niv
((import ./nix/sources.nix).arion + "/nixos-module.nix")
# - or flakes (where arion is a flake input)
arion.nixosModules.arion
# - or other: copy commit hash of arion and replace HASH in:
(builtins.fetchTarball "https://github.com/hercules-ci/arion/archive/HASH.tar.gz") + "/nixos-module.nix")
];
virtualisation.arion = {
backend = "podman-socket"; # or "docker"
projects.example = {
serviceName = "example"; # optional systemd service name, defaults to arion-example in this case
settings = {
# Specify you project here, or import it from a file.
# NOTE: This does NOT use ./arion-pkgs.nix, but defaults to NixOS' pkgs.
imports = [ ./arion-compose.nix ];
};
};
};
}
```
See also:
- xref:hercules-ci-effects:ROOT:reference/nix-functions/runNixOS.adoc[runNixOS Effect]
- xref:hercules-ci-effects:ROOT:reference/nix-functions/runNixOps2.adoc[runNixOps2 Effect]

View file

@ -1,312 +0,0 @@
= Welcome to Arion documentation
== Introduction
Arion is a tool for building and running applications that
consist of multiple docker containers using NixOS modules.
It has special support for docker images that are built with Nix,
for a smooth development experience and improved performance.
It is built on top of https://docs.docker.com/compose/overview/[Docker
Compose], which implements the container orchestration functionality.
Instead of configuring the compositions in YAML files like
`docker-compose.yaml`, Arion uses the https://nixos.org/nix/[Nix]
language to declare the compositions. Because of this, Arion gives you
the ability to declare your deployments, configuration and packaging
in the same language. By replacing multiple tools with a single
language, you decrease your mental load and you can more easily
refactor and maintain your configurations.
Although Arion can be used as a Docker Compose with an improved
configuration front end, there is more to be gained from integrating
with Nix. In particular, the more structured approach of Nix compared
to Dockerfiles allows the following:
* Build components of your image in *parallel, automatically*
* *Share packages between images*, regardless of the order they were
added
* Improve performance by *skipping container
image creation*
* Work with *structured data instead of strings*,
templates and a multitude of expression languages
* Refactor across deployments, configuration and packaging
Arion allows to compose containers with different granularity:
* <<Minimal: Plain command using nixpkgs>>
* <<NixOS: run only one systemd service>>
* <<NixOS: run full OS>>
* <<Docker image from DockerHub>>
Full NixOS is supported on
* docker-compose + podman with docker socket (NixOS >= 21.05)
* docker-compose + docker, before cgroupsv2 (NixOS < 21.05)
`podman-compose` support is currently WIP on a separate branch.
== Installation
=== Nix
```bash
$ nix-env -iA arion -f https://github.com/hercules-ci/arion/tarball/master
```
=== NixOS
Add this module to your NixOS configuration:
```nix
{ pkgs, ... }: {
environment.systemPackages = [
pkgs.arion
# Do install the docker CLI to talk to podman.
# Not needed when virtualisation.docker.enable = true;
pkgs.docker-client
];
# Arion works with Docker, but for NixOS-based containers, you need Podman
# since NixOS 21.05.
virtualisation.docker.enable = false;
virtualisation.podman.enable = true;
virtualisation.podman.dockerSocket.enable = true;
virtualisation.podman.defaultNetwork.dnsname.enable = true;
# Use your username instead of `myuser`
users.extraUsers.myuser.extraGroups = ["podman"];
}
```
////
== Not installing: use it in a project
TODO: describe: using nix-shell or in a script, building images as
part of nix-build, pinning, see also todomvc-nix.
TODO: exposed Nix functions: arion.build, arion.eval (a bit of IFD)
////
== Usage
Arion is configured declaratively with two files:
=== arion-pkgs.nix
Arion needs `arion-pkgs.nix` to import nixpkgs, for example:
```nix
import <nixpkgs> { system = "x86_64-linux"; }
```
or more sophisticated (recommended) setup with https://github.com/nmattia/niv[Niv].
=== arion-compose.nix
Describe containers using NixOS-style modules. There are a few options:
==== Minimal: Plain command using nixpkgs
`examples/minimal/arion-compose.nix`
[,nix]
----
{ pkgs, ... }:
{
project.name = "webapp";
services = {
webserver = {
image.enableRecommendedContents = true;
service.useHostStore = true;
service.command = [ "sh" "-c" ''
cd "$$WEB_ROOT"
${pkgs.python3}/bin/python -m http.server
'' ];
service.ports = [
"8000:8000" # host:container
];
service.environment.WEB_ROOT = "${pkgs.nix.doc}/share/doc/nix/manual";
service.stop_signal = "SIGINT";
};
};
}
----
==== NixOS: run full OS
`examples/full-nixos/arion-compose.nix`:
[,nix]
----
{
project.name = "full-nixos";
services.webserver = { pkgs, lib, ... }: {
nixos.useSystemd = true;
nixos.configuration.boot.tmp.useTmpfs = true;
nixos.configuration.services.nginx.enable = true;
nixos.configuration.services.nginx.virtualHosts.localhost.root = "${pkgs.nix.doc}/share/doc/nix/manual";
nixos.configuration.services.nscd.enable = false;
nixos.configuration.system.nssModules = lib.mkForce [];
nixos.configuration.systemd.services.nginx.serviceConfig.AmbientCapabilities =
lib.mkForce [ "CAP_NET_BIND_SERVICE" ];
service.useHostStore = true;
service.ports = [
"8000:80" # host:container
];
};
}
----
==== Docker image from DockerHub
```nix
{
services.postgres = {
service.image = "postgres:10";
service.volumes = [ "${toString ./.}/postgres-data:/var/lib/postgresql/data" ];
service.environment.POSTGRES_PASSWORD = "mydefaultpass";
};
}
```
==== NixOS: run only one systemd service
Running individual units from NixOS is possible using an experimental script.
See `examples/nixos-unit/arion-compose.nix`.
=== Run
Start containers and watch their logs:
```bash
$ arion up -d
$ arion logs -f
```
You can go to `examples/*/` and run these commands to give it a quick try.
=== Inspect the config
While developing an arion project, you can make use of `arion repl`, which launches
a `nix repl` on the project configuration.
```
$ arion repl
Launching a repl for you. To get started:
To see deployment-wide configuration
type config. and use tab completion
To bring the top-level Nixpkgs attributes into scope
type :a (config._module.args.pkgs) // { inherit config; }
Welcome to Nix. Type :? for help.
Loading '../../src/nix/eval-composition.nix'...
Added 5 variables.
nix-repl> config.services.webserver.service.command
[ "sh" "-c" "cd \"$$WEB_ROOT\"\n/nix/store/66fbv9mmx1j4hrn9y06kcp73c3yb196r-python3-3.8.9/bin/python -m http.server\n" ]
nix-repl>
```
== Build with Nix
You can build a project with `nix-build` using an expression like
```nix
arion.build { modules = [ ./arion-compose.nix ]; pkgs = import ./arion-pkgs.nix; }
```
If you deploy with xref:hercules-ci-effects:ROOT:reference/nix-functions/runArion.adoc[runArion],
and your `pkgs` variable is equivalent to `import ./arion-pkgs.nix`, you can use:
```nix
let
deployment = pkgs.effects.runArion { /* ... */ });
in deployment.prebuilt
```
== Project Status
This project was born out of a process supervision need for local
development environments while working on
https://www.hercules-ci.com[Hercules CI]. (It was also born out of
ancient Greek deities disguised as horses. More on that later.)
Arion can be used for simple single host deployments, using Docker's TLS
client verification, or https://search.nixos.org/options?channel=unstable&show=virtualisation.podman.networkSocket.enable&query=virtualisation.podman[`virtualisation.podman.networkSocket` options].
Remote deployments do not support `useHostStore`, although an SSH-based deployment method could support this.
Docker Swarm is not currently supported.
Arion has run successfully on Linux distributions other than NixOS, but we only perform CI for Arion on NixOS.
== How it works
Arion is essentially a thin wrapper around Nix and docker-compose. When
it runs, it does the following:
* Evaluate the configuration using Nix, producing a
`docker-compose.yaml` and a garbage collection root
* Invoke `docker-compose`
* Clean up the garbage collection root
Most of the interesting stuff happens in Arions Nix expressions, where
it runs the module system (known from NixOS) and provides the
configuration that makes the Docker Compose file do the things it needs
to do.
One of the more interesting built-in modules is the
https://github.com/hercules-ci/arion/blob/master/src/nix/modules/service/host-store.nix[host-store.nix module] which
performs the bind mounts to make the host Nix store available in the
container.
== FAQ
=== Do I need to use Hercules CI?
Nope, its just Nix and Docker Compose under the hood.
It does xref:hercules-ci-effects:ROOT:reference/nix-functions/runArion.adoc[integrate] nicely though.
=== What about garbage collection?
Arion removes the need for garbage collecting docker images, delegating
this task to Nix when using `service.useHostStore`.
Arion creates a garbage collection root that it cleans up after completing
the command. This means that `arion up -d` should not be used with `useHostStore`
in production. Instead, disable `useHostStore`, which will use `dockerTools` to
generate images that can be used in production.
=== Why is my container not running latest code?
Rebuild the image using `arion up -d --always-recreate-deps <name>` or simply `arion up -d`.
Like `docker-compose restart`, `arion restart` does not update the image before starting.
=== What is messing with my environment variables?
Docker Compose performs its own environment variable substitution. This
can be a little annoying in `services.command` for example. Either
reference a script from `pkgs.writeScript` or escape the dollar sign as
`$$`.
=== Why name it ``Arion``?
Arion comes from Greek mythology. Poseidon, the god of Docker -- I mean the seas --
had his eye on Demeter. Demeter tried to trick him by disguising as a
horse, but Poseidon saw through the deception and they had Arion.
So Arion is a super fast divine horse; the result of some weird mixing.
Also it talks.
(And we felt morally obliged to name our stuff after Greek mythology)

View file

@ -1,3 +0,0 @@
# Arion Options
include::partial$arion-options.adoc[]

View file

@ -1,20 +0,0 @@
{ pkgs ? import ../nix {} }:
let
eval = pkgs.lib.evalModules {
modules = import ../src/nix/modules.nix;
};
options = pkgs.nixosOptionsDoc {
options = eval.options;
};
in (pkgs.runCommand "agent-options.adoc" { } ''
cat >$out <<EOF
= Arion options
EOF
cat ${options.optionsAsciiDoc} >>$out
'').overrideAttrs (o: {
# Work around https://github.com/hercules-ci/hercules-ci-agent/issues/168
allowSubstitutes = true;
})

Binary file not shown.

View file

@ -1,30 +0,0 @@
{ pkgs, ... }:
let
sh = pkgs.stdenv.mkDerivation {
name = "sh";
phases = [ "installPhase" ];
installPhase = ''
mkdir -p "$out"/bin
ln -s ${pkgs.bash}/bin/sh "$out"/bin/sh
'';
};
in{
config.project.name = "webapp";
config.services = {
webserver = {
image.contents = [ sh ];
service.useHostStore = true;
service.command = [ "sh" "-c" ''
cd "$$WEB_ROOT"
${pkgs.python3}/bin/python -m http.server
'' ];
service.ports = [
"8000:8000" # host:container
];
service.environment.WEB_ROOT = "${pkgs.nix.doc}/share/doc/nix/manual";
service.stop_signal = "SIGINT";
};
};
}

View file

@ -1,13 +0,0 @@
let
flake = if builtins ? getFlake
then (builtins.getFlake (toString ./.)).pkgs
else (import flake-compat { src = ./.; }).defaultNix;
# NB: this is lazy
lock = builtins.fromJSON (builtins.readFile ./flake.lock);
inherit (lock.nodes.flake-compat.locked) owner repo rev narHash;
flake-compat = builtins.fetchTarball {
url = "https://github.com/${owner}/${repo}/archive/${rev}.tar.gz";
sha256 = narHash;
};
in
flake.pkgs

View file

@ -1,44 +0,0 @@
{
"nodes": {
"flake-compat": {
"flake": false,
"locked": {
"lastModified": 1606424373,
"narHash": "sha256-oq8d4//CJOrVj+EcOaSXvMebvuTkmBJuT5tzlfewUnQ=",
"owner": "edolstra",
"repo": "flake-compat",
"rev": "99f1c2157fba4bfe6211a321fd0ee43199025dbf",
"type": "github"
},
"original": {
"owner": "edolstra",
"repo": "flake-compat",
"type": "github"
}
},
"nixpkgs": {
"locked": {
"lastModified": 1618853290,
"narHash": "sha256-K4fddnrGOcKL+6CEchRrVmepiwvwvHxB87goqBTI5Bs=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "9a1672105db0eebe8ef59f310397435f2d0298d0",
"type": "github"
},
"original": {
"owner": "NixOS",
"ref": "nixos-20.09",
"repo": "nixpkgs",
"type": "github"
}
},
"root": {
"inputs": {
"flake-compat": "flake-compat",
"nixpkgs": "nixpkgs"
}
}
},
"root": "root",
"version": 7
}

View file

@ -1,19 +0,0 @@
{
description = "A very basic flake";
inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-20.09";
inputs.flake-compat.url = "github:edolstra/flake-compat";
inputs.flake-compat.flake = false;
outputs = { self, nixpkgs, ... }: {
pkgs = nixpkgs.legacyPackages.x86_64-linux;
# # alternative:
# pkgs = import nixpkgs { config = { }; overlays = [ ]; system = "x86_64-linux"; };
packages.x86_64-linux.hello = nixpkgs.legacyPackages.x86_64-linux.hello;
defaultPackage.x86_64-linux = self.packages.x86_64-linux.hello;
};
}

View file

@ -1,15 +1,9 @@
{ {
project.name = "full-nixos"; docker-compose.services.webserver = { pkgs, ... }: {
services.webserver = { pkgs, lib, ... }: {
nixos.useSystemd = true; nixos.useSystemd = true;
nixos.configuration.boot.tmp.useTmpfs = true; nixos.configuration.boot.tmpOnTmpfs = true;
nixos.configuration.networking.useDHCP = false;
nixos.configuration.services.nginx.enable = true; nixos.configuration.services.nginx.enable = true;
nixos.configuration.services.nginx.virtualHosts.localhost.root = "${pkgs.nix.doc}/share/doc/nix/manual"; nixos.configuration.services.nginx.virtualHosts.localhost.root = "${pkgs.nix.doc}/share/doc/nix/manual";
nixos.configuration.services.nscd.enable = false;
nixos.configuration.system.nssModules = lib.mkForce [];
nixos.configuration.systemd.services.nginx.serviceConfig.AmbientCapabilities =
lib.mkForce [ "CAP_NET_BIND_SERVICE" ];
service.useHostStore = true; service.useHostStore = true;
service.ports = [ service.ports = [
"8000:80" # host:container "8000:80" # host:container

View file

@ -1,6 +1,2 @@
# Instead of pinning Nixpkgs, we can opt to use the one in NIX_PATH # Instead of pinning Nixpkgs, we can opt to use the one in NIX_PATH
import <nixpkgs> { import <nixpkgs> {}
# We specify the architecture explicitly. Use a Linux remote builder when
# calling arion from other platforms.
system = "x86_64-linux";
}

View file

@ -1,10 +1,8 @@
{ pkgs, ... }: { pkgs, ... }:
{ {
project.name = "webapp"; config.docker-compose.services = {
services = {
webserver = { webserver = {
image.enableRecommendedContents = true;
service.useHostStore = true; service.useHostStore = true;
service.command = [ "sh" "-c" '' service.command = [ "sh" "-c" ''
cd "$$WEB_ROOT" cd "$$WEB_ROOT"
@ -14,7 +12,6 @@
"8000:8000" # host:container "8000:8000" # host:container
]; ];
service.environment.WEB_ROOT = "${pkgs.nix.doc}/share/doc/nix/manual"; service.environment.WEB_ROOT = "${pkgs.nix.doc}/share/doc/nix/manual";
service.stop_signal = "SIGINT";
}; };
}; };
} }

View file

@ -1,6 +1,2 @@
# Instead of pinning Nixpkgs, we can opt to use the one in NIX_PATH # Instead of pinning Nixpkgs, we can opt to use the one in NIX_PATH
import <nixpkgs> { import <nixpkgs> {}
# We specify the architecture explicitly. Use a Linux remote builder when
# calling arion from other platforms.
system = "x86_64-linux";
}

View file

@ -17,27 +17,17 @@
*/ */
{ {
project.name = "nixos-unit"; docker-compose.services.webserver = { config, pkgs, ... }: {
services.webserver = { config, pkgs, ... }: {
nixos.configuration = {config, lib, options, pkgs, ...}: { nixos.configuration = {config, pkgs, ...}: {
boot.isContainer = true; boot.isContainer = true;
services.nginx = { services.nginx.enable = true;
enable = true; services.nginx.virtualHosts.localhost.root = "${pkgs.nix.doc}/share/doc/nix/manual";
virtualHosts.localhost.root = "${pkgs.nix.doc}/share/doc/nix/manual";
} // lib.optionalAttrs (options?services.nginx.stateDir) {
# Work around a problem in NixOS 20.03
stateDir = "/var/lib/nginx";
};
system.build.run-nginx = pkgs.writeScript "run-nginx" '' system.build.run-nginx = pkgs.writeScript "run-nginx" ''
#!${pkgs.bash}/bin/bash #!${pkgs.bash}/bin/bash
PATH='${config.systemd.services.nginx.environment.PATH}' PATH='${config.systemd.services.nginx.environment.PATH}'
echo nginx:x:${toString config.users.users.nginx.uid}:${toString config.users.groups.nginx.gid}:nginx web server user:/var/empty:/bin/sh >>/etc/passwd echo nginx:x:${toString config.users.users.nginx.uid}:${toString config.users.groups.nginx.gid}:nginx web server user:/var/empty:/bin/sh >>/etc/passwd
echo nginx:x:${toString config.users.groups.nginx.gid}:nginx >>/etc/group echo nginx:x:${toString config.users.groups.nginx.gid}:nginx >>/etc/group
echo 'nobody:x:65534:65534:Unprivileged account do not use:/var/empty:/run/current-system/sw/bin/nologin' >>/etc/passwd
echo 'nogroup:x:65534:' >>/etc/group
mkdir -p /var/log/nginx /run/nginx/ /var/cache/nginx /var/lib/nginx/{,logs,proxy_temp,client_body_temp,fastcgi_temp,scgi_temp,uwsgi_temp} /tmp/nginx_client_body
chown nginx /var/log/nginx /run/nginx/ /var/cache/nginx /var/lib/nginx/{,logs,proxy_temp,client_body_temp,fastcgi_temp,scgi_temp,uwsgi_temp} /tmp/nginx_client_body
${config.systemd.services.nginx.runner} ${config.systemd.services.nginx.runner}
''; '';
}; };

View file

@ -1,6 +1,2 @@
# Instead of pinning Nixpkgs, we can opt to use the one in NIX_PATH # Instead of pinning Nixpkgs, we can opt to use the one in NIX_PATH
import <nixpkgs> { import <nixpkgs> {}
# We specify the architecture explicitly. Use a Linux remote builder when
# calling arion from other platforms.
system = "x86_64-linux";
}

View file

@ -1,64 +0,0 @@
/*
An example of
- traefik HTTP reverse proxy
- minimal images
- routing via docker labels
Run `arion up -d` and open http://nix-docs.localhost/
*/
{ lib, pkgs, ... }: {
config.project.name = "traefik";
config.networks = {
traefik-custom = {
name = "traefik-custom";
ipam = {
config = [{
subnet = "172.32.0.0/16";
gateway = "172.32.0.1";
}];
};
};
};
config.services = {
traefik = {
image.command = [
"${pkgs.traefik}/bin/traefik"
"--api.insecure=true"
"--providers.docker=true"
"--providers.docker.exposedbydefault=false"
"--entrypoints.web.address=:80"
];
service = {
container_name = "traefik";
stop_signal = "SIGINT";
ports = [ "80:80" "8080:8080" ];
volumes = [ "/var/run/docker.sock:/var/run/docker.sock:ro" ];
networks = [ "traefik-custom" ];
};
};
nix-docs = {
image.command = ["${pkgs.writeScript "entrypoint" ''
#!${pkgs.bash}/bin/bash
cd ${pkgs.nix.doc}/share/doc/nix/manual
${pkgs.python3}/bin/python -m http.server
''}"];
service.container_name = "simple-service";
service.stop_signal = "SIGINT";
service.labels = {
"traefik.enable" = "true";
"traefik.http.routers.nix-docs.rule" = "Host(`nix-docs.localhost`)";
"traefik.http.routers.nix-docs.entrypoints" = "web";
"traefik.http.services.nix-docs.loadBalancer.server.port" = "8000";
};
service.networks = {
traefik-custom = {
ipv4_address = "172.32.0.5";
};
};
};
};
}

View file

@ -1,6 +0,0 @@
# Instead of pinning Nixpkgs, we can opt to use the one in NIX_PATH
import <nixpkgs> {
# We specify the architecture explicitly. Use a Linux remote builder when
# calling arion from other platforms.
system = "x86_64-linux";
}

View file

@ -1,107 +0,0 @@
{
"nodes": {
"flake-parts": {
"inputs": {
"nixpkgs-lib": [
"nixpkgs"
]
},
"locked": {
"lastModified": 1722555600,
"narHash": "sha256-XOQkdLafnb/p9ij77byFQjDf5m5QYl9b2REiVClC+x4=",
"owner": "hercules-ci",
"repo": "flake-parts",
"rev": "8471fe90ad337a8074e957b69ca4d0089218391d",
"type": "github"
},
"original": {
"owner": "hercules-ci",
"repo": "flake-parts",
"type": "github"
}
},
"flake-parts_2": {
"inputs": {
"nixpkgs-lib": [
"hercules-ci-effects",
"nixpkgs"
]
},
"locked": {
"lastModified": 1712014858,
"narHash": "sha256-sB4SWl2lX95bExY2gMFG5HIzvva5AVMJd4Igm+GpZNw=",
"owner": "hercules-ci",
"repo": "flake-parts",
"rev": "9126214d0a59633752a136528f5f3b9aa8565b7d",
"type": "github"
},
"original": {
"id": "flake-parts",
"type": "indirect"
}
},
"haskell-flake": {
"locked": {
"lastModified": 1675296942,
"narHash": "sha256-u1X1sblozi5qYEcLp1hxcyo8FfDHnRUVX3dJ/tW19jY=",
"owner": "srid",
"repo": "haskell-flake",
"rev": "c2cafce9d57bfca41794dc3b99c593155006c71e",
"type": "github"
},
"original": {
"owner": "srid",
"ref": "0.1.0",
"repo": "haskell-flake",
"type": "github"
}
},
"hercules-ci-effects": {
"inputs": {
"flake-parts": "flake-parts_2",
"nixpkgs": [
"nixpkgs"
]
},
"locked": {
"lastModified": 1719226092,
"narHash": "sha256-YNkUMcCUCpnULp40g+svYsaH1RbSEj6s4WdZY/SHe38=",
"owner": "hercules-ci",
"repo": "hercules-ci-effects",
"rev": "11e4b8dc112e2f485d7c97e1cee77f9958f498f5",
"type": "github"
},
"original": {
"owner": "hercules-ci",
"repo": "hercules-ci-effects",
"type": "github"
}
},
"nixpkgs": {
"locked": {
"lastModified": 1722630782,
"narHash": "sha256-hMyG9/WlUi0Ho9VkRrrez7SeNlDzLxalm9FwY7n/Noo=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "d04953086551086b44b6f3c6b7eeb26294f207da",
"type": "github"
},
"original": {
"owner": "NixOS",
"ref": "nixos-unstable",
"repo": "nixpkgs",
"type": "github"
}
},
"root": {
"inputs": {
"flake-parts": "flake-parts",
"haskell-flake": "haskell-flake",
"hercules-ci-effects": "hercules-ci-effects",
"nixpkgs": "nixpkgs"
}
}
},
"root": "root",
"version": 7
}

View file

@ -1,98 +0,0 @@
{
description = "Arion - use Docker Compose via Nix";
inputs = {
nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
haskell-flake.url = "github:srid/haskell-flake/0.1.0";
flake-parts.url = "github:hercules-ci/flake-parts";
flake-parts.inputs.nixpkgs-lib.follows = "nixpkgs";
hercules-ci-effects.url = "github:hercules-ci/hercules-ci-effects";
hercules-ci-effects.inputs.nixpkgs.follows = "nixpkgs";
};
outputs = inputs@{ self, flake-parts, ... }:
flake-parts.lib.mkFlake { inherit inputs; } ({ config, lib, extendModules, ... }: {
imports = [
inputs.haskell-flake.flakeModule
inputs.hercules-ci-effects.flakeModule
inputs.flake-parts.flakeModules.easyOverlay
./docs/flake-module.nix
./tests/flake-module.nix
];
systems = inputs.nixpkgs.lib.systems.flakeExposed;
perSystem = { config, self', inputs', pkgs, system, final, ... }:
let h = pkgs.haskell.lib.compose; in
{
overlayAttrs = {
inherit (config.packages) arion;
arionTestingFlags = {
dockerSupportsSystemd = false;
};
};
packages.default = config.packages.arion;
packages.overlay-test = final.arion;
packages.arion = import ./nix/arion.nix { inherit pkgs; };
haskellProjects.haskell-package = {
# not autodetected: https://github.com/srid/haskell-flake/issues/49
packages.arion-compose.root = ./.;
overrides =
self: super: {
arion-compose =
lib.pipe super.arion-compose [
(h.addBuildTools [ pkgs.nix ])
(h.overrideCabal (o: {
src = pkgs.lib.sourceByRegex ./. [
".*[.]cabal"
"LICENSE"
"src/?.*"
"README.asciidoc"
"CHANGELOG.md"
];
preCheck = ''
export NIX_LOG_DIR=$TMPDIR
export NIX_STATE_DIR=$TMPDIR
export NIX_PATH=nixpkgs=${pkgs.path}
'';
}))
];
};
};
devShells.default = config.devShells.haskell-package.overrideAttrs (o: {
nativeBuildInputs = o.nativeBuildInputs or [ ] ++ [
pkgs.docker-compose
pkgs.nixpkgs-fmt
config.haskellProjects.haskell-package.haskellPackages.releaser
];
});
};
hercules-ci.flake-update = {
enable = true;
autoMergeMethod = "merge";
when = {
hour = [ 2 ];
dayOfMonth = [ 5 ];
};
};
herculesCI.ciSystems = [
# "aarch64-darwin"
# "aarch64-linux"
"x86_64-darwin"
"x86_64-linux"
];
flake = {
debug = { inherit inputs config lib; };
lib = {
eval = import ./src/nix/eval-composition.nix;
build = args@{ ... }:
let composition = self.lib.eval args;
in composition.config.out.dockerComposeYaml;
};
nixosModules.arion = ./nixos-module.nix;
};
});
}

View file

@ -1,13 +0,0 @@
#!/usr/bin/env nix-shell
#!nix-shell ./shell.nix
#!nix-shell -i bash
set -eux -o pipefail
cd "$(dirname "${BASH_SOURCE[0]}")"
ghcid \
--command 'cabal v2-repl arion-compose:arion-unit-tests --flags ghci --write-ghc-environment-files=never' \
--test=Main.main \
--reload=src/haskell \
--restart=arion-compose.cabal \
;

View file

@ -1,13 +0,0 @@
# Like the upstreamable expression but wired up for the local arion.
{ pkgs ? import ./. {}
, lib ? pkgs.lib
, haskell ? pkgs.haskell
, haskellPackages ? pkgs.haskellPackages
, arion-compose ? import ./haskell-arion-compose.nix { inherit pkgs haskellPackages; }
, runCommand ? pkgs.runCommand
}:
import ./upstreamable/default.nix {
inherit pkgs lib haskell runCommand;
haskellPackages = haskellPackages // { inherit arion-compose; };
evalSrc = ./..;
}

View file

@ -1,10 +0,0 @@
(import
(
let lock = builtins.fromJSON (builtins.readFile ./flake.lock); in
fetchTarball {
url = "https://github.com/edolstra/flake-compat/archive/009399224d5e398d03b22badca40a37ac85412a1.tar.gz";
sha256 = "sha256:0xcr9fibnapa12ywzcnlf54wrmbqqb96fmmv8043zhsycws7bpqy";
}
)
{ src = ../.; }
).defaultNix

13
nix/default.nix Normal file
View file

@ -0,0 +1,13 @@
/**
* This is the entry-point for all nix execution in this project.
*/
{ nixpkgsSrc ? ./nixpkgs.nix, ... }:
import (import ./nixpkgs.nix) {
# Makes the config pure as well. See <nixpkgs>/top-level/impure.nix:
config = {
};
overlays = [
# all the packages are defined there:
(import ./overlay.nix)
];
}

View file

@ -1,20 +0,0 @@
# NOTE: This file produces a haskell library, not the arion package!
{ pkgs ? import ./default.nix {}, haskellPackages ? pkgs.haskellPackages }:
let
inherit (pkgs.haskell.lib) overrideCabal addBuildTools;
in
overrideCabal (addBuildTools (haskellPackages.callCabal2nix "arion-compose" ./.. {}) [pkgs.nix]) (o: o // {
src = pkgs.lib.sourceByRegex ../. [
".*[.]cabal"
"LICENSE"
"src/?.*"
"README.asciidoc"
];
preCheck = ''
export NIX_LOG_DIR=$TMPDIR
export NIX_STATE_DIR=$TMPDIR
export NIX_PATH=nixpkgs=${pkgs.path}
'';
})

5
nix/nixpkgs.nix Normal file
View file

@ -0,0 +1,5 @@
# to update: $ nix-prefetch-url --unpack url
builtins.fetchTarball {
url = "https://github.com/NixOS/nixpkgs/archive/be445a9074f139d63e704fa82610d25456562c3d.tar.gz";
sha256 = "15dc7gdspimavcwyw9nif4s59v79gk18rwsafylffs9m1ld2dxwa";
}

5
nix/overlay.nix Normal file
View file

@ -0,0 +1,5 @@
self: super: {
arion = super.callPackage ../arion.nix {};
tests = super.callPackage ../tests {};
doc = super.callPackage ../doc {};
}

View file

@ -1,88 +0,0 @@
args@
{ pkgs
, lib
, haskellPackages
, haskell
, runCommand
# Allow this expression file to be used more efficiently in situations where
# the sources are more readily available. Unpacking haskellPackages.arion-compose.src
# is not always the best choice for arion.eval.
, evalSrc ? null
}:
let
/* This derivation builds the arion tool.
It is based on the arion-compose Haskell package, but adapted and extended to
- have the correct name
- have a smaller closure size
- have functions to use Arion from inside Nix: arion.eval and arion.build
- make it self-contained by including docker-compose
*/
arion =
justStaticExecutables (
overrideCabal
arion-compose
cabalOverrides
);
inherit (haskell.lib) justStaticExecutables overrideCabal;
inherit (haskellPackages) arion-compose;
cabalOverrides = o: {
buildTools = (o.buildTools or []) ++ [pkgs.makeWrapper];
passthru = (o.passthru or {}) // {
inherit eval build;
};
# Patch away the arion-compose name. Unlike the Haskell library, the program
# is called arion (arion was already taken on hackage).
pname = "arion";
src = arion-compose.src;
# PYTHONPATH
#
# We close off the python module search path!
#
# Accepting directories from the environment into the search path
# tends to break things. Docker Compose does not have a plugin
# system as far as I can tell, so I don't expect this to break a
# feature, but rather to make the program more robustly self-
# contained.
postInstall = ''${o.postInstall or ""}
mkdir -p $out/libexec
mv $out/bin/arion $out/libexec
makeWrapper $out/libexec/arion $out/bin/arion \
--unset PYTHONPATH \
--prefix PATH : ${lib.makeBinPath [ pkgs.docker-compose ]} \
;
'';
};
# Unpacked sources for evaluation by `eval`
evalSrc' = args.evalSrc or (runCommand "arion-src" {}
"mkdir $out; tar -C $out --strip-components=1 -xf ${arion-compose.src}");
/* Function for evaluating a composition
Re-uses this Nixpkgs evaluation instead of `arion-pkgs.nix`.
Returns the module system's `config` and `options` variables.
*/
eval = args@{...}:
import (evalSrc' + "/src/nix/eval-composition.nix")
({ inherit pkgs; } // args);
/* Function to derivation of the docker compose yaml file
NOTE: The output will change: https://github.com/hercules-ci/arion/issues/82
This function is particularly useful on CI.
*/
build = args@{...}:
let composition = eval args;
in composition.config.out.dockerComposeYaml;
in arion

View file

@ -1,118 +0,0 @@
{ config, lib, options, pkgs, ... }:
let
inherit (lib)
attrValues
mkIf
mkOption
mkMerge
types
;
cfg = config.virtualisation.arion;
projectType = types.submoduleWith {
modules = [ projectModule ];
};
projectModule = { config, name, ... }: {
options = {
settings = mkOption {
description = ''
Arion project definition, otherwise known as arion-compose.nix contents.
See <link xlink:href="https://docs.hercules-ci.com/arion/options/">https://docs.hercules-ci.com/arion/options/</link>.
'';
type = arionSettingsType name;
visible = "shallow";
};
_systemd = mkOption { internal = true; };
serviceName = mkOption {
description = "The name of the Arion project's systemd service";
type = types.str;
default = "arion-${name}";
};
};
config = {
_systemd.services.${config.serviceName} = {
wantedBy = [ "multi-user.target" ];
after = [ "sockets.target" ];
path = [
cfg.package
cfg.docker.client.package
];
environment.ARION_PREBUILT = config.settings.out.dockerComposeYaml;
script = ''
echo 1>&2 "docker compose file: $ARION_PREBUILT"
arion --prebuilt-file "$ARION_PREBUILT" up
'';
};
};
};
arionSettingsType = name:
(cfg.package.eval { modules = [{ project.name = lib.mkDefault name; }]; }).type or (
throw "lib.evalModules did not produce a type. Please upgrade Nixpkgs to nixos-unstable or >=nixos-21.11"
);
in
{
disabledModules = [ "virtualisation/arion.nix" ];
options = {
virtualisation.arion = {
backend = mkOption {
type = types.enum [ "podman-socket" "docker" ];
description = ''
Which container implementation to use.
'';
};
package = mkOption {
type = types.package;
default = (import ./. { inherit pkgs; }).arion;
description = ''
Arion package to use. This will provide <literal>arion</literal>
executable that starts the project.
It also must provide the arion <literal>eval</literal> function as
an attribute.
'';
};
docker.client.package = mkOption {
type = types.package;
internal = true;
};
projects = mkOption {
type = types.attrsOf projectType;
default = { };
description = ''
Arion projects to be run as a service.
'';
};
};
};
config = mkIf (cfg.projects != { }) (
mkMerge [
{
systemd = mkMerge (map (p: p._systemd) (attrValues cfg.projects));
}
(mkIf (cfg.backend == "podman-socket") {
virtualisation.docker.enable = false;
virtualisation.podman.enable = true;
virtualisation.podman.dockerSocket.enable = true;
virtualisation.podman.defaultNetwork =
if options?virtualisation.podman.defaultNetwork.settings
then { settings.dns_enabled = true; } # since 2023-01 https://github.com/NixOS/nixpkgs/pull/199965
else { dnsname.enable = true; }; # compat <2023
virtualisation.arion.docker.client.package = pkgs.docker-client;
})
(mkIf (cfg.backend == "docker") {
virtualisation.docker.enable = true;
virtualisation.arion.docker.client.package = pkgs.docker;
})
]
);
}

6
repl
View file

@ -1,6 +0,0 @@
#!/usr/bin/env nix-shell
#!nix-shell -i bash
# A Haskell REPL for hacking on the Haskell code
cabal new-repl --write-ghc-environment-files=never

View file

@ -1,15 +0,0 @@
#!/usr/bin/env nix-shell
#!nix-shell -i bash
#!nix-shell ./shell.nix
# For quick manual testing of a hacked arion
# NB: Only works inside the project directory
cabal \
new-run \
--write-ghc-environment-files=never \
:pkg:arion-compose:exe:arion \
-- \
"$@" \
;

View file

@ -1,15 +0,0 @@
#!/usr/bin/env bash
projectRoot="$(dirname ${BASH_SOURCE[0]})"
resultLink="$projectRoot/result-run-arion-quick"
[[ -e "$resultLink" ]] || {
echo 1>&2 "You don't have a prebuilt arion yet; building it."
nix-build "$projectRoot" -A arion --out-link "$resultLink"
}
echo 1>&2 "Note that you will need to rm '$resultLink' to rebuild the arion executable when needed."
export arion_compose_datadir="$projectRoot/src"
exec "$resultLink/bin/arion" "$@"

View file

@ -1,6 +0,0 @@
#!/usr/bin/env bash
# For manual testing of a hacked arion built via Nix.
# Works when called from outside the project directory.
exec nix run -f "$(dirname ${BASH_SOURCE[0]})" arion "$@"

View file

@ -1 +0,0 @@
(builtins.getFlake ("git+file://" + toString ./.)).devShells.${builtins.currentSystem}.default

317
src/arion Executable file
View file

@ -0,0 +1,317 @@
#!/usr/bin/env bash
# Close off the python module search path
#
# Accepting directories from the environment into the search path
# tends to break things. Docker Compose does not have a plugin
# system as far as I can tell, so I don't expect this to break a
# feature, but rather to make the program more robustly self-
# contained.
unset PYTHONPATH
set -euo pipefail
export PATH="@path@:$PATH"
nix_dir="@nix_dir@"
docker_compose_args=()
files=()
command="docker-compose"
pkgs_argument="./arion-pkgs.nix"
debug() {
# echo "$@"
:
}
while test $# != 0; do
case "$1" in
-f|--file)
shift
files+=("$1")
;;
-f*)
files+=("${1/#-f/}")
;;
--file=*)
files+=("${1/#--file=}")
;;
--pkgs)
shift
pkgs_argument="$1"
;;
-h|--help|help)
command="help"
shift
break
;;
cat)
command="$1"
shift
break
;;
repl)
command="$1"
shift
break
;;
exec)
command="$1"
shift
break
;;
docker-compose)
command="docker-compose"
shift
break
;;
*)
break
;;
esac
shift
done
while test $# != 0; do
docker_compose_args+=("$1")
shift
done
case "$command" in
help)
cat <<EOF
Arion wraps your system's docker-compose, providing a NixOps-like
experience for simple container deployments.
Usage:
arion up|logs|... - execute docker-compose commands
arion cat - display raw docker-compose.yaml
arion config - validate and display the config file
arion repl - explore the config interactively
arion help
arion docker-compose help
arion docker-compose help up|logs|...
Top-level arion options
These must be provided before the command.
--file FILE Use FILE instead of the default ./arion-compose.nix
Can be specified multiple times for a merged configuration.
--pkgs EXPR Use EXPR instead of ./arion-pkgs.nix to get the
Nixpkgs attrset used for bootstrapping and evaluating
the configuration.
EOF
exit 0
;;
*)
;;
esac
if [[ ${#files[@]} == 0 ]]; then
files=("./arion-compose.nix")
fi
debug docker_compose_args: "${docker_compose_args[@]}"
debug files: "${files[@]}"
docker_compose_yaml=.tmp-nix-docker-compose-$$-$RANDOM.yaml
cleanup() {
rm -f $docker_compose_yaml
}
trap cleanup EXIT
modules="["
for file in "${files[@]}"; do
case "$file" in
/*)
modules="$modules (/. + $(printf '"%q"' "$file"))"
;;
*)
modules="$modules (./. + $(printf '"/%q"' "$file"))"
;;
esac
done
modules="$modules ]"
debug modules: "$modules"
old_IFS="$IFS"
IFS=""
args=(
)
IFS="$old_IFS"
for arg in "${args[@]}"; do
echo "arg: $arg"
done
do_eval() {
echo 1>&2 "Evaluating configuration..."
# read-write-mode is required for import from derivation
nix-instantiate \
"$nix_dir/eval-composition.nix" \
--eval \
--read-write-mode \
--json \
--argstr uid "$UID" \
--arg modules "$modules" \
--arg pkgs "$pkgs_argument" \
--show-trace \
--attr 'config.build.dockerComposeYamlText' \
| jq -r . >$docker_compose_yaml;
}
do_build() {
echo 1>&2 "Building configuration..."
nix-build \
"$nix_dir/eval-composition.nix" \
--out-link $docker_compose_yaml \
--argstr uid "$UID" \
--arg modules "$modules" \
--arg pkgs "$pkgs_argument" \
--show-trace \
--attr 'config.build.dockerComposeYaml' \
>/dev/null ;
echo 1>&2 "Ensuring required images are loaded..."
jq -r <"$docker_compose_yaml" \
'.["x-arion"].images | map(" - " + .imageName + ":" + .imageTag) | join("\n")'
eval "$(
jq -r '.["docker-compose"]["x-arion"].images as $images
| .["existing-images"] as $loaded
| $images
| map(
if $loaded[.imageName + ":" + .imageTag]
then ""
else "docker load <" + .image + ";" end
)
| join("\n")
' <<EOF
{
"docker-compose": $(cat $docker_compose_yaml),
"existing-images": {
$(docker images \
--filter "dangling=false" \
--format '"{{.Repository}}:{{.Tag}}": true,')
"": false
}
}
EOF
)"
}
do_repl() {
# nix repl doesn't autocall its <FILES> arguments
# so we improvise. We need a file in this directory
# to make sure that all paths are as expected :(
trap do_repl_cleanup EXIT;
REPL_TMP=.tmp-repl-$$-$RANDOM
cat <<EOF
Launch a repl for you, using a temporary file: $REPL_TMP.
This loads the configuration from the modules
${files[*]}
To get started:
To see deployment-wide configuration
type config. and hit TAB
To see the services
type config.docker-compose.evaluatedServices TAB or ENTER
To bring the top-level Nixpkgs attributes into scope
type :a (config._module.args.pkgs) // { inherit config; }
EOF
cat >"$REPL_TMP" <<EOF
import $nix_dir/eval-composition.nix {
uid = "$UID";
modules = $modules;
pkgs = $pkgs_argument;
}
EOF
nix repl \
"$REPL_TMP" \
;
}
do_repl_cleanup() {
rm -f $REPL_TMP
}
run_exec() {
case "${#docker_compose_args[@]}" in
0)
echo "As an argument to exec, please specify a service"
exit 1
;;
1)
case "${docker_compose_args[0]}" in
-*|--*)
echo "As an argument to exec, please specify a service"
echo "Note that executing the default command currently does not support"
echo "docker-compose options."
# This requires parsing the options, in order to figure out
# which service to invoke.
exit 1
;;
*)
serviceName="${docker_compose_args[0]}"
do_eval
default_args=()
while read arg; do
default_args+=("$arg")
done < <(
jq < "$docker_compose_yaml" \
--arg serviceName "$serviceName" \
-r \
'.["x-arion"].serviceInfo[$serviceName].defaultExec | tostream | .[1] | select(.)'
)
docker-compose -f $docker_compose_yaml exec "$serviceName" "${default_args[@]}"
;;
esac
;;
*)
do_eval
docker-compose -f $docker_compose_yaml exec "${docker_compose_args[@]}"
;;
esac
}
case "$command" in
cat)
do_eval
jq . < "$docker_compose_yaml"
;;
repl)
do_repl
;;
exec)
run_exec "$@"
;;
docker-compose)
if [[ ${#docker_compose_args[@]} != 0
&& ${docker_compose_args[0]} != "help"
&& ${docker_compose_args[0]} != "version"
]]; then
case "${docker_compose_args[0]}" in
help|version)
:
;;
config|down|events|exec|images|kill|logs|pause|port|ps|rm|stop|top|unpause)
do_eval
;;
*)
do_build
;;
esac
fi
docker-compose -f $docker_compose_yaml "${docker_compose_args[@]}"
;;
esac

View file

@ -1,5 +1,3 @@
FROM scratch FROM scratch
# scratch itself can't be run. COPY passwd /etc/passwd
ADD tarball.tar.gz /
# This seems like a no-op:
CMD []

2
src/arion-image/passwd Normal file
View file

@ -0,0 +1,2 @@
root:x:0:0:System administrator:/root:/bin/sh
nobody:x:65534:65534:Unprivileged account (don't use!):/var/empty:/run/current-system/sw/bin/nologin

View file

@ -0,0 +1 @@
/run/system/bin/sh

View file

@ -0,0 +1 @@
/run/system/usr/bin/env

View file

@ -1,324 +0,0 @@
{-# LANGUAGE NoImplicitPrelude #-}
{-# LANGUAGE RecordWildCards #-}
{-# LANGUAGE ApplicativeDo #-}
{-# LANGUAGE OverloadedStrings #-}
import Protolude hiding (Down, option)
import Arion.Nix
import Arion.Aeson
import Arion.Images (loadImages)
import qualified Arion.DockerCompose as DockerCompose
import Arion.Services (getDefaultExec)
import Arion.ExtendedInfo (loadExtendedInfoFromPath, ExtendedInfo(images, projectName))
import Options.Applicative
import Control.Monad.Fail
import qualified Data.Text as T
import qualified Data.Text.IO as T
import Data.Aeson(Value)
import System.Posix.User (getRealUserID)
data CommonOptions =
CommonOptions
{ files :: NonEmpty FilePath
, pkgs :: Text
, nixArgs :: [Text]
, prebuiltComposeFile :: Maybe FilePath
, noAnsi :: Bool
, compatibility :: Bool
, logLevel :: Maybe Text
}
deriving (Show)
newtype DockerComposeArgs =
DockerComposeArgs { unDockerComposeArgs :: [Text] }
ensureConfigFile :: [FilePath] -> NonEmpty FilePath
ensureConfigFile [] = "./arion-compose.nix" :| []
ensureConfigFile (x:xs) = x :| xs
parseOptions :: Parser CommonOptions
parseOptions = do
files <-
ensureConfigFile <$>
many (strOption
( short 'f'
<> long "file"
<> metavar "FILE"
<> help "Use FILE instead of the default ./arion-compose.nix. \
\Can be specified multiple times for a merged configuration" ))
pkgs <- T.pack <$> strOption
( short 'p'
<> long "pkgs"
<> metavar "EXPR"
<> showDefault
<> value "./arion-pkgs.nix"
<> help "Use Nix expression EXPR to get the Nixpkgs attrset used for bootstrapping \
\and evaluating the configuration." )
showTrace <- flag False True (long "show-trace"
<> help "Causes Nix to print out a stack trace in case of Nix expression evaluation errors. Specify before command.")
-- TODO --option support (https://github.com/pcapriotti/optparse-applicative/issues/284)
userNixArgs <- many (T.pack <$> strOption (long "nix-arg" <> metavar "ARG" <> help "Pass an extra argument to nix. Example: --nix-arg --option --nix-arg substitute --nix-arg false"))
prebuiltComposeFile <- optional $ strOption
( long "prebuilt-file"
<> metavar "JSONFILE"
<> help "Do not evaluate and use the prebuilt JSONFILE instead. Causes other evaluation-related options to be ignored." )
noAnsi <- flag False True (long "no-ansi"
<> help "Avoid ANSI control sequences")
compatibility <- flag False True (long "no-ansi"
<> help "If set, Docker Compose will attempt to convert deploy keys in v3 files to their non-Swarm equivalent")
logLevel <- optional $ fmap T.pack $ strOption (long "log-level" <> metavar "LEVEL" <> help "Set log level (DEBUG, INFO, WARNING, ERROR, CRITICAL)")
pure $
let nixArgs = userNixArgs <|> "--show-trace" <$ guard showTrace
in CommonOptions{..}
textArgument :: Mod ArgumentFields [Char] -> Parser Text
textArgument = fmap T.pack . strArgument
parseCommand :: Parser (CommonOptions -> IO ())
parseCommand =
hsubparser
( command "cat" (info (pure runCat) (progDesc "Spit out the docker compose file as JSON" <> fullDesc))
<> command "repl" (info (pure runRepl) (progDesc "Start a nix repl for the whole composition" <> fullDesc))
<> command "exec" (info (parseExecCommand) (progDesc "Execute a command in a running container" <> fullDesc))
)
<|>
hsubparser
( commandDC runBuildAndDC "build" "Build or rebuild services"
<> commandDC runBuildAndDC "bundle" "Generate a Docker bundle from the Compose file"
<> commandDC runEvalAndDC "config" "Validate and view the Compose file"
<> commandDC runBuildAndDC "create" "Create services"
<> commandDC runEvalAndDC "down" "Stop and remove containers, networks, images, and volumes"
<> commandDC runEvalAndDC "events" "Receive real time events from containers"
<> commandDC runDC "help" "Get help on a command"
<> commandDC runEvalAndDC "images" "List images"
<> commandDC runEvalAndDC "kill" "Kill containers"
<> commandDC runEvalAndDC "logs" "View output from containers"
<> commandDC runEvalAndDC "pause" "Pause services"
<> commandDC runEvalAndDC "port" "Print the public port for a port binding"
<> commandDC runEvalAndDC "ps" "List containers"
<> commandDC runBuildAndDC "pull" "Pull service images"
<> commandDC runBuildAndDC "push" "Push service images"
<> commandDC runBuildAndDC "restart" "Restart services"
<> commandDC runEvalAndDC "rm" "Remove stopped containers"
<> commandDC runBuildAndDC "run" "Run a one-off command"
<> commandDC runBuildAndDC "scale" "Set number of containers for a service"
<> commandDC runBuildAndDC "start" "Start services"
<> commandDC runEvalAndDC "stop" "Stop services"
<> commandDC runEvalAndDC "top" "Display the running processes"
<> commandDC runEvalAndDC "unpause" "Unpause services"
<> commandDC runBuildAndDC "up" "Create and start containers"
<> commandDC runDC "version" "Show the Docker-Compose version information"
<> metavar "DOCKER-COMPOSE-COMMAND"
<> commandGroup "Docker Compose Commands:"
)
parseAll :: Parser (IO ())
parseAll =
flip ($) <$> parseOptions <*> parseCommand
parseDockerComposeArgs :: Parser DockerComposeArgs
parseDockerComposeArgs =
DockerComposeArgs <$>
many (argument (T.pack <$> str) (metavar "DOCKER-COMPOSE ARGS..."))
commandDC
:: (Text -> DockerComposeArgs -> CommonOptions -> IO ())
-> Text
-> Text
-> Mod CommandFields (CommonOptions -> IO ())
commandDC run cmdStr helpText =
command
(T.unpack cmdStr)
(info
(run cmdStr <$> parseDockerComposeArgs)
(progDesc (T.unpack helpText) <> fullDesc <> forwardOptions))
--------------------------------------------------------------------------------
runDC :: Text -> DockerComposeArgs -> CommonOptions -> IO ()
runDC cmd (DockerComposeArgs args) _opts = do
DockerCompose.run DockerCompose.Args
{ files = []
, otherArgs = [cmd] ++ args
}
runBuildAndDC :: Text -> DockerComposeArgs -> CommonOptions -> IO ()
runBuildAndDC cmd dopts opts = do
withBuiltComposeFile opts $ callDC cmd dopts opts True
runEvalAndDC :: Text -> DockerComposeArgs -> CommonOptions -> IO ()
runEvalAndDC cmd dopts opts = do
withComposeFile opts $ callDC cmd dopts opts False
callDC :: Text -> DockerComposeArgs -> CommonOptions -> Bool -> FilePath -> IO ()
callDC cmd dopts opts shouldLoadImages path = do
extendedInfo <- loadExtendedInfoFromPath path
when shouldLoadImages $ loadImages (images extendedInfo)
let firstOpts = projectArgs extendedInfo <> commonArgs opts
DockerCompose.run DockerCompose.Args
{ files = [path]
, otherArgs = firstOpts ++ [cmd] ++ unDockerComposeArgs dopts
}
projectArgs :: ExtendedInfo -> [Text]
projectArgs extendedInfo =
do
n <- toList (projectName extendedInfo)
["--project-name", n]
commonArgs :: CommonOptions -> [Text]
commonArgs opts = do
guard (noAnsi opts)
["--no-ansi"]
<> do
guard (compatibility opts)
["--compatibility"]
<> do
l <- toList (logLevel opts)
["--log-level", l]
withBuiltComposeFile :: CommonOptions -> (FilePath -> IO r) -> IO r
withBuiltComposeFile opts cont = case prebuiltComposeFile opts of
Just prebuilt -> do
cont prebuilt
Nothing -> do
args <- defaultEvaluationArgs opts
Arion.Nix.withBuiltComposition args cont
withComposeFile :: CommonOptions -> (FilePath -> IO r) -> IO r
withComposeFile opts cont = case prebuiltComposeFile opts of
Just prebuilt -> do
cont prebuilt
Nothing -> do
args <- defaultEvaluationArgs opts
Arion.Nix.withEvaluatedComposition args cont
getComposeValue :: CommonOptions -> IO Value
getComposeValue opts = case prebuiltComposeFile opts of
Just prebuilt -> do
decodeFile prebuilt
Nothing -> do
args <- defaultEvaluationArgs opts
Arion.Nix.evaluateComposition args
defaultEvaluationArgs :: CommonOptions -> IO EvaluationArgs
defaultEvaluationArgs co = do
uid <- getRealUserID
pure EvaluationArgs
{ evalUid = fromIntegral uid
, evalModules = files co
, evalPkgs = pkgs co
, evalWorkDir = Nothing
, evalMode = ReadWrite
, evalUserArgs = nixArgs co
}
runCat :: CommonOptions -> IO ()
runCat co = do
v <- getComposeValue co
T.hPutStrLn stdout (pretty v)
runRepl :: CommonOptions -> IO ()
runRepl co = do
putErrText
"Launching a repl for you. To get started:\n\
\\n\
\To see deployment-wide configuration\n\
\ type config. and use tab completion\n\
\To bring the top-level Nixpkgs attributes into scope\n\
\ type :a (config._module.args.pkgs) // { inherit config; }\n\
\"
Arion.Nix.replForComposition =<< defaultEvaluationArgs co
detachFlag :: Parser Bool
detachFlag = flag False True (long "detach" <> short 'd' <> help "Detached mode: Run command in the background.")
privilegedFlag :: Parser Bool
privilegedFlag = flag False True (long "privileged" <> help "Give extended privileges to the process.")
userOption :: Parser Text
userOption = strOption (long "user" <> short 'u' <> help "Run the command as this user.")
noTTYFlag :: Parser Bool
noTTYFlag = flag False True (short 'T' <> help "Disable pseudo-tty allocation. By default `exec` allocates a TTY.")
indexOption :: Parser Int
indexOption = option
(auto >>= \i -> i <$ unless (i >= 1) (fail "container index must be >= 1"))
(long "index" <> value 1 <> help "Index of the container if there are multiple instances of a service.")
envOption :: Parser (Text, Text)
envOption = option (auto >>= spl) (long "env" <> short 'e' <> help "Set environment variables (can be used multiple times, not supported in Docker API < 1.25)")
where spl s = case T.break (== '=') s of
(_, "") -> fail "--env parameter needs to combine key and value with = sign"
(k, ev) -> pure (k, T.drop 1 ev)
workdirOption :: Parser Text
workdirOption = strOption (long "workdir" <> short 'w' <> metavar "DIR" <> help "Working directory in which to start the command in the container.")
parseExecCommand :: Parser (CommonOptions -> IO ())
parseExecCommand = runExec
<$> detachFlag
<*> privilegedFlag
<*> optional userOption
<*> noTTYFlag
<*> indexOption
<*> many envOption
<*> optional workdirOption
<*> textArgument (metavar "SERVICE")
<*> orEmpty' (
(:) <$> argument (T.pack <$> str) (metavar "COMMAND")
<*> many (argument (T.pack <$> str) (metavar "ARG"))
)
orEmpty' :: (Alternative f, Monoid a) => f a -> f a
orEmpty' m = fromMaybe mempty <$> optional m
runExec :: Bool -> Bool -> Maybe Text -> Bool -> Int -> [(Text, Text)] -> Maybe Text -> Text -> [Text] -> CommonOptions -> IO ()
runExec detach privileged user noTTY index envs workDir service commandAndArgs opts =
withComposeFile opts $ \path -> do
extendedInfo <- loadExtendedInfoFromPath path
commandAndArgs'' <- case commandAndArgs of
[] -> do
cmd <- getDefaultExec path service
case cmd of
[] -> do
putErrText "You must provide a command via service.defaultExec or on the command line."
exitFailure
_ ->
pure cmd
x -> pure x
let commandAndArgs' = case commandAndArgs'' of
[] -> ["/bin/sh"]
x -> x
let args = concat
[ ["exec"]
, ("--detach" <$ guard detach :: [Text])
, "--privileged" <$ guard privileged
, "-T" <$ guard noTTY
, (\(k, v) -> ["--env", k <> "=" <> v]) =<< envs
, join $ toList (user <&> \u -> ["--user", u])
, ["--index", show index]
, join $ toList (workDir <&> \w -> ["--workdir", w])
, [service]
, commandAndArgs'
]
DockerCompose.run DockerCompose.Args
{ files = [path]
, otherArgs = projectArgs extendedInfo <> commonArgs opts <> args
}
main :: IO ()
main =
(join . arionExecParser) (info (parseAll <**> helper) fullDesc)
where
arionExecParser = customExecParser (prefs showHelpOnEmpty)

View file

@ -1,27 +0,0 @@
module Arion.Aeson where
import Prelude ()
import Data.Aeson
import qualified Data.ByteString.Lazy as BL
import qualified Data.Text.Lazy as TL
import qualified Data.Text.Lazy.Builder as TB
import qualified Data.Aeson.Encode.Pretty
import Data.Aeson.Encode.Pretty ( defConfig
, confCompare
, confTrailingNewline
)
import Protolude
pretty :: ToJSON a => a -> Text
pretty =
TL.toStrict
. TB.toLazyText
. Data.Aeson.Encode.Pretty.encodePrettyToTextBuilder' config
where config = defConfig { confCompare = compare, confTrailingNewline = True }
decodeFile :: FromJSON a => FilePath -> IO a
decodeFile fp = do
b <- BL.readFile fp
case eitherDecode b of
Left e -> panic (toS e)
Right v -> pure v

View file

@ -1,30 +0,0 @@
{-# LANGUAGE OverloadedStrings #-}
module Arion.DockerCompose where
import Prelude ( )
import Protolude
import System.Process
data Args = Args
{ files :: [FilePath]
, otherArgs :: [Text]
}
run :: Args -> IO ()
run args = do
let fileArgs = files args >>= \f -> ["--file", f]
allArgs = fileArgs ++ map toS (otherArgs args)
procSpec = proc "docker-compose" allArgs
-- hPutStrLn stderr ("Running docker-compose with " <> show allArgs :: Text)
withCreateProcess procSpec $ \_in _out _err procHandle -> do
exitCode <- waitForProcess procHandle
case exitCode of
ExitSuccess -> pass
ExitFailure 1 -> exitFailure
ExitFailure {} -> do
throwIO $ FatalError $ "docker-compose failed with " <> show exitCode

View file

@ -1,37 +0,0 @@
{-# LANGUAGE DeriveAnyClass #-}
{-# LANGUAGE DeriveGeneric #-}
{-# LANGUAGE OverloadedStrings #-}
{-
Parses the x-arion field in the generated compose file.
-}
module Arion.ExtendedInfo where
import Prelude()
import Protolude
import Data.Aeson as Aeson
import Arion.Aeson
import Control.Lens
import Data.Aeson.Lens
data Image = Image
{ image :: Maybe Text -- ^ image tar.gz file path
, imageExe :: Maybe Text -- ^ path to exe producing image tar
, imageName :: Text
, imageTag :: Text
} deriving (Eq, Show, Generic, Aeson.ToJSON, Aeson.FromJSON)
data ExtendedInfo = ExtendedInfo {
projectName :: Maybe Text,
images :: [Image]
} deriving (Eq, Show)
loadExtendedInfoFromPath :: FilePath -> IO ExtendedInfo
loadExtendedInfoFromPath fp = do
v <- decodeFile fp
pure ExtendedInfo {
-- TODO: use aeson derived instance?
projectName = v ^? key "x-arion" . key "project" . key "name" . _String,
images = (v :: Aeson.Value) ^.. key "x-arion" . key "images" . _Array . traverse . _JSON
}

View file

@ -1,72 +0,0 @@
{-# LANGUAGE DeriveGeneric #-}
{-# LANGUAGE DeriveAnyClass #-}
{-# LANGUAGE OverloadedStrings #-}
module Arion.Images
( loadImages
) where
import Prelude()
import Protolude hiding (to)
import qualified System.Process as Process
import qualified Data.Text as T
import Arion.ExtendedInfo (Image(..))
type TaggedImage = Text
-- | Subject to change
loadImages :: [Image] -> IO ()
loadImages requestedImages = do
loaded <- getDockerImages
let
isNew i =
-- On docker, the image name is unmodified
(imageName i <> ":" <> imageTag i) `notElem` loaded
-- On podman, you used to automatically get a localhost prefix
-- however, since NixOS 22.05, this expected to be part of the name instead
&& ("localhost/" <> imageName i <> ":" <> imageTag i) `notElem` loaded
traverse_ loadImage . filter isNew $ requestedImages
loadImage :: Image -> IO ()
loadImage Image { image = Just imgPath, imageName = name } =
withFile (toS imgPath) ReadMode $ \fileHandle -> do
let procSpec = (Process.proc "docker" [ "load" ]) {
Process.std_in = Process.UseHandle fileHandle
}
Process.withCreateProcess procSpec $ \_in _out _err procHandle -> do
e <- Process.waitForProcess procHandle
case e of
ExitSuccess -> pass
ExitFailure code ->
panic $ "docker load failed with exit code " <> show code <> " for image " <> name <> " from path " <> imgPath
loadImage Image { imageExe = Just imgExe, imageName = name } = do
let loadSpec = (Process.proc "docker" [ "load" ]) { Process.std_in = Process.CreatePipe }
Process.withCreateProcess loadSpec $ \(Just inHandle) _out _err loadProcHandle -> do
let streamSpec = Process.proc (toS imgExe) []
Process.withCreateProcess streamSpec { Process.std_out = Process.UseHandle inHandle } $ \_ _ _ streamProcHandle ->
withAsync (Process.waitForProcess loadProcHandle) $ \loadExitAsync ->
withAsync (Process.waitForProcess streamProcHandle) $ \streamExitAsync -> do
r <- waitEither loadExitAsync streamExitAsync
case r of
Right (ExitFailure code) -> panic $ "image producer for image " <> name <> " failed with exit code " <> show code <> " from executable " <> imgExe
Right ExitSuccess -> pass
Left _ -> pass
loadExit <- wait loadExitAsync
case loadExit of
ExitFailure code -> panic $ "docker load failed with exit code " <> show code <> " for image " <> name <> " produced by executable " <> imgExe
_ -> pass
pass
loadImage Image { imageName = name } = do
panic $ "image " <> name <> " doesn't specify an image file or imageExe executable"
getDockerImages :: IO [TaggedImage]
getDockerImages = do
let procSpec = Process.proc "docker" [ "images", "--filter", "dangling=false", "--format", "{{.Repository}}:{{.Tag}}" ]
map toS . T.lines . toS <$> Process.readCreateProcess procSpec ""

View file

@ -1,183 +0,0 @@
{-# LANGUAGE OverloadedStrings #-}
module Arion.Nix
( evaluateComposition
, withEvaluatedComposition
, buildComposition
, withBuiltComposition
, replForComposition
, EvaluationArgs(..)
, EvaluationMode(..)
) where
import Prelude ( )
import Protolude
import Arion.Aeson ( pretty )
import Data.Aeson
import qualified Data.String
import qualified System.Directory as Directory
import System.Process
import qualified Data.ByteString.Lazy as BL
import Paths_arion_compose
import qualified Data.Text.IO as T
import qualified Data.List.NonEmpty as NE
import Control.Arrow ( (>>>) )
import System.IO.Temp ( withTempFile )
import System.IO ( hClose )
data EvaluationMode =
ReadWrite | ReadOnly
data EvaluationArgs = EvaluationArgs
{ evalUid :: Int
, evalModules :: NonEmpty FilePath
, evalPkgs :: Text
, evalWorkDir :: Maybe FilePath
, evalMode :: EvaluationMode
, evalUserArgs :: [Text]
}
evaluateComposition :: EvaluationArgs -> IO Value
evaluateComposition ea = do
evalComposition <- getEvalCompositionFile
let commandArgs =
[ "--eval"
, "--strict"
, "--json"
, "--attr"
, "config.out.dockerComposeYamlAttrs"
]
args =
[ evalComposition ]
++ commandArgs
++ modeArguments (evalMode ea)
++ argArgs ea
++ map toS (evalUserArgs ea)
procSpec = (proc "nix-instantiate" args)
{ cwd = evalWorkDir ea
, std_out = CreatePipe
}
withCreateProcess procSpec $ \_in outHM _err procHandle -> do
let outHandle = fromMaybe (panic "stdout missing") outHM
out <- BL.hGetContents outHandle
v <- Protolude.evaluate (eitherDecode out)
exitCode <- waitForProcess procHandle
case exitCode of
ExitSuccess -> pass
ExitFailure 1 -> exitFailure
ExitFailure {} -> do
throwIO $ FatalError $ "evaluation failed with " <> show exitCode
case v of
Right r -> pure r
Left e -> throwIO $ FatalError ("Couldn't parse nix-instantiate output" <> show e)
-- | Run with docker-compose.yaml tmpfile
withEvaluatedComposition :: EvaluationArgs -> (FilePath -> IO r) -> IO r
withEvaluatedComposition ea f = do
v <- evaluateComposition ea
withTempFile "." ".tmp-arion-docker-compose.yaml" $ \path yamlHandle -> do
T.hPutStrLn yamlHandle (pretty v)
hClose yamlHandle
f path
buildComposition :: FilePath -> EvaluationArgs -> IO ()
buildComposition outLink ea = do
evalComposition <- getEvalCompositionFile
let commandArgs =
[ "--attr"
, "config.out.dockerComposeYaml"
, "--out-link"
, outLink
]
args =
[ evalComposition ]
++ commandArgs
++ argArgs ea
++ map toS (evalUserArgs ea)
procSpec = (proc "nix-build" args) { cwd = evalWorkDir ea }
withCreateProcess procSpec $ \_in _out _err procHandle -> do
exitCode <- waitForProcess procHandle
case exitCode of
ExitSuccess -> pass
ExitFailure 1 -> exitFailure
ExitFailure {} -> do
throwIO $ FatalError $ "nix-build failed with " <> show exitCode
-- | Do something with a docker-compose.yaml.
withBuiltComposition :: EvaluationArgs -> (FilePath -> IO r) -> IO r
withBuiltComposition ea f = do
withTempFile "." ".tmp-arion-docker-compose.yaml" $ \path emptyYamlHandle -> do
hClose emptyYamlHandle
-- Known problem: kills atomicity of withTempFile; won't fix because we should manage gc roots,
-- impl of which will probably avoid this "problem". It seems unlikely to cause issues.
Directory.removeFile path
buildComposition path ea
f path
replForComposition :: EvaluationArgs -> IO ()
replForComposition ea = do
evalComposition <- getEvalCompositionFile
let args =
[ "repl", evalComposition ]
++ argArgs ea
++ map toS (evalUserArgs ea)
procSpec = (proc "nix" args) { cwd = evalWorkDir ea }
withCreateProcess procSpec $ \_in _out _err procHandle -> do
exitCode <- waitForProcess procHandle
case exitCode of
ExitSuccess -> pass
ExitFailure 1 -> exitFailure
ExitFailure {} -> do
throwIO $ FatalError $ "nix repl failed with " <> show exitCode
argArgs :: EvaluationArgs -> [[Char]]
argArgs ea =
[ "--argstr"
, "uid"
, show $ evalUid ea
, "--arg"
, "modules"
, modulesNixExpr $ evalModules ea
, "--arg"
, "pkgs"
, toS $ evalPkgs ea
]
getEvalCompositionFile :: IO FilePath
getEvalCompositionFile = getDataFileName "nix/eval-composition.nix"
modeArguments :: EvaluationMode -> [[Char]]
modeArguments ReadWrite = [ "--read-write-mode" ]
modeArguments ReadOnly = [ "--readonly-mode" ]
modulesNixExpr :: NonEmpty FilePath -> [Char]
modulesNixExpr =
NE.toList >>> fmap pathExpr >>> Data.String.unwords >>> wrapList
where
pathExpr :: FilePath -> [Char]
pathExpr path | isAbsolute path = "(/. + \"/${" <> toNixStringLiteral path <> "}\")"
| otherwise = "(./. + \"/${" <> toNixStringLiteral path <> "}\")"
isAbsolute ('/' : _) = True
isAbsolute _ = False
wrapList s = "[ " <> s <> " ]"
toNixStringLiteral :: [Char] -> [Char]
toNixStringLiteral = show -- FIXME: custom escaping including '$'

View file

@ -1,37 +0,0 @@
{-# LANGUAGE DeriveGeneric #-}
{-# LANGUAGE DeriveAnyClass #-}
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE CPP #-}
module Arion.Services
( getDefaultExec
) where
import Prelude()
import Protolude hiding (to)
import qualified Data.Aeson as Aeson
#if MIN_VERSION_lens_aeson(1,2,0)
import qualified Data.Aeson.Key as AK
#endif
import Arion.Aeson (decodeFile)
import Control.Lens
import Data.Aeson.Lens
#if MIN_VERSION_lens_aeson(1,2,0)
type Key = AK.Key
mkKey :: Text -> Key
mkKey = AK.fromText
#else
type Key = Text
mkKey :: Text -> Key
mkKey = identity
#endif
-- | Subject to change
getDefaultExec :: FilePath -> Text -> IO [Text]
getDefaultExec fp service = do
v <- decodeFile fp
pure ((v :: Aeson.Value) ^.. key "x-arion" . key "serviceInfo" . key (mkKey service) . key "defaultExec" . _Array . traverse . _String)

View file

@ -1,74 +0,0 @@
{-# LANGUAGE OverloadedStrings #-}
module Arion.NixSpec
( spec
)
where
import Protolude
import Test.Hspec
import qualified Data.List.NonEmpty as NEL
import Arion.Aeson
import Arion.Nix
import qualified Data.Text as T
import qualified Data.Text.IO as T
spec :: Spec
spec = describe "evaluateComposition" $ do
it "matches an example" $ do
x <- Arion.Nix.evaluateComposition EvaluationArgs
{ evalUid = 123
, evalModules = NEL.fromList
["src/haskell/testdata/Arion/NixSpec/arion-compose.nix"]
, evalPkgs = "import <nixpkgs> { system = \"x86_64-linux\"; }"
, evalWorkDir = Nothing
, evalMode = ReadOnly
, evalUserArgs = ["--show-trace"]
}
let actual = pretty x
expected <- T.readFile "src/haskell/testdata/Arion/NixSpec/arion-compose.json"
censorPaths actual `shouldBe` censorPaths expected
it "matches an build.context example" $ do
x <- Arion.Nix.evaluateComposition EvaluationArgs
{ evalUid = 1234
, evalModules = NEL.fromList
["src/haskell/testdata/Arion/NixSpec/arion-context-compose.nix"]
, evalPkgs = "import <nixpkgs> { system = \"x86_64-linux\"; }"
, evalWorkDir = Nothing
, evalMode = ReadOnly
, evalUserArgs = ["--show-trace"]
}
let actual = pretty x
expected <- T.readFile "src/haskell/testdata/Arion/NixSpec/arion-context-compose.json"
censorPaths actual `shouldBe` censorPaths expected
censorPaths :: Text -> Text
censorPaths = censorImages . censorStorePaths
censorStorePaths :: Text -> Text
censorStorePaths x = case T.breakOn "/nix/store/" x of
(prefix, tl) | (tl :: Text) == "" -> prefix
(prefix, tl) -> prefix <> "<STOREPATH>" <> censorPaths
(T.dropWhile isNixNameChar $ T.drop (T.length "/nix/store/") tl)
-- Probably slow, due to not O(1) <>
censorImages :: Text -> Text
censorImages x = case T.break (\c -> c == ':' || c == '"') x of
(prefix, tl) | tl == "" -> prefix
(prefix, tl) | let imageId = T.take 33 (T.drop 1 tl)
, T.last imageId == '\"'
-- Approximation of nix hash validation
, T.all (\c -> (c >= '0' && c <= '9') || (c >= 'a' && c <= 'z')) (T.take 32 imageId)
-> prefix <> T.take 1 tl <> "<HASH>" <> censorImages (T.drop 33 tl)
(prefix, tl) -> prefix <> T.take 1 tl <> censorImages (T.drop 1 tl)
-- | WARNING: THIS IS LIKELY WRONG: DON'T REUSE
isNixNameChar :: Char -> Bool
isNixNameChar c | c >= '0' && c <= '9' = True
isNixNameChar c | c >= 'a' && c <= 'z' = True
isNixNameChar c | c >= 'A' && c <= 'Z' = True
isNixNameChar c | c == '-' = True
isNixNameChar c | c == '.' = True
isNixNameChar c | c == '_' = True -- WRONG?
isNixNameChar _ = False -- WRONG?

View file

@ -1,12 +0,0 @@
module Spec
( spec
)
where
import Test.Hspec
import qualified Arion.NixSpec
spec :: Spec
spec = do
describe "Arion.Nix" Arion.NixSpec.spec

View file

@ -1,10 +0,0 @@
module Main where
import Prelude()
import Protolude
import Test.Hspec.Runner
import qualified Spec
main :: IO ()
main = hspecWith config Spec.spec
where config = defaultConfig { configColorMode = ColorAlways }

View file

@ -1,57 +0,0 @@
{
"networks": {
"default": {
"name": "unit-test-data"
}
},
"services": {
"webserver": {
"command": [
"/usr/sbin/init"
],
"environment": {
"NIX_REMOTE": "",
"PATH": "/usr/bin:/run/current-system/sw/bin/",
"container": "docker"
},
"image": "localhost/webserver:<HASH>",
"ports": [
"8000:80"
],
"stop_signal": "SIGRTMIN+3",
"sysctls": {},
"tmpfs": [
"/run",
"/run/wrappers",
"/tmp:exec,mode=777"
],
"tty": true,
"volumes": [
"/sys/fs/cgroup:/sys/fs/cgroup:ro",
"/nix/store:/nix/store:ro"
]
}
},
"version": "3.4",
"volumes": {},
"x-arion": {
"images": [
{
"imageExe": "<STOREPATH>",
"imageName": "localhost/webserver",
"imageTag": "<HASH>"
}
],
"project": {
"name": "unit-test-data"
},
"serviceInfo": {
"webserver": {
"defaultExec": [
"/run/current-system/sw/bin/bash",
"-l"
]
}
}
}
}

View file

@ -1,13 +0,0 @@
{
project.name = "unit-test-data";
services.webserver = { pkgs, ... }: {
nixos.useSystemd = true;
nixos.configuration.boot.tmp.useTmpfs = true;
nixos.configuration.services.nginx.enable = true;
nixos.configuration.services.nginx.virtualHosts.localhost.root = "${pkgs.nix.doc}/share/doc/nix/manual";
service.useHostStore = true;
service.ports = [
"8000:80" # host:container
];
};
}

View file

@ -1,41 +0,0 @@
{
"networks": {
"default": {
"name": "unit-test-data"
}
},
"services": {
"webserver": {
"build": {
"context": "<STOREPATH>"
},
"environment": {},
"ports": [
"8080:80"
],
"sysctls": {},
"volumes": []
}
},
"version": "3.4",
"volumes": {},
"x-arion": {
"images": [
{
"imageExe": "<STOREPATH>",
"imageName": "localhost/webserver",
"imageTag": "<HASH>"
}
],
"project": {
"name": "unit-test-data"
},
"serviceInfo": {
"webserver": {
"defaultExec": [
"/bin/sh"
]
}
}
}
}

View file

@ -1,9 +0,0 @@
{
project.name = "unit-test-data";
services.webserver.service = {
build.context = "${./build-context}";
ports = [
"8080:80"
];
};
}

View file

@ -1,4 +0,0 @@
FROM nginx
RUN echo this is a dockerfile to be built

View file

@ -1,45 +0,0 @@
{
"services": {
"webserver": {
"environment": {
"container": "docker"
},
"image": "webserver:xr4ljmz3qfcwlq9rl4mr4qdrzw93rl70",
"ports": [
"8000:80"
],
"stop_signal": "SIGRTMIN+3",
"sysctls": {},
"tmpfs": [
"/run",
"/run/wrappers",
"/tmp:exec,mode=777"
],
"tty": true,
"volumes": [
"/sys/fs/cgroup:/sys/fs/cgroup:ro"
]
}
},
"version": "3.4",
"x-arion": {
"images": [
{
"image": "/nix/store/xr4ljmz3qfcwlq9rl4mr4qdrzw93rl70-docker-image-webserver.tar.gz",
"imageName": "webserver",
"imageTag": "xr4ljmz3qfcwlq9rl4mr4qdrzw93rl70"
}
],
"project": {
"name": null
},
"serviceInfo": {
"webserver": {
"defaultExec": [
"/run/current-system/sw/bin/bash",
"-l"
]
}
}
}
}

View file

@ -12,26 +12,26 @@ let
inherit (pkgs) lib; inherit (pkgs) lib;
composition = lib.evalModules { composition = lib.evalModules {
check = true;
modules = builtinModules ++ modules; modules = builtinModules ++ modules;
}; };
builtinModules = [ builtinModules = [
argsModule argsModule
] ++ import ./modules.nix; ./modules/composition/docker-compose.nix
./modules/composition/host-environment.nix
./modules/composition/images.nix
./modules/composition/service-info.nix
];
argsModule = { argsModule = {
_file = ./eval-composition.nix; _file = ./eval-composition.nix;
key = ./eval-composition.nix; key = ./eval-composition.nix;
config._module.args.pkgs = lib.mkIf (pkgs != null) (lib.mkForce pkgs); config._module.args.pkgs = lib.mkIf (pkgs != null) (lib.mkForce pkgs);
config._module.args.check = true;
config.host.nixStorePrefix = hostNixStorePrefix; config.host.nixStorePrefix = hostNixStorePrefix;
config.host.uid = lib.toInt uid; config.host.uid = lib.toInt uid;
}; };
in in
# Typically you need composition.config.out.dockerComposeYaml # Typically you need composition.config.build.dockerComposeYaml
composition // { composition
# throw in lib and pkgs for repl convenience
inherit lib;
inherit (composition._module.args) pkgs;
}

32
src/nix/eval-service.nix Normal file
View file

@ -0,0 +1,32 @@
{ lib, pkgs, ... }:
{ modules, host, name }:
let
composite = lib.evalModules {
check = true;
modules = builtinModules ++ modules;
};
builtinModules = [
argsModule
./modules/service/default-exec.nix
./modules/service/docker-compose-service.nix
./modules/service/extended-info.nix
./modules/service/host-store.nix
./modules/service/host.nix
./modules/service/image.nix
./modules/service/nixos.nix
./modules/service/nixos-init.nix
./modules/service/ngrok.nix
];
argsModule = {
_file = ./eval-service.nix;
key = ./eval-service.nix;
config._module.args.pkgs = lib.mkForce pkgs;
config.host = host;
config.service.name = name;
};
in
composite

View file

@ -1,21 +0,0 @@
{ lib }:
let
link = url: text: ''[${text}](${url})'';
composeSpecRev = "55b450aee50799a2f33cc99e1d714518babe305e";
serviceRef = fragment:
''See ${link "https://github.com/compose-spec/compose-spec/blob/${composeSpecRev}/05-services.md#${fragment}" "Compose Spec Services #${fragment}"}'';
networkRef = fragment:
''See ${link "https://github.com/compose-spec/compose-spec/blob/${composeSpecRev}/06-networks.md#${fragment}" "Compose Spec Networks #${fragment}"}'';
in
{
inherit
link
networkRef
serviceRef
;
}

View file

@ -1,8 +0,0 @@
[
./modules/composition/docker-compose.nix
./modules/composition/host-environment.nix
./modules/composition/images.nix
./modules/composition/networks.nix
./modules/composition/service-info.nix
./modules/composition/composition.nix
]

View file

@ -1,28 +0,0 @@
{ config, lib, ... }:
let
inherit (lib) types mkOption;
link = url: text:
''[${text}](${url})'';
in
{
options = {
_module.args = mkOption {
internal = true;
};
project.name = mkOption {
description = ''
Name of the project.
See ${link "https://docs.docker.com/compose/reference/envvars/#compose_project_name" "COMPOSE_PROJECT_NAME"}
This is not optional, because getting the project name from a directory name tends to produce different results for different repo checkout location names.
'';
type = types.str;
};
};
config = {
docker-compose.extended.project.name = config.project.name;
};
}

View file

@ -3,53 +3,26 @@
This is a composition-level module. This is a composition-level module.
It defines the low-level options that are read by arion, like It defines the low-level options that are read by arion, like
- out.dockerComposeYaml - build.dockerComposeYaml
It declares options like It declares options like
- services - docker-compose.services
*/ */
compositionArgs@{ lib, config, options, pkgs, ... }: { pkgs, lib, config, ... }:
let let
inherit (lib) types; evalService = name: modules: pkgs.callPackage ../../eval-service.nix {} { inherit name modules; inherit (config) host; };
service = {
imports = [ argsModule ] ++ import ../service/all-modules.nix;
};
argsModule =
{ name, # injected by types.submodule
...
}: {
_file = ./docker-compose.nix;
key = ./docker-compose.nix;
config._module.args.pkgs = lib.mkDefault compositionArgs.pkgs;
config.host = compositionArgs.config.host;
config.composition = compositionArgs.config;
config.service.name = name;
};
in in
{ {
imports = [
../lib/assert.nix
(lib.mkRenamedOptionModule ["docker-compose" "services"] ["services"])
];
options = { options = {
out.dockerComposeYaml = lib.mkOption { build.dockerComposeYaml = lib.mkOption {
type = lib.types.package; type = lib.types.package;
description = "A derivation that produces a docker-compose.yaml file for this composition."; description = "A derivation that produces a docker-compose.yaml file for this composition.";
readOnly = true;
}; };
out.dockerComposeYamlText = lib.mkOption { build.dockerComposeYamlText = lib.mkOption {
type = lib.types.str; type = lib.types.string;
description = "The text of out.dockerComposeYaml."; description = "The text of build.dockerComposeYaml.";
readOnly = true;
};
out.dockerComposeYamlAttrs = lib.mkOption {
type = lib.types.attrsOf lib.types.unspecified;
description = "The text of out.dockerComposeYaml.";
readOnly = true;
}; };
docker-compose.raw = lib.mkOption { docker-compose.raw = lib.mkOption {
type = lib.types.attrs; type = lib.types.attrs;
@ -59,26 +32,26 @@ in
type = lib.types.attrs; type = lib.types.attrs;
description = "Attribute set that will be turned into the x-arion section of the docker-compose.yaml file."; description = "Attribute set that will be turned into the x-arion section of the docker-compose.yaml file.";
}; };
services = lib.mkOption { docker-compose.services = lib.mkOption {
type = lib.types.attrsOf (lib.types.submodule service);
description = "An attribute set of service configurations. A service specifies how to run an image as a container.";
};
docker-compose.volumes = lib.mkOption {
type = lib.types.attrsOf lib.types.unspecified;
description = "A attribute set of volume configurations.";
default = {}; default = {};
type = with lib.types; attrsOf (coercedTo unspecified (a: [a]) (listOf unspecified));
description = "A attribute set of service configurations. A service specifies how to run an image. Each of these service configurations is specified using modules whose options are described in the Service Options section.";
};
docker-compose.evaluatedServices = lib.mkOption {
type = lib.types.attrsOf lib.types.attrs;
description = "Attribute set of evaluated service configurations.";
readOnly = true;
}; };
}; };
config = { config = {
out.dockerComposeYaml = pkgs.writeText "docker-compose.yaml" config.out.dockerComposeYamlText; build.dockerComposeYaml = pkgs.writeText "docker-compose.yaml" config.build.dockerComposeYamlText;
out.dockerComposeYamlText = builtins.toJSON (config.out.dockerComposeYamlAttrs); build.dockerComposeYamlText = builtins.toJSON (config.docker-compose.raw);
out.dockerComposeYamlAttrs = config.assertWarn config.docker-compose.raw;
docker-compose.evaluatedServices = lib.mapAttrs evalService config.docker-compose.services;
docker-compose.raw = { docker-compose.raw = {
version = "3.4"; version = "3.4";
services = lib.mapAttrs (k: c: c.out.service) config.services; services = lib.mapAttrs (k: c: c.config.build.service) config.docker-compose.evaluatedServices;
x-arion = config.docker-compose.extended; x-arion = config.docker-compose.extended;
volumes = config.docker-compose.volumes;
}; };
}; };
} }

View file

@ -15,7 +15,7 @@
}; };
host.nixStorePrefix = lib.mkOption { host.nixStorePrefix = lib.mkOption {
type = lib.types.str; type = lib.types.string;
default = ""; default = "";
example = "/mnt/foo"; example = "/mnt/foo";
description = '' description = ''
@ -23,9 +23,9 @@
stored at an alternate location without altering the format of stored at an alternate location without altering the format of
store paths. store paths.
For example: instead of mounting the host's `/nix/store` as the For example: instead of mounting the host's /nix/store as the
container's `/nix/store`, this will mount `/mnt/foo/nix/store` container's /nix/store, this will mount /mnt/foo/nix/store
as the container's `/nix/store`. as the container's /nix/store.
''; '';
}; };

View file

@ -4,39 +4,31 @@ let
serviceImages = serviceImages =
lib.mapAttrs addDetails ( lib.mapAttrs addDetails (
lib.filterAttrs filterFunction config.services lib.filterAttrs filterFunction config.docker-compose.evaluatedServices
); );
filterFunction = serviceName: service: filterFunction = serviceName: service:
builtins.addErrorContext "while evaluating whether the service ${serviceName} defines an image" builtins.addErrorContext "while evaluating whether the service ${serviceName} defines an image"
service.image.nixBuild; service.config.image.nixBuild;
addDetails = serviceName: service: addDetails = serviceName: service:
builtins.addErrorContext "while evaluating the image for service ${serviceName}" builtins.addErrorContext "while evaluating the image for service ${serviceName}"
(let (let
inherit (service) build; inherit (service.config) build;
in { in {
image = build.image.outPath;
imageName = build.imageName or service.image.name; imageName = build.imageName or service.image.name;
imageTag = imageTag =
if build.image.imageTag != "" if build.image.imageTag != ""
then build.image.imageTag then build.image.imageTag
else lib.head (lib.strings.splitString "-" (baseNameOf build.image.outPath)); else lib.head (lib.strings.splitString "-" (baseNameOf build.image.outPath));
} // (if build.image.isExe or false });
then {
imageExe = build.image.outPath;
}
else {
image = build.image.outPath;
}
)
);
in in
{ {
options = { options = {
build.imagesToLoad = lib.mkOption { build.imagesToLoad = lib.mkOption {
type = listOf unspecified; type = listOf unspecified;
internal = true; description = "List of dockerTools image derivations.";
description = "List of `dockerTools` image derivations.";
}; };
}; };
config = { config = {

View file

@ -1,53 +0,0 @@
{ config, lib, ... }:
let
inherit (lib)
mkOption
optionalAttrs
types
;
inherit (import ../../lib.nix { inherit lib; })
link
;
in
{
options = {
networks = mkOption {
type = types.lazyAttrsOf (types.submoduleWith {
modules = [
../networks/network.nix
];
});
description = ''
See ${link "https://docs.docker.com/compose/compose-file/06-networks/" "Docker Compose Networks"}
'';
};
enableDefaultNetwork = mkOption {
type = types.bool;
description = ''
Whether to define the default network:
```nix
networks.default = {
name = config.project.name;
};
```
'';
default = true;
};
};
config = {
networks = optionalAttrs config.enableDefaultNetwork {
default = {
name = config.project.name;
};
};
docker-compose.raw.networks =
lib.mapAttrs (k: v: v.out) config.networks;
};
}

View file

@ -5,13 +5,16 @@
*/ */
{ config, lib, ... }: { config, lib, ... }:
let let
inherit (lib) mapAttrs filterAttrs;
serviceInfo = serviceInfo =
filterAttrs (_k: v: v != {}) lib.mapAttrs getInfo (
(mapAttrs (_serviceName: service: service.out.extendedInfo) lib.filterAttrs filterFunction config.docker-compose.evaluatedServices
config.services );
);
filterFunction = _serviceName: service:
# shallow equality suffices for emptiness test
builtins.attrNames service.config.build.extendedInfo != [];
getInfo = _serviceName: service: service.config.build.extendedInfo;
in in
{ {

View file

@ -1,2 +0,0 @@
`assertions.nix`: Nixpkgs
`assert.nix`: extracted from Nixpkgs, adapted

View file

@ -1,33 +0,0 @@
{ config, lib, pkgs, ... }:
# based on nixpkgs/nixos/modules/system/activation/top-level.nix
let
inherit (lib)
concatStringsSep
filter
mkOption
showWarnings
types
;
# Handle assertions and warnings
failedAssertions = map (x: x.message) (filter (x: !x.assertion) config.assertions);
assertWarn = if failedAssertions != []
then throw "\nFailed assertions:\n${concatStringsSep "\n" (map (x: "- ${x}") failedAssertions)}"
else showWarnings config.warnings;
in
{
imports = [ ./assertions.nix ];
options.assertWarn = mkOption {
type = types.unspecified; # a function
# It's for the wrapping program to know about this. User need not care.
internal = true;
readOnly = true;
};
config = { inherit assertWarn; };
}

View file

@ -1,34 +0,0 @@
{ lib, ... }:
with lib;
{
options = {
assertions = mkOption {
type = types.listOf types.unspecified;
internal = true;
default = [];
example = [ { assertion = false; message = "you can't enable this for that reason"; } ];
description = ''
This option allows modules to express conditions that must
hold for the evaluation of the system configuration to
succeed, along with associated error messages for the user.
'';
};
warnings = mkOption {
internal = true;
default = [];
type = types.listOf types.str;
example = [ "The `foo' service is deprecated and will go away soon!" ];
description = ''
This option allows modules to show warnings to users during
the evaluation of the system configuration.
'';
};
};
# impl of assertions is in <nixpkgs/nixos/modules/system/activation/top-level.nix>
}

View file

@ -1,131 +0,0 @@
{ config, lib, options, ... }:
let
inherit (lib)
mkOption
optionalAttrs
types
;
inherit (import ../../lib.nix { inherit lib; })
networkRef
;
in
{
options = {
driver = mkOption {
description = ''
`"none"`, `"host"`, or a platform-specific value.
${networkRef "driver"}
'';
type = types.str;
};
driver_opts = mkOption {
description = ''
${networkRef "driver_opts"}
'';
type = types.lazyAttrsOf types.raw or types.unspecified;
};
attachable = mkOption {
description = ''
${networkRef "attachable"}
'';
type = types.bool;
example = true;
};
enable_ipv6 = mkOption {
description = ''
Whether we've entered the 21st century yet.
${networkRef "enable_ipv6"}
'';
type = types.bool;
};
ipam = mkOption {
# TODO model sub-options
description = ''
Manage IP addresses.
${networkRef "ipam"}
'';
type = types.raw or types.unspecified;
};
internal = mkOption {
description = ''
Achieves "external isolation".
${networkRef "internal"}
'';
defaultText = false;
type = types.bool;
};
labels = mkOption {
description = ''
Metadata.
${networkRef "labels"}
'';
# no list support, because less expressive wrt overriding
type = types.attrsOf types.str;
};
external = mkOption {
description = ''
When `true`, don't create or destroy the network, but assume that it
exists.
${networkRef "external"}
'';
type = types.bool;
};
name = mkOption {
description = ''
Set a custom name for the network.
It shares a namespace with other projects' networks. `name` is used as-is.
Note the `default` network's default `name` is set to `project.name` by Arion.
${networkRef "name"}
'';
type = types.str;
};
out = mkOption {
internal = true;
description = ''
This network's contribution to the docker compose yaml file
under the `networks.''${name}` key.
'';
type = lib.types.attrsOf lib.types.raw or lib.types.unspecified;
};
};
config = {
out =
lib.mapAttrs
(k: opt: opt.value)
(lib.filterAttrs
(k: opt: opt.isDefined)
{
inherit (options)
driver
driver_opts
attachable
enable_ipv6
ipam
internal
labels
external
name
;
}
);
};
}

View file

@ -1,13 +0,0 @@
[
./default-exec.nix
./docker-compose-service.nix
./extended-info.nix
./host-store.nix
./context.nix
./image.nix
./image-recommended.nix
./nixos.nix
./nixos-init.nix
../lib/assert.nix
./check-sys_admin.nix
]

View file

@ -1,30 +0,0 @@
{ config, lib, name, ... }:
let
inherit (lib)
concatStringsSep
optional
;
dynamicUserServices = lib.attrNames (
lib.filterAttrs
(k: v:
v.enable &&
v.serviceConfig.DynamicUser or false)
config.nixos.evaluatedConfig.systemd.services
);
in
{
config = {
warnings =
optional (config.nixos.useSystemd && !(config.service.capabilities.SYS_ADMIN or false) && dynamicUserServices != []) (
''In service ${name}, the following units require `SYS_ADMIN` capability
because of DynamicUser.
${concatStringsSep "\n" (map (srv: " - services.${name}.nixos.configuration.systemd.services.${srv}") dynamicUserServices)}
You can avoid DynamicUser or use
services.${name}.service.capabilities.SYS_ADMIN = true;
''
);
};
}

Some files were not shown because too many files have changed in this diff Show more