Skip to content
This repository has been archived by the owner on Aug 17, 2023. It is now read-only.

Add documentation for Flakes #23

Draft
wants to merge 4 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 46 additions & 0 deletions docs/code/Flakes/Explanation/Why.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
---
title: Why Flakes?
date: 2021-02-10
---

Nix Flakes are a new feature of the "nix package manager". They unify the typical ways of declaring inputs and ouputs of a project under a common syntax.

But why use them?

## Why?

- Unify the management of inputs (make redundant tools like `niv`).
- Remove sources of impurity that existed in the previous model.
- Standardize the output format of repositories.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One other benefit of this new format is that it facilitates composability, so it becomes easier to connect one flake's outputs to other flake's inputs.

- A Nix expression typically creates a "package" for a repository, for example, the cardano-wallet expression creates a binary package which is available at "cardano-wallet". The location of the attribute in the output attribute set is completely arbitrary. Another repository might put their "package" at "packages.cardano-doohickey".
- Nix flakes attempt to standardize the typical output locations of a Nix expression, so that packages are always available at the same location.

## Concepts

From an abstract point of view, a derivation in Nix is a function from inputs to ouputs. Inputs are typically libraries and executables, and the output is typically another library or executable.

Flakes extend the input/output model of derivations to the entire repository. A single file sits at the root of the repository called `flake.nix`, that might look something like this:

```
{
description = "Example flake";

inputs = {
cardano-db-sync.url = "github:input-output-hk/cardano-db-sync?ref=refs/tags/12.0.0";
haskellNix.url = "github:input-output-hk/haskell.nix";
# ... other inputs
};

outputs = { cardano-db-sync, haskellNix }: {
packages.x86_64-linux.linode-cli = pkgs.python3Packages.callPackage ./pkgs/linode-cli.nix {};
# ... other outputs
};
}
```

It's conceptually quite simple. Given these inputs, this repository produces a set of outputs.

IOHK previously made use of `niv`, `niv` was just a different method of pinning inputs that wrote to the file "sources.json". The flakes model isn't necessarily "better" than `niv`, but flakes have other benefits.

By running `nix flake lock`, the inputs of the repository are pinned down to specific revisions and SHAs and written to `flake.lock`.

16 changes: 16 additions & 0 deletions docs/code/Flakes/Reference/CommonErrors.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
---
title: Common Errors
date: 2021-02-09
---

```
error: getting status of '/nix/store/1dg75ahm07ah26phnd1jy1jq06s680ps-source/some-file.nix': No such file or directory
```

Make sure the file is staged to version control. Flakes can't see files that are not tracked by version control.
sevanspowell marked this conversation as resolved.
Show resolved Hide resolved

```
warning: Git tree '/home/rodney/iohk/cardano-wallet' is dirty
```

It can be hidden (if desired) by using the allow-dirty = true option in nix.conf.
50 changes: 50 additions & 0 deletions docs/code/Flakes/Reference/Invocations.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
---
title: Invocations and Pathing
date: 2021-02-10
---

## Invocations

`nix-build` is now `nix build`

`nix-shell` is now `nix develop`

`nix-repl default.nix` is now:

```
$ nix repl
> :lf .
# Get some information:
> legacyPackages.x86_64-linux.pkgs.stdenv.hostPlatform.libc
# Build cardano-wallet:
> :b packages.x86_64-linux.cardano-wallet
```

## Pathing

`nix-build -A cardano-wallet` is now `nix build .#packages.x86_64-linux.cardano-wallet`

i.e.

```
nix build $FLAKE_PATH
$FLAKE_PATH = $PATH_TO_FLAKE_FILE#$ATTRIBUTE
$FLAKE_PATH = .#packages.x86_64-linux.cardano-cli
```

The `$PATH_TO_FLAKE_FILE` can be quite complicated:

```
· .: The flake in the current directory.
· /home/alice/src/patchelf: A flake in some other directory.
· nixpkgs: The nixpkgs entry in the flake registry.
· nixpkgs/a3a3dda3bacf61e8a39258a0ed9c924eeca8e293: The nixpkgs entry in the flake registry, with its Git revision overridden to a specific value.
· github:NixOS/nixpkgs: The master branch of the NixOS/nixpkgs repository on GitHub.
· github:NixOS/nixpkgs/nixos-20.09: The nixos-20.09 branch of the nixpkgs repository.
· github:NixOS/nixpkgs/a3a3dda3bacf61e8a39258a0ed9c924eeca8e293: A specific revision of the nixpkgs repository.
· github:edolstra/nix-warez?dir=blender: A flake in a subdirectory of a GitHub repository.
· git+https://github.com/NixOS/patchelf: A Git repository.
· git+https://github.com/NixOS/patchelf?ref=master: A specific branch of a Git repository.
· git+https://github.com/NixOS/patchelf?ref=master&rev=f34751b88bd07d7f44f5cd3200fb4122bf916c7e: A specific branch and revision of a Git repository.
· https://github.com/NixOS/patchelf/archive/master.tar.gz: A tarball flake.
```
112 changes: 112 additions & 0 deletions docs/code/Flakes/Reference/Schema.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
---
title: Nix Flake Schema
date: 2021-02-09
---

The `flake.nix` has the following schema:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a feeling this will go out of date, and the original sources could just be linked.
On the other hand, the flake schema is not well documented.
The resources I use are:

  1. Schema - https://nixos.wiki/wiki/Flakes#Flake_schema
  2. Input references - https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-flake.html#flake-references
  3. Installables - https://nixos.org/manual/nix/stable/command-ref/new-cli/nix.html#installables


```
{
description = "A string";
inputs = *input_schema;
outputs = *output_schema:
}
```

## Input schema

An example input attrset:

```
{
# A GitHub repository.
import-cargo = {
type = "github";
owner = "edolstra";
repo = "import-cargo";
};

import-cargo-alt.url = github:edolstra/import-cargo;

nixpkgs.url = "nixpkgs";

cardano-db-sync.url = "github:input-output-hk/cardano-db-sync?ref=refs/tags/12.0.0";

nixpkgs-alt.url = "github:NixOS/nixpkgs/nixos-21.11";
}
```


## Output schema

There is a standard schema for the outputs of a flake:

```
{ self, cardano-db-sync, nixpkgs, import-cargo, ... } @ inputs: {
# Packages built by the repository.
# Executed by `nix build .#<name>`
packages."<system>"."<name>" = derivation;

# Executed by `nix build .`
defaultPackage."<system>" = derivation;

# A default development shell for the project.
# Used by `nix develop`.
devShell."<system>" = derivation;

# A development shell for the project.
# Used by `nix develop .#<name>`
devShells."<system>"."<name>" = derivation;

# Hydra build jobs
hydraJobs."<attr>"."<system>" = derivation;

# Automated checks.
# Executed by `nix flake check`
checks."<system>"."<name>" = derivation;

# Used for nixpkgs packages, also accessible via `nix build .#<name>`
# Typically the repository will provide the version of nixpkgs used in the repository here:
# E.g.
# legacyPackages = import nixpkgs {
# inherit system overlays;
# };
legacyPackages."<system>"."<name>" = derivation;

# Used by `nix flake init -t <flake>#<name>`
templates."<name>" = { path = "<store-path>"; description = ""; };

# Default overlay, consumed by other flakes. Overlays are typically overlaid onto nixpkgs to add extra packages to the package set. Potentially for another project or for a system configuration.
overlay = final: prev: { };

# Same idea as overlay but a list or attrset of them.
overlays = {};

# Default module, consumed by other flakes. NixOS modules are used by NixOS configurations to "modularize" configuration.
# For example, by importing the "cardano-node.nixosModule" I can run a simple node on my computer by enabling the option: "services.cardano-node.enable = true".
nixosModule = { config }: { options = {}; config = {}; };

# Same idea as nixosModule but a list or attrset of them.
nixosModules = {};

# Used with `nixos-rebuild --flake .#<hostname>`
# nixosConfigurations."<hostname>".config.system.build.toplevel must be a derivation
nixosConfigurations."<hostname>" = {};

# Used by `nix flake init -t <flake>`
defaultTemplate = {
path = "<store-path>";
description = "template description goes here?";
};

# Executed by `nix run .#<name>`
apps."<system>"."<name>" = {
type = "app";
program = "<store-path>";
};

# Default application
# Executed by `nix run . -- <args?>`
defaultApp."<system>" = { type = "app"; program = "..."; };
}
```
37 changes: 37 additions & 0 deletions docs/code/Flakes/Tutorials/Hacking.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
---
title: Create a Development Environment
date: 2021-02-09
order: 3
---

## Hacking

`nix-shell` has been replaced by `nix develop`, and by default it builds the attribute `devShell.x86_64-linux`, so a simple python development environment would look like this:
sevanspowell marked this conversation as resolved.
Show resolved Hide resolved

`flake.nix`
```
{
description = "A very basic flake";

inputs = {
nixpkgs.url = "github:NixOS/nixpkgs/nixos-21.11";
};

outputs = { self, nixpkgs }:
let
pkgs = nixpkgs.legacyPackages.x86_64-linux
in {
devShell.x86_64-linux = pkgs.mkShell {
nativeBuildInputs = [
pkgs.python3
];
};
};
}
```
Comment on lines +13 to +31
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we cut down the examples so that they are still self-contained, but only deal with the topic of this page?
For example, remove pinning of nixpkgs, unless the page is about version pinning.

But here is a more interesting example of what could be put in the shell:

Suggested change
{
description = "A very basic flake";
inputs = {
nixpkgs.url = "github:NixOS/nixpkgs/nixos-21.11";
};
outputs = { self, nixpkgs }:
let
pkgs = nixpkgs.legacyPackages.x86_64-linux
in {
devShell.x86_64-linux = pkgs.mkShell {
nativeBuildInputs = [
pkgs.python3
];
};
};
}
```
{
description = "Flake with a development environment";
outputs = { self, nixpkgs }: {
devShell.x86_64-linux =
with nixpkgs.legacyPackages.x86_64-linux;
mkShell {
nativeBuildInputs = [
(python3.withPackages (ps: [ ps.numpy ps.toolz ps.requests ]))
];
};
};
}
```
This is an example of adding Python and some Python Wheels to the shell. More information (for those interested) can be found in the
[Nixpkgs Manual Python section](https://nixos.org/manual/nixpkgs/stable/#python).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This builds on tutorial 2, and so I left the version pinning in to minimize required code changes from the last flake.nix, but feel free to remove.


We could then run `nix develop` and we would enter a shell with access to python.

## Reference

[nix develop man page](https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-develop.html)
41 changes: 41 additions & 0 deletions docs/code/Flakes/Tutorials/Installation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
---
title: Installation
date: 2021-02-09
order: 1
---

Presuming nix is already installed, Nix flakes are available from Nix version 2.4 and up.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


## Non-NixOS

```
$ nix-env -iA nixpkgs.nixUnstable # or nixpkgs.nix_2_4
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Simplest way is probably to ask users to run:

nix upgrade-nix

If that doesn't work, then reinstall it using the instructions here: https://nixos.org/manual/nix/stable/installation/installing-binary.html#multi-user-installation

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't work on NixOS

  nix upgrade-nix 
error: Nix on NixOS must be upgraded via 'nixos-rebuild'

# or nixpkgs.nix_2_5
# or nixpkgs.nix_2_6, etc.
```

Then edit either `~/.config/nix/nix.conf` or `/etc/nix/nix.conf` and add:

```
experimental-features = nix-command flakes
```

## NixOS
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NixOS-specific instructions probably aren't necessary. Especially because nix.extraOptions has become nix.settings.experimental-features in the latest nixos-unstable versions.


Add the following to your NixOS configuration:

`/etc/nixos/configuration.nix`
```
nix = {
package = pkgs.nixUnstable; # or pkgs.nix_2_4
# or pkgs.nix_2_5
# or pkgs.nix_2_6, etc.
extraOptions = ''
experimental-features = nix-command flakes
'';
};
```

Then

`nixos-rebuild switch`
Loading