<amedia>

varl's nixtapes vol.1

Viktor Varland (varl)

TL;DR

At the end of this post we will have reached a point where we can load a shell with specific tools, and when we leave that shell, the global environment is untouched.

volume 1: Setting up Nix

Background

Over the last year or so I have experimented with using nix to manage my development environments across multiple machines.

I like to keep e.g. my personal projects and work projects separated as much as possible, so on my work projects I don't want my personal development stack (tools, versions, etc.) to be loaded and when I work on my personal projects the same applies.

I also want the development environments to be declarative, but the actual use case is that I want them to be portable to other machines, so I can spin up a new development environment fairly quickly.

There are other ways to achieve full isolation, e.g. a complete virtual machine, but I don't like the overhead that brings.

I also like the way I have my global environment setup, and prefer to use that as a base, and then bring whatever tools I need for the context I am working in into it.

I'm not sure when it happened, but nix the package manager (not "nix the language", and not "nix the operating system"), is quite mature[1] across both Linux and MacOS, so I decided to brush off my "nix the language" concerns and dive into it. Again.

Prerequisites

  • A supported OS:
    • Linux
    • MacOS
    • Windows with WSL2 with systemd enabled[2]
  • Comfortable using a shell
  • A high tolerance for obtuse languages[3]

A note on "Nixes"

Nix is somewhat of an overloaded term, as it can mean:

  • Nix the language
  • Nix the CLI
  • Nix the OS[4]
  • Nix the package manager
  • Nix the utilities
  • Nix the store

I will use a subset of those in this guide, and do my best to differeniate them.

The in-scope Nix terms will be:

  • Nix the language (try as I might, there is no avoiding it)
  • Nix the package manager
  • Nix the utilities

This will give us all we need to accomplish our goal.

A note on Nix installers

There are many ways to install Nix (the package manager) without NixOS, and it works on any Linux distribution alongside the system package manager.

It also works on MacOS, but there are a few rough patches of terrain. We can mostly avoid them, or implement workarounds for the nasty ones.

Official installer

https://nixos.org/download

The official installer is available for Linux, MacOS, and Windows and is the most unopinionated of the lot. It provides a predictable environment, with opt-in to experimental functionality (that we will not use in this article anyway), and is managed by the Nix developers.

Distro installer

Arch Linux, Gentoo, Debian, Alpine, etc. all provide packages for their distro that can be used with various caveats.

I use the Arch Linux nix package on my workstation, for example, and haven't had issues with it. It may be slightly out-of-date versus the official one, but the package maintainer has been fast with updates so I have never had issues.

I would be more sceptical of the Debian stable package version, and perhaps opt for the official installer on that distro. Your mileage may vary, but you are on Linux so you should be used to research and inform your own choices.

Determinate Nix installer

https://github.com/DeterminateSystems/nix-installer

This installer is developed by Determinate Systems[5] and is an opinionated installer, in that it makes different choices on how Nix should be installed and setup than the official installer does. It leans into the advanced functionality (often experimental) that the official installer leaves off by default, and goes as far as to disable or dissuade users from using the stable nix utilities.

It's a great installer, but if used, some configuration options must be manually reverted to the official defaults. I tried this installer, but wound up uninstalling and using the official one to avoid diving into all the changes and assumptions they make.

Installing using the official installer

If you decided to install Nix using the distro package manager or the determinate systems' installer, feel free to jump ahead.

Linux

Given you have Linux running systemd, with SELinux disabled, and can authenticate with sudo, the multi-user installation method is recommended:

sh <(curl -L https://nixos.org/nix/install) --daemon

If you cannot use the multi-user installation, you must use the single-user installation method:

sh <(curl -L https://nixos.org/nix/install) --no-daemon

Follow along the installation, answering the prompts, and you will end up with a Nix installation.

MacOS

For MacOS a multi-user installation is recommended:

sh <(curl -L https://nixos.org/nix/install)

Same deal here, follow along the instructions and the result should be a working Nix installation.

Using the environment

Now that we have Nix setup, we can try out a few tricks. If you haven't, you will need to open a new shell.

Starting a shell with specific tools

nix-shell reference manual: https://nixos.org/manual/nix/stable/command-ref/nix-shell

nix-shell starts an interactive shell based on a Nix expression, but we will stick to our guns and define packages instead of an expression.

We can start a pure environment that inherits nothing from the global environment:

nix-shell --pure

In which we don't even have ping defined:

[nix-shell:~/dev]$ ping
bash: ping: command not found

Type exit to go back to the global shell, and try this instead:

nix-shell --pure --packages cowsay

[nix-shell:~/dev]$ cowsay "hi nixer !"
 ____________
< hi nixer ! >
 ------------
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

Hit Ctrl-D or type exit again. Multiple packages can be listed on the commandline separated by spaces:

nix-shell --pure --packages less vim

Notice that when we request packages we don't have, we get these messages:

these 2 paths will be fetched (8.06 MiB download, 38.84 MiB unpacked):
  /nix/store/h5m8kaai6x64j1q6r7ffvq20f06r77m3-less-633
  /nix/store/6j38m8vm8gp9a8qpw3b7dj9g50x1w95n-vim-9.0.1562
copying path '/nix/store/h5m8kaai6x64j1q6r7ffvq20f06r77m3-less-633' from 'https://cache.nixos.org'...
copying path '/nix/store/6j38m8vm8gp9a8qpw3b7dj9g50x1w95n-vim-9.0.1562' from 'https://cache.nixos.org'...

The short version is that each package and version is hashed and stored in the Nix store (/nix). The consequence of this is that multiple versions of the same package can exist at the same time (at the cost of disk space) without causing conflicts across the environment. There is also a long version[6]. Probably a longer version somewhere as well.

Installing packages into the environment

nix-shell reference manual: https://nixos.org/manual/nix/stable/command-ref/nix-env#name

Being able to run a shell with specific packages is handy, and nix-shell will be a underlying driver further into the nixtapes. Instead let's take a look at the nix-env command.

Using nix-env we can manipulate a Nix user environment, which means un/installing packages, searching for packages, and, rolling back the environment.

Let's find some packages to install.

nix-env --query --available vim

vim-9.0.1562

Great, love that version. Let's install it:

nix-env --install vim-9.0.1562

installing 'vim-9.0.1562'
building '/nix/store/08xr544knfcahpn9xykql2xzsg374pxl-user-environment.drv'...

which vim

/home/varl/.nix-profile/bin/vim

readlink /home/varl/.nix-profile/bin/vim

/nix/store/6j38m8vm8gp9a8qpw3b7dj9g50x1w95n-vim-9.0.1562/bin/vim

Given that /home/varl/.nix-profile/bin is added to my PATH before any paths that has my system version of vim, it will be picked up correctly and when I run vim I will get the nix-installed version. No shell-trickery and vim will be available across any other interactive shells I open and use.

It is also possible to preserve installed alternative derivations of a package, but I haven't had to use it much.

One thing we will use a lot though is the --remove-all flag. This removes all the installed packages in an environment, and only installs the target package in the environment, so if we run:

nix-env --install vim nodejs python3

installing 'vim-9.0.1562'
installing 'nodejs-18.16.1'
installing 'python3-3.12.0b3'
these 3 paths will be fetched (65.51 MiB download, 219.36 MiB unpacked):
  /nix/store/pxv7fa7ysw18kqrlvs1g0f9q66l7paz3-nodejs-18.16.1-libv8
  /nix/store/3v1hjf626mh7mdii28m0srdbl8ch3dka-python3-3.12.0b3
  /nix/store/b9a3j1rvcgj4wxpb30yzdi7ba62g3ha8-python3-3.12.0b3-debug
copying path '/nix/store/pxv7fa7ysw18kqrlvs1g0f9q66l7paz3-nodejs-18.16.1-libv8' from 'https://cache.nixos.org'...
copying path '/nix/store/b9a3j1rvcgj4wxpb30yzdi7ba62g3ha8-python3-3.12.0b3-debug' from 'https://cache.nixos.org'...
copying path '/nix/store/3v1hjf626mh7mdii28m0srdbl8ch3dka-python3-3.12.0b3' from 'https://cache.nixos.org'...
building '/nix/store/jid36dzng4pjqph3f0bdyzmsvaq5fl0h-user-environment.drv'...

And then do:

nix-env --install --remove-all vim

Only vim will be installed, and python3 and nodejs are purged from the environment. This is an important feature, as it allows us to install any tools and versions we want to play around with nix-env --install, and then go back to scratch without worrying about leaving a mess in our environments.

Uninstalling packages from the environment

Uninstalling is without drama:

nix-env --uninstall vim

Environment generations

Every time we use nix-env to modify our environment, Nix creates a new generation. We can jump between generations, rollback, and delete them.

nix-env --rollback

This rolls back the current environment one generation, and is just a convenience wrapper around --list-generations and --switch-generation.

nix-env --list-generations

...
 113   2023-06-21 16:39:06
 114   2023-08-08 16:25:50
 115   2023-08-08 16:36:01
 116   2023-08-08 16:37:42   (current)

Using the id in the left column, we can jump to different generations of our environment.

Deleting generations (they add up) can be done on a one-by-one basis, or using smart values like +5 (save last 5) and 30d (save last 30 days).

nix-env --delete-generations 113
nix-env --delete-generations +5
nix-env --delete-generations 30d

Where are we now ?

We have installed Nix (the package manager) and the Nix utilities (nix-* commands) that we need.

We have explored how to create ad-hoc shell environments that only have specific packages available, which is useful for trying out new things and switching between versions of e.g. language runtimes.

We have learned how to manipulate the user's environment by installing packages that persist in the environment (as opposed to the ephemeral ones in nix-shell).

We've seen that Nix creates generations of the user's environments, and that we can switch between generations or simply rollback the environment to a previous state.

These are the building blocks for managing our user's environment, as well as the various development environments that we strive for.

/v.

  1. Well, nix-env and other nix-* commands are mature, but the next-gen all-in-one nix CLI is definitely not mature. It's not even final: https://nixos.org/manual/nix/stable/command-ref/experimental-commands ↩︎

  2. Configuration guide: https://devblogs.microsoft.com/commandline/systemd-support-is-now-available-in-wsl/ ↩︎

  3. We will be using a bare minimum of "nix the language" to accomplish our goals, so there are better ways to do most things I do, but I don't understand any of them so I'm sticking to what I can only describe as "simple but dumb". ↩︎

  4. Most often called NixOS but searching for "nix" often winds up in NixOS-land which is not always helpful. ↩︎

  5. A consultancy that provides services related to Nix training for teams and companies. ↩︎

  6. Understanding the Nix store by looking at how Nix works: https://nixos.org/guides/how-nix-works.html ↩︎