A development box that persists every configuration

Overview

The devbox provides a virtualbox linux operating system that would typically run on top of a Windows host. The box is based on NixOS and make sure any setting can be persisted.

The main configuration file is box.dhall. It sits in your shared folder.

devbox

First time setup

  • Have a quick look at the Troubleshooting section before to get started.

  • You can install Git for windows in order to get a terminal console on Windows.

Preparation

Software requirements

  • Virtualbox 6.1.x (tested with 6.1.6)

Please do update if you have older version(s).

Within the host os (usually Windows), create a folder C:\Users\virtualbox\devbox-20.03.

Make sure your windows user has read/write permission on the folder. On Windows, right-click on the folder and follow Properties  Sharing.

This will be your shared folder between the host (Windows) and the guest (/vagrant in the linux box). We will use the term SHARED_DIR to refer to this folder throughout the rest of the guide.

The SHARED_DIR contains personal configuration files.

typical SHARED_DIR
C:\Users\virtualbox\devbox-20.03
   ├── [ssh-keys] (1)
   │      ├── xxxx_rsa
   │      ├── xxxx_rsa.pub
   │      ├── xxxx_github_rsa
   │      └── xxxx_github_rsa.pub
   ├── box.dhall (2)
   ├── local-home.nix  (2)
   └── [local-configuration.nix] (3)
1 ssh-keys folder
2 user configuration
3 optional system configuration

You don’t need to create any of the user configuration files. If the file is not yet present in the SHARED_DIR, it will be created by updateConfig. You should then edit the file and fill the requested information[1].

Type updateConfig again to activate those changes.

  • If you don’t use the default host location for the SHARED_DIR, you will need to manually change the setting using the Virtualbox UI. When doing so, please keep the name vagrant together with an empty mount point.

  • When editing files from the SHARED_DIR, maintain the unix file encoding.

ssh keys

Without your personal ssh key pairs you won’t be able to pull or push to bitbucket repositories.

  1. in the SHARED_DIR, create a directory ssh-keys.

  2. in this ssh-keys subfolder, copy your Bitbucket key pair, rename them 'cirb_rsa' and 'cirb_rsa.pub' respectively if the filename differs.

  3. in the ssh-keys subfolder, copy your Github key pair, rename them 'cirb_github_rsa' and 'cirb_github_rsa.pub'. If you use the same key pair, copy the previous pair and rename accordingly.

Box import

Get the latest devbox-20.03.x file from Artifactory

To import the ova image file, just click on the file. The virtualbox UI will pop up. You can accept all defaults.

Usage

User provisioning

The most common operation is to modify your user configuration through box.dhall in your SHARED_DIR.

You might also want to modify your user configuration through the included home-manager. Such more advanced and optional configuration is done by editing the local-home.nix file which also sits in your SHARED_DIR.

To activate these changes, you execute the command updateConfig in a console[2].

If you wish to update the files that provision the devbox, you do so with the updateUser command.

System provisioning

The advanced user might want to update the system configuration. This is useful to enable middleware such as a postgresql database, an elk cluster or a minishift.

If you want to go into this level of configuration, first import in your SHARED_DIR the example configuration file:

curl -o local-configuration.nix https://raw.githubusercontent.com/CIRB/devbox/master/system/local-configuration.nix

To publish your system configuration change, you execute the command updateSystem in a console.

Operating system (nixos)

The devbox is based on NixOS version 20.03. NixOS is a Linux distribution based on the nix package manager.

Compare to more traditional distributions, NixOS favors declarative settings and brings to the table the following properties:

  • reproducability: everything is reproducible (packages are identical binary wise).

  • atomicity: configuration changes can be rollback.

  • isolation: on a project, user or system level, processes are built in an isolated environment.

By using nixos we are striving for a system that we can destroy and re-create at will without fuss.

The entry point for the system setup is the /etc/nixos/configuration.nix file.

Useful command

# rebuild nixos after changing the system configuration file
→ sudo nixos-rebuild switch

# clean-up the store (whenever the disk usage is too high)
→ sudo nix-collect-garbage -d

Windows manager (xmonad)

The devbox comes with a tiling windows manager called xmonad. Such a minimal approach has been chosen for 3 reasons:

  • Efficiency: the box needs to consume as minimum CPU/Mem resources as possible

  • Simplicity: the window manager is basic but yet quite flexible

  • Practicality: the desktop is distraction free.

If you want another UI, based on Gnome, KDE, i3wm or Pantheon, that’s easy enough. Please ask for assistance on our #CICD Team channel[3]

Minimal cheat sheet

Command Description

Ctrl+Space[4]

Open app launcher

Super+t

New terminal

Super+1..9

Go to nth desktop

Super+Right

Go to next desktop

Super+Left

Go to previous desktop

Super+Shift+1..9

Switch to nth desktop

Super+Space

Change between horizontal, vertical and stack layout

Super+Tab

In stack layout, switch the stacked window

Super+q

Close current window

Super+m , Super+h

Resize windows

Super+w

Put window back into tiling

F 1

Open this README in a browser

Layout

The active layout is displayed on the status bar:

layout indicator
Symbol Name

horizontal

horizontal tiled

vertical

full (stack)

vertical tiled

You can also change layout by clicking on the symbol in the status bar.

Terminal emulator (termite)

The devbox uses the termite terminal.

Command Description

Control+Shift+c/v

Copy/paste to/from clipboard

Control+Shift++/-/=

Increase/Decrease/Reset font size

Control+Tab

Start scrollback completion

Control+Shift+Space

Start selection mode

The default user shell is zsh but you can easily switch to bash by changing this line. Both allows for predictable history and completion.

You can use autojump to quickly navigate from one directory to another using the j shortcut:

# Go to your directory (you can omit the `cd`)
→ cd projects/cicd/puppet/bos
# Teach autojump that you like bos
→ autojump -i 20
# Open a new terminal. You can now quickly go to ~/projects/cicd/puppet/bos
→ j bos

If you wish to add some plugins to oh-my-zsh the setting is http://stash.cirb.lan/projects/CICD/repos/devbox/browse/.config/nixpkgs/modules/profiles/zsh.nix#35 [here][5].

Another useful shortcut is sshi to quickly log to a remote machine. It is the equivalent of ssh -i ~/.ssh/xxxx_rsa $LOGINID@$1.

Application launcher (albert)

Albert is an application launcher similar to Alfred in OS X. You can launch albert with Control+Space[6]. Use it to:

  • open any application, directory or file by typing its name (type 'File' to open Nautilus)

  • shutdown/reboot the vm

  • search the internet with gg

Projects (mr)

The myrepos project help you to automate the cloning of multiple git repositories. The usage of mr is optional but quite useful when you work with dozens of repositories.

Mr files are pointers to repositories you want to clone. These are defined in the same CIRB dotfiles repository. You may propose any new repository definition via a pull request.

To activate an available repository in your devbox, add the name of the file to the mr.repos list in SHARED_DIR/box.dhall.

As soon as you type updateConfig on the command line, the enabled repos will be automatically cloned and configured into your box. For instance if puppet-bos is in the list, you will find a ~/projects/bric/box/puppet-bos folder. In that case, the next step would be to go in that folder and enter make on the command line.

Some mr description files purposefully prevent the repository update by using the skip marker. For those repositories, updateConfig won’t do anything.

In that case and more particularly when you first boot a new devbox, you might want to force the update. In the home folder, use the following command:

force mr repo updates
~ → mr -f up (1)
1 -f is forcing the update

You can ask a git status of all your repositories with

~ → mr st

Local environment (direnv/lorri)

direnv removes the need to nix-shell into a project folder. It prevents the project derivation to be garbage collected as the dependency graph is rebuild whenever the project dependencies changes.

If you feel the initiation time when you switch from folder to folder is too long, you can enable Lorri in your box.dhall configuration.

Tips & tricks

The customization of the devbox operates at different levels and ensure configuration - personal or shared - survives the destruction of the box.

Install local packages

To install a package, just do:

→ nix-env -i geany (1)
1 install the geany package for the vagrant user. The suffix env expresses the fact that only the user environment is affected.

This imperative approach is useful for testing but the change is not persistent. When you settle down on a list of packages, please add them to local-home.nix to gain persistence.

You can quickly search for packages online at nixos.org

Open a shell

You can open a shell with the nix-shell commmand. For instance to open a shell that have JAVA_HOME set:

→ nix-shell -p jdk11

Storing the box configuration file

You might want to go one step further and store your box.dhall file on Github or the like. This is pretty easy thanks to the powerful dhall configuration language. Replace the content of SHARED_DIR/box.dhall with one simple line containing an url that points to the config file. Here is an example:

SHARED_DIR/box.dhall
https://raw.githubusercontent.com/PierreR/devbox-config/master/box.dhall

Life cycle

The devbox will be maintained continuously. Two version are expected to be released every year in April and October following the NixOS release cycle.

The versioning scheme follows the one of NixOS. For instance 19.09.buildnumber.

When a new major version is released, breaking changes are expected. It is required to destroy and re-create a box from scratch. An operation that take no longer than a few minutes. Please look at the Changelog to gather information and read the updated README if necessary. You would commonly need to make some adjustements to your personal configuration file such as dhall.box or local-configuration.nix before importing them in the new SHARED_DIR.

Keep the old box and the new one for a while and make sure you aren’t losing any customization.

Question, support, discussion are handled in the team #cicd channel.

Using another unix distribution

The devbox provided an easy way to get started. Nevertheless if you prefer to work with another linux derivation (such as Ubuntu for instance) it is possible to do so and still benefit from the binary cache the CICD team provided.

Follow these simple steps:

  1. install nix on your linux machine following these instructions. It is actually a single command:

    curl -L https://nixos.org/nix/install | sh
  2. export the NIX_PATH environment variable:

    export NIX_PATH=$NIX_PATH:nixpkgs-overlays=http://stash.cirb.lan/CICD/nixpkgs-overlays/archive/master.tar.gz
  3. Add the current nixpkgs pointer

    cat > ~/.nix-channels <<EOF
    https://releases.nixos.org/nixos/20.03/nixos-20.03.1950.48723f48ab9/nixexprs.tar.xz nixpkgs (1)
    EOF
    1 You will need to manually follow the update of the pointer.
  4. Update your nix channel

    nix-channel --update
  5. configure the binary cache

    cat > ~/.config/nix/nix.conf <<EOF
    substituters = https://repository.irisnet.be/artifactory/nix/
    trusted-public-keys = nix.cirb.org:CeHSF470ofjLTEwKr4nyGXlH8Haxelxu1RItfo7yOZU= cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
    sandbox = false
    EOF
  6. configure nix

    cat > ~/.config/nixpkgs/config.nix <<EOF
    { allowUnfree = true; allowBroken = true; }
    EOF
  7. install direnv

Troubleshooting

Ssh-key with passphrase

Using cirb ssh-keys with a passphrase will currently cause difficulties with mr, a tool we use to handle git repositories in a declarative way. It will prevent some of the automation to go through (nothing critical though). If possible, please regenerate a pair of keys with no passphrase and register them in stash.cirb.lan[7].

For Windows 10 users

Everything usually just works on Windows 10. That said, keep in mind that various issues have been reported concerning incompatibilities between hyper-v and virtualbox. It is likely that such compatibilities arise again after a windows or virtualbox update.

We will try to offer a hyper-v compatible box using packer (no virtualbox) in the future. Packer has just announced such a support. Stay tune.

In the meanwhile, if you face such a issue you will have to disable hyper-v to get a working devbox. Please proceed as follow:

  1. disable hyper-v

  2. use virtualbox as hypervisor

  3. Use the devbox for your docker need (docker is included out of the box) or install some docker tools based on virtualbox

The only use of hyper-v we have detected so far is by the latest version of Docker for Windows.
For Mac users

There are known critical issues with Virtualbox and VPN clients on macosx.

For VMware users

There is currently an issue with packer & VMware Workstation. Since 6.1, Virtualbox supports nested hardware-virtualization on Intel CPUs (starting with 5th generation Core i, codename Broadwell). Support for VMware Workstation is not provided.

FAQ

How is the box generated ?

The box (of about 4G) is generated in two stages. The first step generates a minimal generic ova file with packer (currently packer-1.5.5). The second step applies some customizations to produce a more heavy tailored box (desktop, …​).

Stage 1

  1. On a linux host[8], use the provided nixbox.sh script to clone the nixbox repository within the shared directory.

  2. On the host OS, in the nixbox directory, execute the following commands:

    packer.exe build nixos-x86_64.json --only virtualbox-iso

Stage 2

  1. Create the SHARED_DIR

  2. Import the ova image file using the Virtualbox UI.

  3. Start the imported vm, login as vagrant

    1. cd into /etc/devbox-20.03.x

    2. make system

    3. reboot

    4. cd into /etc/devbox-20.03.x

    5. make user

  4. Export the vm

  5. Push the ova to a mirror site.

Eclipse: how to add plugins ?

If you enable eclipse in $SHARED_DIR/box.dhall, you might want to add the egit and m2e plugins.

enable egit and m2e
→ cd ~/bootstrap
→ make eclipse-extraplugins

If you miss other plugins, let us know.


1. a comment section is present at the top.
2. If you want to skip the mr step, you can use home-manager switch
3. If such request is popular, we will add an help section about it.
4. you can change this binding in box.dhall
5. let us know if such customization is of any interest and we will add a key in dhall.box for it.
6. you can change the hotkey in the box.dhall configuration file.
7. We will improve the situation in a future release if it is tagged as a major concern.
8. Cloning in Windows will cause many encoding issues!