A development box that persists every configuration

Overview

The devbox provides a virtualbox linux operating system that would typically run on top of a Windows host. The box is based on NixOS and make sure any setting can be persisted.

The main configuration file is box.dhall. It sits in your shared folder.

devbox

Software requirements

  • Virtualbox 6.0.x (tested with 6.0.12)

Please do update if you have older version(s).

First time setup

  • Have a quick look at the Troubleshooting section before to get started.

  • You can install Git for windows in order to get a terminal console on Windows.

Preparation

Within the host os (usually Windows), create a folder C:\Users\virtualbox\devbox-19.09.

Make sure your windows user has read/write permission on the folder. On Windows, right-click on the folder and follow Properties  Sharing.

This will be your shared folder between the host (Windows) and the guest (/vagrant in the linux box). We will use the term SHARED_DIR to refer to this folder throughout the rest of the guide.

The SHARED_DIR contains personal configuration files.

typical SHARED_DIR
C:\Users\virtualbox\devbox-19.09
   ├── [ssh-keys] (1)
   │      ├── xxxx_rsa
   │      ├── xxxx_rsa.pub
   │      ├── xxxx_github_rsa
   │      └── xxxx_github_rsa.pub
   ├── box.dhall (2)
   ├── local-home.nix  (2)
   └── [local-configuration.nix] (3)
1 ssh-keys folder
2 user configuration
3 optional system configuration

You don’t need to create any of the user configuration files. If the file is not yet present in the SHARED_DIR, it will be created by updateConfig. You should then edit the file and fill the requested information[1].

Type updateConfig again to activate those changes.

  • If you don’t use the default host location for the SHARED_DIR, you will need to manually change the setting using the Virtualbox UI. When doing so, please keep the name vagrant together with an empty mount point.

  • When editing files from the SHARED_DIR, maintain the unix file encoding.

ssh keys

Without your personal ssh key pairs you won’t be able to pull or push to bitbucket repositories.

  1. in the SHARED_DIR, create a directory ssh-keys.

  2. in this ssh-keys subfolder, copy your Bitbucket key pair, rename them 'cirb_rsa' and 'cirb_rsa.pub' respectively if the filename differs.

  3. in the ssh-keys subfolder, copy your Github key pair, rename them 'cirb_github_rsa' and 'cirb_github_rsa.pub'. If you use the same key pair, copy the previous pair and rename accordingly.

Box import

Get the latest devbox-19.09.x file from Artifactory

To import the ova image file, just click on the file. The virtualbox UI will pop up. You can accept all defaults.

Usage

The provisioning of the devbox is done by 2 repositories (both on Bitbucket):

The devbox repository is quite small. It holds the ignition files necessary to bootstrap a devbox. This repository will be automatically present in the devbox in the ~/bootstrap folder.

The dotfiles repository holds the user configuration files. The git repository is hidden from the user through the use of vcsh. The vcsh program is also helpful in case of crash to recover a clean user state.

User provisioning

The most common operation is to modify your user configuration through box.dhall in your SHARED_DIR.

You might also want to modify your user configuration through the included home-manager. Such more advanced and optional configuration is done by editing the local-home.nix file which also sits in your SHARED_DIR.

To activate these changes, you execute the command updateConfig in a console.

Devbox update

If you wish to update the files that provision the devbox, you do so by pulling them from the dotfiles and devbox source repositories.

The command updateUser defined in ~/.zsh_custom/fun.zsh does this before activating the user configuration (updateConfig would only do the later).

You can also do these steps in a more fine grain way. For instance to pull the changes from dotfiles:

→ cd ~/bootstrap
→ git st (1)
→ git pull
1 get the status

Similarly for the dotfiles:

→ vcsh dotfiles
→ git st
→ git pull --ff-only
→ exit (1)
1 don’t forget to exit the vcsh mode.

Advanced system provisioning

The advanced user might want to update the system configuration. This is useful to enable middleware such as a postgresql database, an elk cluster or a minishift.

If you want to go into this level of configuration, first import in your SHARED_DIR the example configuration file:

curl -o local-configuration.nix https://raw.githubusercontent.com/CIRB/devbox/master/system/local-configuration.nix

To publish your system configuration change, you execute the command updateSystem in a console.

Operating system (nixos)

The devbox is based on NixOS version 19.09. NixOS is a Linux distribution based on the nix package manager.

Compare to more traditional distributions, NixOS favors declarative settings and brings to the table the following properties:

  • reproducability: everything is reproducible (packages are identical binary wise).

  • atomicity: configuration changes can be rollback.

  • isolation: on a project, user or system level, processes are built in an isolated environment.

By using nixos we are striving for a system that we can destroy and re-create at will without fuss.

The entry point for the system setup is the /etc/nixos/configuration.nix file.

Useful command

# rebuild nixos after changing the system configuration file
→ sudo nixos-rebuild switch

# clean-up the store (whenever the disk usage is too high)
→ sudo nix-collect-garbage -d

Windows manager (xmonad)

The devbox comes with a tiling windows manager called xmonad. Such a minimal approach has been chosen for 3 reasons:

  • Efficiency: the box needs to consume as minimum CPU/Mem resources as possible

  • Simplicity: the window manager is basic but yet quite flexible

  • Practicality: the desktop is distraction free.

If you want another UI, based on Gnome, KDE, i3wm or Pantheon, that’s easy enough. Please ask for assistance on our #CICD Team channel[2]

Minimal cheat sheet

Command Description

Ctrl+Space[3]

Open app launcher

Super+t

New terminal

Super+1..9

Go to nth desktop

Super+Right

Go to next desktop

Super+Left

Go to previous desktop

Super+Shift+1..9

Switch to nth desktop

Super+Space

Change between horizontal, vertical and stack layout

Super+Tab

In stack layout, switch the stacked window

Super+q

Close current window

Super+m , Super+h

Resize windows

Super+w

Put window back into tiling

F 1

Open this README in a browser

Layout

The active layout is displayed on the status bar:

layout indicator
Symbol Name

horizontal

horizontal tiled

vertical

full (stack)

vertical tiled

You can also change layout by clicking on the symbol in the status bar.

Terminal emulator (termite)

The devbox uses the termite terminal.

Command Description

Control+Shift+c/v

Copy/paste to/from clipboard

Control+Shift++/-/=

Increase/Decrease/Reset font size

Control+Tab

Start scrollback completion

Control+Shift+Space

Start selection mode

The default user shell is zsh which allows for predictable history and completion.

You can use autojump to quickly navigate from one directory to another using the j shortcut:

# Go to your directory (you can omit the `cd`)
→ cd projects/cicd/puppet/bos
# Teach autojump that you like bos
→ autojump -i 20
# Open a new terminal. You can now quickly go to ~/projects/cicd/puppet/bos
→ j bos

If you wish to add some plugins to oh-my-zsh the setting is here[4].

Another useful shortcut is sshi to quickly log to a remote machine. It is the equivalent of ssh -i ~/.ssh/xxxx_rsa $LOGINID@$1.

Application launcher (albert)

Albert is an application launcher similar to Alfred in OS X. You can launch albert with Control+Space[5]. Use it to:

  • open any application, directory or file by typing its name (type 'File' to open Nautilus)

  • shutdown/reboot the vm

  • search the internet with gg

Projects (mr)

The myrepos project help you to automate the cloning of multiple git repositories. The usage of mr is optional but quite useful when you work on dozens of repositories.

Mr files are pointers to repositories you want to clone. These are defined in the same CIRB dotfiles repository. You may propose any new repository definition via a pull request.

To activate an available repository in your devbox, add the name of the file to the mr.repos list in 'SHARED_DIR/box.dhall'.

As soon as you type updateConfig on the command line, the enabled repos will be automatically cloned and configured into your box. For instance if puppet-bos is in the list, you will find a ~/projects/bric/box/puppet-bos folder. In that case, the next step would be to go in that folder and enter make on the command line.

To update all registered mr repository, do

→ mr -f up (1)
1 you usually don’t want to automatically update such repository on provisioning. That’s the reason behind the -f (force) flag.

You can ask a git status of all your repositories with

→ mr st

Local environment (direnv & lorri)

Lorri removes the need to nix-shell into a project folder. It prevents the project derivation to be garbage collected as the dependency graph is rebuild whenever the project dependencies changes.

Tips & Tricks

The customization of the devbox operates at different levels and ensure configuration - personal or shared - survives the destruction of the box.

Install local packages

To install a package, just do:

→ nix-env -i geany (1)
1 install the geany package for the vagrant user. The suffix env expresses the fact that only the user environment is affected.

This imperative approach is useful for testing but the change is not persistent. When you settle down on a list of packages, please add them to local-home.nix to gain persistence.

You can quickly search for packages online at nixos.org

Open a shell

You can open a shell with the nix-shell commmand. For instance to open a shell that have JAVA_HOME set:

→ nix-shell -p jdk11

Generic dotfiles

The standard CIRB dotfiles repositry maintains a very general default configuration for the devbox. It is continuously evolving and improving. Please feel welcomed to propose any change via a pull request.

Personal mr repositories

You can register any source repository by adding it to mr.config in SHARED_DIR/box.dhall. This is for personal configuration that you don’t want to share. After saving the file, you update your box with updateUser.

Storing the box configuration file

You might want to go one step further and store your box.dhall file on Github or the like. This is pretty easy thanks to the powerful dhall configuration language. Replace the content of SHARED_DIR/box.dhall with one simple line containing an url that points to the config file. Here is an example:

SHARED_DIR/box.dhall
https://raw.githubusercontent.com/PierreR/devbox-config/master/box.dhall

Life cycle

The devbox will be maintained continuously. Two version are expected to be released every year in April and October following the NixOS release cycle.

The versioning scheme follows the one of NixOS. For instance 19.09.buildnumber.

When a new major version is released, breaking changes are expected. It is required to destroy and re-create a box from scratch. An operation that take no longer than a few minutes. Please look at the Changelog to gather information and read the updated README if necessary. You would commonly need to make some adjustements to your personal configuration file such as dhall.box or local-configuration.nix before importing them in the new SHARED_DIR.

Keep the old box and the new one for a while and make sure you aren’t losing any customization.

Question, support, discussion are handled in the team #cicd channel.

Without virtualbox

It is possible to use the devbox without the use of Virtualbox for instance within a Nixos linux or a vmware nixos guest.

If you have a nixbox linux up and running, follow these steps:

  1. mount your configuration into your nixos box, for instance `/vagrant/shared`[6]

  2. add an env variable named SHARED_DIR that points to the mounted dir[6]

  3. git clone ssh://git@stash.cirb.lan:7999/devb/devbox.git bootstrap

  4. cd bootstrap

  5. make home-manager

  6. (logout/login)

  7. make user

If you want to use vmware workstation, we can provide an up-to-dated vmdk image. You still need to choose a SHARED_DIR to host your configuration files as explain in the preparation paragraph.

Troubleshooting

Ssh-key with passphrase

Using cirb ssh-keys with a passphrase will currently cause difficulties with mr, a tool we use to handle git repositories in a declarative way. It will prevent some of the automation to go through (nothing critical though). If possible, please regenerate a pair of keys with no passphrase and register them in stash.cirb.lan[7].

For Windows 10 users

Everything usually just works on Windows 10. That said, keep in mind that various issues have been reported concerning incompatibilities between hyper-v and virtualbox. It is likely that such compatibilities arise again after a windows or virtualbox update.

We will try to offer a hyper-v compatible box using packer (no virtualbox) in the future. Packer has just announced such a support. Stay tune.

In the meanwhile, if you face such a issue you will have to disable hyper-v to get a working devbox. Please proceed as follow:

  1. disable hyper-v

  2. use virtualbox as hypervisor

  3. Use the devbox for your docker need (docker is included out of the box) or install some docker tools based on virtualbox

The only use of hyper-v we have detected so far is by the latest version of Docker for Windows.
For Mac users

There are known critical issues with Virtualbox and VPN clients on macosx.

For VMware users

There is currently an issue with packer & VMware Workstation. If you want to use VMware, please ask for assistance.

FAQ

Eclipse: how to add plugins ?

If you enable eclipse in $SHARED_DIR/box.dhall, you might want to add the egit and m2e plugins.

enable egit and m2e
→ cd ~/bootstrap
→ make eclipse-extraplugins

If you miss other plugins, let us know.

How is the box generated ?

The box (of about 4G) is generated in two stages. The first step generates a minimal generic vagrant box file with packer (currently packer-1.4.4). The second step applies some customization to produce a more heavy tailored box (desktop, …​).

Stage 1

  1. On a linux host[8], use the provided nixbox.sh script to clone the nixbox repository within the shared directory.

  2. On the host OS, in the nixbox directory, execute the following commands:

    packer.exe build nixos-x86_64.json --only virtualbox-iso

Stage 2

  1. Create the SHARED_DIR

  2. Import the ova image file using the Virtualbox UI.

  3. Start the imported vm, login as vagrant

    1. cd into /etc/devbox-19.09.x

    2. make system

    3. make home-manager

    4. reboot

    5. cd into /etc/devbox-19.09.x

    6. make user

  4. Export the vm

  5. Push the ova to a mirror site.


1. a comment section is present at the top.
2. If such request is popular, we will add an help section about it.
3. you can change this binding in box.dhall
4. let us know if such customization is of any interest and we will add a key in dhall.box for it.
5. you can change the hotkey in the box.dhall configuration file.
6. not required if you don’t want to configure anything
7. We will improve the situation in a future release if it is tagged as a major concern.
8. Cloning in Windows will cause many encoding issues!