Better project environment management with direnv and asdf

Load per project environmental settings and runtimes while ridding yourself of multiple runtime managers (rvm, nvm, etc.)

Each software project exists in its own world, particularly with web applications. There is a specific set of environmental variables needed, a specific version of the language runtime, and perhaps a specific version of node for the JavaScript build tools. Docker and other containerization efforts package up this little world, and it works. However, some people, myself included, like to develop without that layer of abstraction and leave docker for deployment. So, when developing in the base operating system, we need to manage this environment for each project. Many tools have arisen to manage the various language runtime versions you may need: rvm/chruby for Ruby, nvm for Javascript, exenv for elixir, a whole cattle car for python that no one is happy with. Solutions for environmental variables are less clear and more dependent on the problem. The least automatic, but easiest, method being sourcing a shell script containing export s on entering the project.

Ideally, our computer would just set up this environment whenever we switched between projects. With two tools, direnv and asdf we can get close to this ideal. As soon as you cd into your project directory, your environment variables will load for you, along with proper versions of your run times and build tools. Either tool can be used in isolation, combined, or as I’ll shown at the end, direnv can be installed and loaded as an asdf plugin for added benefits. Let’s see the two in action first:

# create and enter a new project
~ ➜ mkdir demo-project
~ ➜ cd demo-project

# add our secret
~/demo-project ➜ echo 'export SOME_SECRET=foo' > .envrc
direnv: error /Users/michael/demo-project/.envrc is blocked. Run `direnv allow` to approve its content

# allow our secret
~/demo-project ➜ direnv allow

# set our project's ruby to 2.5.8
~/demo-project ➜ asdf local ruby 2.5.8

# create a test script that shows ruby version and our secret from the environmental variable
echo "puts \"Hello from ruby: #{RUBY_VERSION} \nHere's your secret: #{ENV['SOME_SECRET']}\"" > script.rb

#leave the directory
~/demo-project ➜ cd ..

# ruby is 2.7.5 and secret is not set
~ ➜ ruby --version
ruby 2.7.5p203 (2021-11-24 revision f69aeb8314) [x86_64-darwin21]
~ ➜ echo $SOME_SECRET

# enter the project and ruby is 2.5.8 and our secret is alive and well
~ ➜ cd demo-project
~/demo-project ➜ ruby script.rb
Hello from ruby: 2.5.8
Here's your secret: foo

# show asdf's config file
~/demo-project ➜ cat .tool-versions
ruby 2.5.8

asdf

asdf aims to be a universal runtime version manager. Runtime version managers allow you to switch quickly between tools, which is especially useful when working with interpreted languages. For anyone who has worked in Ruby, they likely employ a version manager such as rvm, rbenv, or chruby to switch between ruby versions depending on the project. asdf solves this problem in a generic way so you can use the same tool for ruby, elixir, erlang, and JavaScript. A host of other run times are supported by the community.

Install asdf via package manager after installing dependencies. For example, on macOS:

brew install asdf

Add the proper shell hooks to your shell config.

echo -e "\n. $(brew --prefix asdf)/libexec/asdf.sh" >> ${ZDOTDIR:-~}/.zshrc

Tip: Add the hooks earlier in your shell loading process (e.g. profile or zshenv) if you are having trouble getting it to integrate with your editor. For example, moving the hooks into zshenv allowed my emacs install to properly pick up the correct ruby version when running tests.

Install plugins for the languages/run times you need.

xcode-select --install
brew install openssl readline
asdf plugin add ruby https://github.com/asdf-vm/asdf-ruby.git

Note that asdf wraps installers for given runtimes, it does not replace them. So in this example, it is wrapping ruby-build which actually handles building and installing ruby. Therefore, for each plugin you need to install dependencies for the install tool. You may wonder what the point is if it is just wrapping the tools. In short, it allows a simple and universal interface for the builders/installers. You don’t have to remember the switches in the install tools or how to update. Contrast installing node and ruby:

# find node versions
nvm ls-remote

# install specific version
nvm install 8.16.2

# update ruby-install
ruby-install

# look up version online you want if not latest
ruby-install ruby 2.5.8

Same number of commands on the asdf side, but with a standardized syntax:

asdf list all nodejs
asdf install nodejs 8.16.2

asdf list all ruby
asdf install ruby 2.5.8

Given the standardized syntax, for a project (or globally), you can then create .tool-versions file which specifies the runtimes for a given project. asdf will actually do this for you via command line calls as we showed in the demo above

~ mkdir demo-project; cd !$
~/demo-project ➜ asdf local ruby 2.5.8
~/demo-project ➜ asdf local nodejs 16.13.1
~/demo-project ➜ cat .tool-versions
ruby 2.5.8
nodejs 16.13.1
~/demo-project ➜ ruby --version
ruby 2.5.8p224 (2020-03-31 revision 67882) [x86_64-darwin21]
~/demo-project ➜ node --version
v16.13.1

direnv

direnv is a Go program that hooks into the shell that allows per directory variables and arbitrary scripts to be run on changing the work directory to the project. Installation is straightforward via package manager. Afterwards you need to add the proper line to your shell config. There is a quick demo, but I’ll summarize with a few additional tips.

1. In a project directory, create an .envrc file with your environmental variable of choice: echo export SOME_SECRET=foo > .envrc
2. You will get a message saying it is blocked. This is a security feature. Type direnv allow since you trust the secret you are setting. Do not trust new .envrc files without inspecting them since arbitrary code can be run.
3. direnv will load the variable. echo $SOME_SECRET to prove this and cd out of the dir. direnv will dutifully unload the variable. echo $SOME_SECRET to prove this. cd back into the directory and confirm your variable is back.

Tip 1: cd back into the project directory. Your terminal will show logs similar to this:

direnv: loading ~/temp/.envrc
direnv: export +SOME_SECRET

We can prevent this by adding the line export DIRENV_LOG_FORMAT""= to our direnv config as so:

mkdir -p ~/.config/direnv
echo 'export DIRENV_LOG_FORMAT=""' >> ~/.config/direnv/direnvrc

You should no longer see output on entering/exiting directories with .envrc files in place.

Tip 2: If using git for your project, add .envrc to your local (or better your global) ignore so you don’t commit this file accidentally:

echo .envrc >> ~/.gitignore or echo .envrc >> ~/.gitignore_global

Integrating asdf and direnv

As stated at the top, you can install either or both of the above tools depending on your aims. If going with the latter, you might consider installing asdf via package manger and then installing direnv as a plugin in for asdf. According to the plugin documentation this approach provides several benefits:

• increased speed per-project because the tool versions are cached for asdf by direnv
• no need to reshim via asdf

The potential drawback to this approach is the complexity. For example, you have to update direnv via asdf as opposed to using package manger updates. Putting a cache into place also could create debugging complexity down the roadI haven’t encountered this yet in my limited usage. I will update if I do..

Installation as a plugin is the same as our language runtimes:

asdf plugin add direnv
asdf install direnv latest
asdf global direnv latest

Rather than setting up direnv shell hook, we route through asdf and provide a helper to do the same at command line usage:

eval "$(asdf exec direnv hook zsh)"
direnv() { asdf exec direnv "$@"; }

We then source asdf helper function in the direnv config:

source "$(asdf direnv hook asdf)"

Finally we can use it in our projects via:

use asdf

If you turned off logging in direnv, you may want to turn it back on briefly to see the loading of asdf tools on entering a directory.

Conclusion

In this post we showed how to setup asdf to reduce our runtime management tools. We added direnv into the mix to support arbitrary per-project setups, particularly for environmental variable management. Finally, we showed how we could integrate these two tools to improve speed and ergonomics of asdf shims.