new opam features: local switches
Among the areas we wanted to improve on for opam 2.0 was the handling of
switches. In opam 1.2, they are simply accessed by a name (the OCaml version
by default), and are always stored into
~/.opam/<name>. This is fine, but can
get a bit cumbersome when many switches are in presence, as there is no way to
sort them or associate them with a given project.
A reminder about switches
For those unfamiliar with it, switches, in opam, are independent prefixes with their own compiler and set of installed packages. The
opam switchcommand allows to create and remove switches, as well as select the currently active one, where operations like
opam installwill operate.
Their uses include easily juggling between versions of OCaml, or of a library, having incompatible packages installed separately but at the same time, running tests without damaging your "main" environment, and, quite often, separation of environment for working on different projects.
You can also select a specific switch for a single command, with
opam install foo --switch other
or even for a single shell session, with
eval $(opam env --switch other)
What opam 2.0 adds to this is the possibility to create so-called local switches, stored below a directory of your choice. This gets users back in control of how switches are organised, and wiping the directory is a safe way to get rid of the switch.
Using within projects
This is the main intended use: the user can define a switch within the source of
a project, for use specifically in that project. One nice side-effect to help
with this is that, if a "local switch" is detected in the current directory or a
parent, opam will select it automatically. Just don't forget to run
eval $(opam env) to make the environment up-to-date before running
The interface simply overloads the
switch-name arguments, wherever they were
present, allowing directory names instead. So for example:
cd ~/src/project opam switch create ./
will create a local switch in the directory
~/src/project. Then, it is for
example equivalent to run
opam list from that directory, or
opam list --switch=~/src/project from anywhere.
Note that you can bypass the automatic local-switch selection if needed by using
--switch argument, by defining the variable
OPAMSWITCH or by using
eval $(opam env --switch <name>)
In practice, the switch contents are placed in a
_opam/ subdirectory. So if
you create the switch
~/src/project, you can browse its contents at
~/src/project/_opam. This is the direct prefix for the switch, so e.g.
binaries can be found directly at
_opam/bin/: easier than searching the opam
root! The opam metadata is placed below that directory, in a
Local switches still share the opam root, and in particular depend on the repositories defined and cached there. It is now possible, however, to select different repositories for different switches, but that is a subject for another post.
Finally, note that removing that
_opam directory is handled transparently by
opam, and that if you want to share a local switch between projects, symlinking
_opam directory is allowed.
This feature has been present in our dev builds for a while, and you can already use it in the current beta.
Limitations and future extensions
It is not, at the moment, possible to move a local switch directory around, mainly due to issues related to relocating the OCaml compiler.
Creating a new switch still implies to recompile all the packages, and even the compiler itself (unless you rely on a system installation). The projected solution is to add a build cache, avoiding the need to recompile the same package with the same dependencies. This should actually be possible with the current opam 2.0 code, by leveraging the new hooks that are made available. Note that relocation of OCaml is also an issue for this, though.
Editing tools like
merlin can also become an annoyance with
the multiplication of switches, because they are not automatically found if not
installed in the current switch. But the
user-setup plugin (run
opam user-setup install) already handles this well, and will access
tuareg from their initial switch, if not found in the current one. You will
still need to install tools that are tightly bound to a compiler version, like
ocp-index, in the switches where you need them, though.
Jeremie Dimino (11 May 2017 at 8 h 27 min):
Thanks, that seems like a useful feature. Regarding relocation of the compiler, shouldn’t it be enough to set the environment variable OCAMLLIB? AFAIK the stdlib directory is the only hardcoded path on the compiler.
Louis Gesbert (11 May 2017 at 8 h 56 min):
Last I checked, there were a few more problematic points, in particular generated bytecode executables statically referring to their interpreter; but yes, in any case, it’s worth experimenting in that direction using the new hooks, to see how it works in practice.
Jeremie Dimino (12 May 2017 at 9 h 13 min):
Indeed, I remember that we had a similar problem in the initial setup to test the public release of Jane Street packages: we were using long paths for the opam roots and the generated #! where too long for the OS… What I did back then is write a program that scanned the tree and rewrote the #! to use “#!/usr/bin/env ocamlrun”.
That could be an option here. The rewriting only need to be done once, since the compiler uses
ocamlc -where/camlheader when generating a bytecode executable.
OCamlPro is a R&D lab founded in 2011, with the mission to help industrial users benefit from state-of-the art programming languages like OCaml and Rust.
We design, create and implement custom ad-hoc software for our clients. We also have a long experience in developing and maintaining open-source tooling for OCaml, such as Opam, TryOCaml, ocp-indent, ocp-index and ocp-browser, and we contribute to the core-development of OCaml, notably with our work on the Flambda optimizer branch.
Another area of expertise is that of Formal Methods, with tools such as our SMT Solver Alt-Ergo (check our Alt-Ergo Users'). We also provide vocational trainings in OCaml and Rust, and we can build courses on formal methods on-demand. Please reach out, we'll be delighted to discuss your challenges: email@example.com or book a quick discussion.