Windows 10 with WSL 2 – Part 3: In-depth installation guide for WSL 2

In this article series, I will explain the benefits of using Windows 10 with Windows Subsystem Linux 2 for ML problems.

The article is published in three parts: In part one we talk about what you need to know before using GPU-accelerated models on your laptop. Then we go through the benefits of using WSL 2 and discussed why you might want to avoid Mac OS in machine learning. And this article is an in-depth install guide for WSL 2 in case you run into problems.

In most cases, you should be fine following the Windows Subsystem for Linux Installation Guide for Windows 10.

If those instructions failed you, however, the most probable cause is earlier WSL installations. While it is not necessary to use Microsoft Windows Insider Builds to run WSL 2, it seems like many problems have been fixed in those nightly builds. In order to get everything working, you should start from a clean set-up, so the first thing to do is open PowerShell as an administrator and run the following:

wsl --list --verbose

If nothing comes up, it’s all good: you have no lurking WSL installations and the installation should be rather simple. Things can come back to haunt you even if you have removed the WSL installation, though.

If you have tried and failed to install WSL 2 and have installed Docker for Desktop for Windows, you might have two WSL thingies in there. They might affect the installation or they might not; if it is possible to remove Docker and reinstall it later, all the better.

1) Using encrypted hard disks and directories

The first problem you might encounter when installing WSL 2, especially if you have installed WSL 1 before, is an issue with encrypted hard disks and/or directories, filesystems, whatever. What happens is that the encryption somehow prevents the system from changing the command:

wsl --set-version 2

The symptom of this problem should be that, when you install a distro, it is still the WSL 1 version instead of WSL 2.

You should be able to check this with the `wsl –list –verbose` command, which will also tell you the distro’s version number. More detailed information is available here in this comment in GitHub by dmchurch.

2 ) Docker context

The next problem could be related to your Docker. The command `wsl –list –verbose` should return three things: your distribution and two Docker entries. Also,

docker context ls

should give you the end point with something like:

npipe:////./pipe/docker_engine

The important thing is that if you upgraded your distribution without deleting it for good, there probably still is some kind of Docker client installed on your WSL machine. Get rid of all Docker related stuff. It might be a good idea to reinstall Docker for Windows.

If you are lazy, you might want to try this one trick that seems to work:

• Make sure you have purged all Docker clients (but maybe check out this comment in GitHub.
• Then do this fun trick: Switch from Linux containers to Windows containers and disable WSL, then switch back to Linux containers and enable WSL – you can check this comment in GitHub.
• Remember to restart your distribution using `wsl.exe –terminate <distro_name>` – comment in GitHub.

3) File permissions

You could encounter this file permission issue when installing the packages and libraries of programming languages (especially Ruby). Windows has its own user privileges over the WSL ones. This means that WSL has to provide all privileges to some important things, which are now included in your PATH.

In practice, this means that all security-sensitive things, like Ruby gems, will likely fail, as they want to protect their geeks from doing stupid things. But do not fix the file permissions! You will just break the WSL. What you want to do instead is to change how the WSL-mounted filesystem masks operate by following these instructions:

• Do this.
• Then restart the distro like in step two of the Docker stuff.

There are a lot of different workarounds to many file-permission-related problems around the internet, and following them before doing this is a good way of tying your system in such a knot that you will want to erase everything and start over. Try this first.

4) Internet access blocked

Your virus protection and firewall may occasionally disconnect WSL 2 from the internet (this hasn’t happened to me since my first successful install, though). It’s a good thing that WSL 2 will be automatically disconnected from the internet if you have updated your system and have unprocessed security warnings. This problem usually starts after a reboot, switching to another user or having the machine sleep long enough.

For me, the solution was simply re-enabling “Reputation-based protection” under “App & browser control” in Windows Security (it gets disabled after pretty much every Windows 10 update) and then resetting the security system in PowerShell. There’s more about this in my comment in GitHub.

It seems that this has been fixed if you are using CUDA and thus have build version 201xx.

If you do not have a firewall-based problem, WSL 2 is also known to get mixed up with the Hyper-V networking adapters if your system configuration changes, for example if you happen to install a VPN connection. These problems do not surface immediately, but only after switching accounts or having the machine in power-saving mode long enough.

There are a lot of bug reports about connectivity issues, but it seems that the best long-term solution is just booting your device after installing or updating things that affect its network behavior, even if Windows doesn’t require a reboot. So if you lose network connectivity, reboot first before trying to apply any fixes that could break your WSL 2 and force you to do a distribution reinstall.

It seems like this problem has also been fixed in the 201xx build.

5) Other issues

WSL 2 can use a lot of RAM, so you might want to consider limiting the amount of RAM to run both Chrome and WSL 2. If you do not run processes that need a lot of memory, WSL 2 should run fine with as little as 512 MB. More about this in GitHub, here.

This memory problem also seems to have been fixed in 201xx, but I’m not entirely sure.

If you reboot WSL 2 manually by using terminate in PowerShell, always launch it first with the proper console rather than Visual Studio Code; Visual Studio Code seems to have limited user rights, so you will reboot the system with limited file system access.

This could also be fixed in the 201xx build, or maybe it’s because I am now running it with an admin user on this machine. It might be a good idea to run WSL 2 with a sys-admin user, as many issues are potentially related to running WSL 2 without admin rights.

It is very important to store your projects in the Linux filesystem and not in the ‘/mnt’ path. All Virtual Machines have some problems with fast FileIO. They are often solved by caching, but because WLS 2 lets you manipulate host applications and thus simultaneously access the host filesystem, caching can sometimes have disastrous consequences.

Your home directory is often the Linux filesystem. You can start the WSL 2 distribution terminal and run `Explorer.exe .` (notice the one extra dot after ‘exe’) to locate it on your machine. One of the reasons for having your projects in the Linux filesystem is that git slows down with larger projects, even when most of the data is in `.gitignore`.

When you are using the Linux filesystem, it will use disk space as a virtual disk mounted from the host file system. If you have unfortunate partitioning on your machine and could run out of disk space, you can migrate your current distribution to a larger hard disk drive, more in GitHub.

“Works on my machine”

These are all the basics you should be aware of when running into problems with WSL 2. The list is not necessarily exhaustive, but if you haven’t messed up your system with previous installs, you are running it as an admin, and you are using the latest Windows 10 builds, you should have very few problems.

I have had zero incidents so far after a successful, proper initial installation, and I’ve been using this for 9 months on two different laptops. When I was using Mac OS, I had many issues regardless of the releases’ purported stability, and major OS updates would often break things.

I’ve had none of these problems lately and can be sure that “works on my machine” means that it works on Linux (even though that’s not always the same as “works on the server” due to processor architectures, but meh, this is the best you can do with a laptop).

Thank you for your time and happy coding!