Remote Jupyter Notebook Setup

If you would like to have the freedom of working remotely, this article is for you. You can easily go through the following instructions to set up a remote connection to your server and run your machine learning experiments in a Jupyter notebook or even JupyterLab.

Image by pxhere

UPDATE (Nov. 30, 2020): I have found VS Code’s automatic port mapping quite convenient. You just need to set up your environment, say using conda. Go to the terminal inside VS Code. Type the jupyter notebook command as usual and the ports will be mapped automatically. Here is the result:

Details of port mapping:

Good job to the developers of this awesome feature! 👏👏👏

If you still like to learn more about the manual way, please continue reading.

I have done the following experiments with an Ubuntu server with an NVIDIA GPU installed.

Let’s SSH to our server and start the process. First of all, you can install Anaconda based on instructions under this website.

With the following commands you can easily set up and activate your conda environment, specifying the name and your preferred Python version. Here we choose Python 3.6 and ‘my_env’ as the name of the environment.

conda create --name my_env python=3.6
source activate my_env

Next step is to install the dependencies required, such as TensorFlow, PyTorch and other packages needed. If your target repository has a ‘requirements.txt’ file, you can use:

pip install -r requirements.txt

We are now ready to run Jupyter notebook on our server. Please note that conda environment has Jupyter notebook preinstalled.

nohup jupyter notebook --no-browser --port=8081 > ~/nohups/my_env.out &

Let’s have a look at this command. ‘nohup’ runs the process in background. This way, if you lose connection to your server, the process does not get terminated. jupyter notebook is the main command. no-browser stops the notebook from starting with a browser, as we like to open it on our local machine. We can specify the port using port parameter. We would like to have a record of the process to access it later using nohup. It can be done using the command: > ~/nohups/my_env.out. The last ampersand character prints out the process PID, in case we need it later. Store it for your future reference.

Each Jupyter notebook has a token for security, to find the token, you can use the following command to see the nohup process output. To exit, you can use Ctrl+C (Does not terminate the process).

tail -f ~/nohups/my_env.out

Let’s keep the server terminal open and open a new one to connect the local machine. On your local machine’s terminal, run the following command to connect to the Jupyter notebook process remotely:

ssh -N -f -L localhost:8881:localhost:8081 username@[IP Address]

The left side of the colon specifies the port that you would like to use on your local machine. The right part is the port specified in server for the notebook process. You should also specify the username and IP address to access the server, e.g. me@100.200.300.15.

You can now open a browser and access the specified localhost port, e.g. localhost:8881. Enter the token and Voila! You have Jupyter notebook running on your server and being accessed locally.

There is still one problem though, the notebook does not probably list your environment as a kernel. You can check that by clicking on ‘New’ at top-right of the screen.

‘my_env’ is not listed.

To fix this, open the server terminal and make sure you have ‘my_env’ environment activated. Run the following commands:

conda install ipykernel
python -m ipykernel install --user --name my_env  --display-name my_env

If you refresh the notebook in your local machine’s browser, you should have ‘my_env’ listed as an environment. To test , open a new notebook with that kernel and run the following commands. You should see the Python version running and Anaconda name.

import sys
print(sys.version)

In case you would like to delete a kernel from Jupyter notebook, open your server’s terminal and run this command to get the list of available kernels:

jupyter kernelspec list

To delete a specific kernel, do as follows:

jupyter kernelspec uninstall kernel-name-to-delete

The following command could be used to delete a conda environment:

conda env remove --name conda-env-to-delete

I hope you enjoyed this tutorial.

Please take a moment to leave a comment or clap to let me know if this article was helpful.

What extra information would have been helpful? What else would you like to learn about?