SCIAMA
High Performance Compute Cluster
Jupyter Notebook on SCIAMA
While Jupyter notebook support is installed as part of the supported cpython3 modules on SCIAMA, running a notebook server directly on the login nodes should be avoided to prevent unnecessary congestion there. Instead, it can be run on the compute nodes. This article describes how to do so and how to access the notebook server from either the login nodes or directly from your own computer.
Running Jupyter notebook server on SCIAMA
So far, Jupyter notebook is supported for the cpython/3.7.1 for both intel_comp/2019.2 and gnu_comp/9.1.0. In order to start a new server, simply load this module, e.g.
Now start a server on a compute node using the following command:
Once executed, you will have to wait until the resources have been allocated. On success, you will see the following output:
....
Resources allocated. Waiting for server to finish init
...........
CLIENT LOCAL:44896 REMOTE:node108:53604 TOKEN:27630cd74b4b88de1fdc63e7ed8c7b084b08064aebf828a2
Jupyter nodebook server is RUNNING. You can access it via a browser on this node at the following URL:
http://localhost:44896/?token=27630cd74b4b88de1fdc63e7ed8c7b084b08064aebf828a2
Press CTRL-c to terminate server
The server keeps running as long as you keep this shell open or explicitly terminate it using CTRL-c. To access it, check out the sections below.
Managing resources
While the standard config (single core with 4 GB/core on the jupyter.q) should be sufficient for most users’ needs, you can also add the standard sbatch arguments to the initial sciama-notebook command to allocate more cores/memory or switch to an alternative queue e.g. the himem.q for extreme memory demands e.g.
Using Jupyter notebook on compute nodes
In order to use jupyter notebook with the server on the compute nodes, simply open a browser on the login nodes (e.g. firefox) and enter the URL stated in the server output. X-forwarding may be sometimes a bit slow. In that case you may want to use an X2Go login with a remote desktop instead.
Using Jupyter notebook on own computer
You can also access the remote jupyter server with a browser on your own computer. This provides a more responsive experience, but also requires another step depending on the OS of your computer.
Linux
For Linux (and potentially Unix), we provide a convenient script. Simply copy this executable script from its location on SCIAMA
/opt/apps/pkgs/cpython/jupyter-sciama-remote.linux
onto your own computer and run it on YOUR computer with the login details as argument e.g.
where you have to use the SAME login node you used to start the server (as described above). If successful, you should see the following output:
Jupyter SSH tunneling: You can access it via a browser on this computer at the following URL:
http://localhost:37652/?token=2736d2a30d799fe9d23e0df13b8abaecb55a0d5d3b92c178
Now you can simply access jupyter notebook from a browser on your computer using the URL provided in the output.
macOS
For macOS, a similar script to the one provided for Linux can be found at
/opt/apps/pkgs/cpython/jupyter-sciama-remote.macos
The only difference in the process is that you always have to provide a local port as a second argument (due to some issues with the automatic port allocation) e.g.
Windows
On Windows, it is a bit more complicated as you will have to set up the SSH tunneling yourself. For detailed instructions, please see HERE (or google). As the remote port, use the one listed after ‘CLIENT LOCAL’ in the server output (e.g. 44896 in the example output shown above), for your local one, you can use any free port (e.g. try 8888).
Once the tunnel is established, copy the URL stated in the server output into the address bar of a browser on your own computer and replace the port number in it with the one picked by you - e.g. in our example replace
http://localhost:44896/?token=...
with
http://localhost:8888/?token=....
Now open the URL and you should be greeted by the jupyter notebook interface.