The Quantum Physicist The Quantum Physicist - 15 days ago 11
Python Question

Is JupyterHub kernel safe across users?

I'm using JupyterHub to share computational power of a big computer among some users. The software that's primarily used is a

ctypes
extension Python script that uses a sophisticated C/C++ code. This code isn't invulnerable to memory problems and crashes.

My question is: If a low-level problem happens with one user and his kernel gets, say, a segmentation fault, will that crash the main server by design and get all users to lose their kernel information? Or is it designed to create a new server for every user that logs in, so that such problems don't happen?

Answer

Even if you were using straight Jupyter Notebook instead of JupyterHub, each kernel is a single process, that runs kind of independent of the notebook server. Crashes of individual kernels will not take down the notebook server.

Check out the architecture documentation. We've been running a setup with a single Jupyter Notebook instance (not even JupyterHub, because Windows :/) for about 3 years now. The only problems occuring are due to resource constraints (e.g. single kernel takes up a lot of memory), but that's solvable on both the OS and organisational level.

Comments