Use Datalayer from CLI
Datalayer provides a Command Line Interface (CLI) allowing to list, create, terminate, open a console and run Notebooks and Python files on a Remote Kernel.
Installation
Install Datalayer with the following command in your terminal. You will need python>=3.9 and pip available on your machine.
pip install datalayer
If you have already installed datalayer, upgrade with the following command.
pip install datalayer --upgrade
Usage
Once installed you can invoke the datalayer command.
datalayer
Datalayer - Version 1.0.0 - Connected as None on https://oss.datalayer.run
[DatalayerCLI] One of `about` `benchmarks` `envs` `kernels` `login` `logout` `web` `who` `whoami` must be specified.
Ask details adding --help to any of the commands.
datalayer --help
The Datalayer CLI application.
Subcommands
===========
Subcommands are launched as `datalayer_core cmd [args]`. For information on
using subcommand 'cmd', do: `datalayer_core cmd -h`.
about
benchmarks
envs
kernels
login
logout
...
datalayer login --help
An application to log into a Remote Kernel provider.
datalayer login
Options
=======
The options below are convenience aliases to configurable class-options,
as listed in the "Equivalent to" description-line of the aliases.
To see all configurable class-options for some <cmd>, use:
<cmd> --help-all
--debug
set log level to logging.DEBUG (maximize logging output)
Equivalent to: [--Application.log_level=10]
...
An about command is available to explain you the features
datalayer about
Datalayer - Version 1.0.0 - Connected as None on https://oss.datalayer.run
About
Datalayer provides a command line tool allowing to list,
create, terminate and open a console against a Remote Kernel.
Read more on https://datalayer.io
You can use dla or even d as shortcut, instead of datalayer if you prefer.
All command lines support an option --run-url to set the URL of the Datalayer RUN cluster providing the Remote Kernel services.
datalayer kernels list --run-url https://oss.datalayer.run
Alternatively, you can set environment variables to avoid adding the params to every command.
export DATALAYER_RUN_URL="https://oss.datalayer.run"
export DATALAYER_TOKEN="MY_TOKEN"
export DATALAYER_EXTERNAL_TOKEN="MY_EXTERNAL_TOKEN"
datalayer whoami
datalayer envs list
datalayer kernels list
We also made it easy for you to get the values. Go to the action menu of the kernel and select the Copy terminal command line action. This will copy in your clipboard the command to run from CLI.
Authentication
Before login, you should have a valid account. You can join Datalayer via 3 ways:
- A Username and Password.
- A Token. This can be a Datalayer Token, or an external one in case of integration with an external provider.
- GitHub authentication.
Currently, the CLI login only supports the first two methods.
To login, you can use the following command:
datalayer login
By default, this command will open a new tab in your web browser to authenticate you. Once authenticated, you can close your browser.
If your cached credentials are obsolete, you will be prompted to log in when you run any of the command lines explained in the rest of this page.
If you have a Token, you can use it to authenticate via the Web page.
If you prefer not to use your web browser, you can add the --no-browser option to any command:
When using the --no-browser option, you will be prompted to either:
- Provide a username and password.
- Provide a token.
datalayer login --no-browser --run-url=...
? How do you want to log in? (Use arrow keys)
» Username/Password
Token
Once authenticated, you can ask details about your profile.
datalayer whoami
Profile
┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━┳━━━━━━━━┳━━━━━━━━┓
┃ ID ┃ H… ┃ First… ┃ Last … ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━╇━━━━━━━━╇━━━━━━━━┩
│ 01J2E2WQ28HS4DM0AXVEY… │ e… │ Eric │ Charl… │
└────────────────────────┴────┴────────┴────────┘
If you are using a token for authentification, you can also use the --token option to provide the token directly in any command.
datalayer envs list --token TOKEN --run-url RUN_URL
You can logout at any time.
datalayer logout
List the Environments
datalayer envs list
Environments
┏━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ ID ┃ Cost p… ┃ Name ┃ Description ┃ ┃ Resources ┃
┡━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ python-s… │ 0.01 │ Python Simple En… │ A Python environment for simp… │ │ {"cpu": "250m", "memory… │
└───────────┴─────────┴───────────────────┴────────────────────────────────┴──┴──────────────────────────┘
Create a Remote Kernel
Use datalayer kernels create ENV_ID --given-name GIVEN_NAME --credits-limit CREDITS_LIMIT.
datalayer kernels create python-simple-env --given-name k1 --credits-limit 3
Jupyter Kernels
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Kernel ID ┃ Kernel Name ┃ Environment ┃ Expired At ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ jupyter-01j2m0zt2kaheyy3yzn021g032 │ k1 │ python-simple-env │ 2024-08-10T17:29:19.355000+02:… │
└────────────────────────────────────┴─────────────┴───────────────────┴─────────────────────────────────┘
List the Remote Kernels
datalayer kernels list
Jupyter Kernels
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Kernel ID ┃ Kernel Name ┃ Environment ┃ Expired At ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ jupyter-01j2m0zt2kaheyy3yzn021g032 │ k1 │ python-simple-env │ 2024-08-10T17:29:19.355000+02:… │
│ jupyter-01j2m1kc0zgq3kjcne4p5avhd8 │ k2 │ python-simple-env │ 2024-07-27T06:52:44.704000+02:… │
└────────────────────────────────────┴─────────────┴───────────────────┴─────────────────────────────────┘
Terminate a Remote Kernel
Use datalayer kernels terminate KERNEL_ID.
datalayer kernels terminate jupyter-01j2ks4xzbr0vwe8zfzcv5ksm2
[KernelTerminateApp] Jupyter Kernels - Version 1.1.9 - Connected as eric on https://oss.datalayer.run
[KernelTerminateApp] Kernel 'jupyter-01j2ks4xzbr0vwe8zfzcv5ksm2' deleted.
Execute Code in a Console
Use datalayer kernels console --kernel KERNEL_ID
datalayer kernels console --kernel jupyter-01j2krcmskpye0sn0nv9a7afhp
[KernelConsoleApp] Jupyter Kernels - Version 1.1.9 - Connected as eric on https://oss.datalayer.run
[KernelConsoleApp] KernelManager using existing kernel server jupyter-01j2krbpy5nh72xcgc8r7jwjh7 expiring at 2024-08-10T17:16:58.139000+02:00
JupyterKernels console 1.1.9
Python 3.11.9 | packaged by conda-forge | (main, Apr 19 2024, 18:36:13) [GCC 12.3.0]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.22.2 -- An enhanced Interactive Python. Type '?' for help.
In [1]: print('👋 Hello from Kernel')
👋 Hello from Kernel
In [2]:
The Kernel ID is optional. If not provided, it will take the first Remote Kernel found.
To keep the Kernel alive when you exit the console, use CTRL-D or quit(keep_kernel=True).
Execute a File
Use datalayer kernels exec FILE_PATH with the following command options:
--kernel-id- Kernel ID of the Remote Kernel to use.--raise- Stop the execution one an exception occurred. By default, all cells of the notebook will be executed even if one cell fails.--timeout seconds--verbose- Display all cell outputs. By default, only cell outputs targeting the error stream will be displayed.
cat > hello.py <<EOF
print('👋 Hello from Kernel')
EOF
datalayer kernels exec hello.py --kernel jupyter-01j2krcmskpye0sn0nv9a7afhp
rm hello.py
Execute a Notebook
Use datalayer kernels exec NOTEBOOK_PATH with the following command options:
--kernel-id- Kernel ID of the Remote Kernel to use.--raise- Stop the execution one an exception occurred. By default, all cells of the notebook will be executed even if one cell fails.--timeout seconds--verbose- Display all cell outputs. By default, only cell outputs targeting the error stream will be displayed.
datalayer kernels exec hello.ipynb --kernel jupyter-01j2krcmskpye0sn0nv9a7afhp