This page describes all CLI commands available in the Layer SDK.
Displays the provisioning of Layer resources based on your current project declaration. In your terminal, navigate into a Layer project folder and then enter:
Layer SDK loads the current project and displays useful information such as contents, log location, and links to the model and feature set locations in the Layer web console.
layer apply [[feature set|model] NAME]
|-d, --dependencies||Displays the target entity, after building all of its dependencies.|
|-D, --no-dependencies||Displays the target entity without building its dependencies.|
# Loads your whole Layer projectlayer apply # Loads your churn model onlylayer apply model churn_model # Loads the same model with it's dependencieslayer apply model churn_model -d
Clone from a remote git repository. The main difference from
git clone is that this command excludes the
.git/ metadata directory.
layer clone [OPTIONS] [REPO_URL]
layer clone https://github.com/myrepo/mytemplates --local-path new-local-path # If a URL is not passed, https://github.com/layerml/examples will be used.layer clone --local-path my-layer-examples # If local-path is not passed, then it will be saved as the name of the# repository (in this case, `examples`).layer clone
Print out information that might help with debugging errors related to a Layer project. This includes the project folder structure, the names of environmental variables (not the values!), and installed packages.
When making bug reports or support queries, please include the output of this command.
In your terminal, navigate to the folder that contains the Layer project you need information about and then run:
Initialize a new, empty Layer project by creating a skeleton
This command creates a hidden folder.
layer init [DIRECTORY]
|DIRECTORY||The relative path to initialize the project. It should be empty or non-existent.||string||The current directory.|
Manage source integrations. This command allows you to list, update, and add integrations to Layer.
layer integration [OPTIONS] COMMAND [ARGS]
layer integration requires one of the following commands:
|add-big-query||Add a BigQuery integration. Requires a name for the new integration.|
|add-snowflake||Add a Snowflake integration. Requires a name for the new integration.|
|ls||List all existing integrations.|
|update||Update an existing integration. Requires the name of the integration you want to update.|
# The name of this integration is newquerylayer integration add-big-query newquery # List all integrations to double check the namelayer integration ls # If I need to update newquerylayer integration update newquery
Terminates a pipeline run for the provided run ID. Get the ID with
layer kill [RUN_ID]
|RUN_ID||ID of the pipeline you want to terminate.||string|
Authenticate user by redirecting to a Layer login URL. If you do not include a URL in the command, then Layer accesses the default URL.
layer login [URL]
|URL||URL of the Layer instance||string||beta.layer.co|
Log in to Layer with the Layer CLI. If you do not include a URL in the command, then Layer accesses the default URL.
layer login-headless [URL]
|URL||URL of the Layer instance||string||beta.layer.co|
Log out of your Layer account.
Shows the logs produced by running a pipeline. You can see the logs of your running and completed pipelines. The logs are particularly useful for debugging problems and monitoring your pipeline activity. For instructions, refer to Pipeline logs.
layer logs [OPTIONS] PIPELINE_RUN_ID
By default, logs produced by all jobs are returned. This includes both feature builds and model trains.
Logs can be saved to a file:
layer logs PIPELINE_RUN_ID > NAME_OF_YOUR_LOG_FILE
|PIPELINE_RUN_ID||ID of the pipeline you want to see the logs of. It is shown in the ||string|
|-f, --follow||Continue checking for new logs and streams them to the terminal or log file. If this is not included, then the command only provides logs that are already available. The ||false|
Layer logs include metadata from Layer in each line, followed by output from Layer initialization logic and the user's code output for models and features. Here is an example:
[2021-10-19 19:42:23, FEATURESET_BUILD, ageband, python-runner-e0d267dc-654f-4b85-98cd-fe36be9c38fd-8tx6p] feature_data['AGE'] = pd.DataFrame(age)[2021-10-19 19:42:24, FEATURESET_BUILD, embarked, python-runner-93210e1f-d101-41eb-81ed-39e13545e845-rt4tz] Built features stored successfully[2021-10-19 19:43:08, MODEL_TRAIN, titanic_survival_model_pred, model-training-cb423bd0-5bb8-4f6f-b82c-73ab0c1ec44b-bk8fw] Starting job.[2021-10-19 19:43:08, MODEL_TRAIN, titanic_survival_model_pred, model-training-cb423bd0-5bb8-4f6f-b82c-73ab0c1ec44b-bk8fw] Using selector: EpollSelector
Everything inside the brackets
 is Layer metadata. The table below describes the content of the log fields.
|Timestamp||Time the event was logged.|
|Task type||Type of task. Options include FEATURESET_BUILD, HYPERPARAMETER_TUNING_TRAIN,, and MODEL_TRAIN.|
|Entity||Entity the task is being performed on. For example, a feature or a model.|
|Container||The container corresponds to the runtime allocated for a specific task execution.|
Layer stores logs in 10-second batches. The logs are ordered based on the time they were produced.
If you view logs from multiple jobs, then the batches might be displayed out of order. Logs of individual jobs will always be displayed in the correct order.
Lists your pipeline runs. For step-by-step instructions, refer to View pipeline runs.
layer runs [OPTIONS] [PIPELINE_RUN_ID]
|PIPELINE_RUN_ID||ID of the pipeline you want to watch. If skipped, all runs will be listed.||string|
|-w, --watch||Watch the specified pipeline run.|
|-W, --no-watch||Return the runs once and exit. (Default)|
|-a, --all||List all runs, including active and the completed ones.|
|-A, --active||List only active runs. (Default)|
Sets up shell completion for Layer CLI commands for your current OS platform and shell. Only
zsh shells are supported for now.
# Set up completion for the current bash shelllayer set-up-shell-completion # Restart shell so your .bashrc or .zshrc is sourced again layer s[TAB][TAB]# Shell should autocomplete to layer startlayer start
Asynchronously starts the execution of your whole Layer project or a single entity if you provide the entity type and name.
layer start [OPTIONS] [[feature set|model] NAME]
|-d, --dependencies||Builds the target entity, after building all of its dependencies.|
|-D, --no-dependencies||Builds the target entity without building its dependencies.|
# Builds your whole Layer projectlayer start # Train your churn model onlylayer start model churn_model # Train the same model after building all of its dependencies in the correct orderlayer start model churn_model -d