Skip to main content

Layer CLI commands

This page describes all CLI commands available in the Layer SDK.

apply#

Displays the provisioning of Layer resources based on your current project declaration. In your terminal, navigate into a Layer project folder and then enter:

layer apply

Layer SDK loads the current project and displays useful information such as contents, log location, and links to the model and feature set locations in the Layer web console.

layer apply [[feature set|model] NAME]
OptionDescription
-d, --dependenciesDisplays the target entity, after building all of its dependencies.
-D, --no-dependenciesDisplays the target entity without building its dependencies.

Examples:

# Loads your whole Layer projectlayer apply
# Loads your churn model onlylayer apply model churn_model
# Loads the same model with it's dependencieslayer apply model churn_model -d

clone#

Clone from a remote git repository. The main difference from git clone is that this command excludes the .git/ metadata directory.

layer clone [OPTIONS] [REPO_URL]

Examples:

layer clone https://github.com/myrepo/mytemplates --local-path new-local-path
# If a URL is not passed, https://github.com/layerml/examples will be used.layer clone --local-path my-layer-examples
# If local-path is not passed, then it will be saved as the name of the# repository (in this case, `examples`).layer clone

diagnose#

Print out information that might help with debugging errors related to a Layer project. This includes the project folder structure, the names of environmental variables (not the values!), and installed packages.

When making bug reports or support queries, please include the output of this command.

In your terminal, navigate to the folder that contains the Layer project you need information about and then run:

layer diagnose

init#

Initialize a new, empty Layer project by creating a skeleton .layer/project.yml.

note

This command creates a hidden folder.

layer init [DIRECTORY]
ArgumentDescriptionValue TypeDefault
DIRECTORYThe relative path to initialize the project. It should be empty or non-existent.stringThe current directory.

integration#

Manage source integrations. This command allows you to list, update, and add integrations to Layer.

layer integration [OPTIONS] COMMAND [ARGS]

layer integration requires one of the following commands:

CommandDescription
add-big-queryAdd a BigQuery integration. Requires a name for the new integration.
add-snowflakeAdd a Snowflake integration. Requires a name for the new integration.
lsList all existing integrations.
updateUpdate an existing integration. Requires the name of the integration you want to update.

Examples:

# The name of this integration is newquerylayer integration add-big-query newquery
# List all integrations to double check the namelayer integration ls
# If I need to update newquerylayer integration update newquery

kill#

Terminates a pipeline run for the provided run ID. Get the ID with layer runs.

layer kill [RUN_ID]
ArgumentDescriptionValue TypeDefault
RUN_IDID of the pipeline you want to terminate.string

login#

Authenticate user by redirecting to a Layer login URL. If you do not include a URL in the command, then Layer accesses the default URL.

layer login [URL]
ArgumentDescriptionValue TypeDefault
URLURL of the Layer instancestringbeta.layer.co

login-headless#

Log in to Layer with the Layer CLI. If you do not include a URL in the command, then Layer accesses the default URL.

layer login-headless [URL]
ArgumentDescriptionValue TypeDefault
URLURL of the Layer instancestringbeta.layer.co

logout#

Log out of your Layer account.

layer logout

logs#

Shows the logs produced by running a pipeline. You can see the logs of your running and completed pipelines. The logs are particularly useful for debugging problems and monitoring your pipeline activity. For instructions, refer to Pipeline logs.

layer logs [OPTIONS] PIPELINE_RUN_ID

By default, logs produced by all jobs are returned. This includes both feature builds and model trains.

Logs can be saved to a file:

layer logs PIPELINE_RUN_ID > NAME_OF_YOUR_LOG_FILE
ArgumentDescriptionValue TypeDefault
PIPELINE_RUN_IDID of the pipeline you want to see the logs of. It is shown in the layer start output(look for Pipeline run id) or obtained via listing runs with layer runs.string
OptionDescriptionDefault
-f, --followContinue checking for new logs and streams them to the terminal or log file. If this is not included, then the command only provides logs that are already available. The --follow option will not allow further input in the terminal until the run is complete.false

Log contents#

Layer logs include metadata from Layer in each line, followed by output from Layer initialization logic and the user's code output for models and features. Here is an example:

[2021-10-19 19:42:23, FEATURESET_BUILD, ageband, python-runner-e0d267dc-654f-4b85-98cd-fe36be9c38fd-8tx6p]   feature_data['AGE'] = pd.DataFrame(age)[2021-10-19 19:42:24, FEATURESET_BUILD, embarked, python-runner-93210e1f-d101-41eb-81ed-39e13545e845-rt4tz] Built features stored successfully[2021-10-19 19:43:08, MODEL_TRAIN, titanic_survival_model_pred, model-training-cb423bd0-5bb8-4f6f-b82c-73ab0c1ec44b-bk8fw] Starting job.[2021-10-19 19:43:08, MODEL_TRAIN, titanic_survival_model_pred, model-training-cb423bd0-5bb8-4f6f-b82c-73ab0c1ec44b-bk8fw] Using selector: EpollSelector

Everything inside the brackets [] is Layer metadata. The table below describes the content of the log fields.

MetadataDescription
TimestampTime the event was logged.
Task typeType of task. Options include FEATURESET_BUILD, HYPERPARAMETER_TUNING_TRAIN,, and MODEL_TRAIN.
EntityEntity the task is being performed on. For example, a feature or a model.
ContainerThe container corresponds to the runtime allocated for a specific task execution.

Log ordering#

Layer stores logs in 10-second batches. The logs are ordered based on the time they were produced.

note

If you view logs from multiple jobs, then the batches might be displayed out of order. Logs of individual jobs will always be displayed in the correct order.

runs#

Lists your pipeline runs. For step-by-step instructions, refer to View pipeline runs.

layer runs [OPTIONS] [PIPELINE_RUN_ID]
ArgumentDescriptionValue TypeDefault
PIPELINE_RUN_IDID of the pipeline you want to watch. If skipped, all runs will be listed.string
OptionDescription
-w, --watchWatch the specified pipeline run.
-W, --no-watchReturn the runs once and exit. (Default)
-a, --allList all runs, including active and the completed ones.
-A, --activeList only active runs. (Default)

set-up-shell-completion#

Sets up shell completion for Layer CLI commands for your current OS platform and shell. Only bash and zsh shells are supported for now.

layer set-up-shell-completion

Examples:

# Set up completion for the current bash shelllayer set-up-shell-completion
# Restart shell so your .bashrc or .zshrc is sourced again
layer s[TAB][TAB]# Shell should autocomplete to layer startlayer start

start#

Asynchronously starts the execution of your whole Layer project or a single entity if you provide the entity type and name.

layer start [OPTIONS] [[feature set|model] NAME]
OptionDescription
-d, --dependenciesBuilds the target entity, after building all of its dependencies.
-D, --no-dependenciesBuilds the target entity without building its dependencies.

Examples:

# Builds your whole Layer projectlayer start
# Train your churn model onlylayer start model churn_model
# Train the same model after building all of its dependencies in the correct orderlayer start model churn_model -d