Basics¶

This page introduces west’s basic concepts and built-in commands along with references to further reading.

Introduction¶

West’s built-in commands allow you to work with projects (Git repositories) under a common workspace directory. You can create a workspace using the west init command.

If you’ve followed the upstream Zephyr getting started guide, your workspace looks like this:

zephyrproject/                 # west topdir
├── .west/                     # marks the location of the topdir
│   └── config                 # per-workspace local configuration file
├── zephyr/                    # .git/ repo    │ the manifest repository,
│   ├── west.yml               # manifest file │ never modified by west
│   └── [... other files ...]                  │ after creation
├── modules/
│   └── lib/
│       └── tinycbor/          # .git/ project
├── net-tools/                 # .git/ project
└── [ ... other projects ...]

Above, zephyrproject is the name of the workspace’s top level directory, or topdir. (The name zephyrproject is just an example – it could be anything, like z, my-zephyr-workspace, etc.)

The topdir contains the .west directory. When west needs to find the topdir, it searches for .west, and uses its parent directory. The search starts from the current working directory (and starts again from the location in the ZEPHYR_BASE environment variable as a fallback if that fails). The file .west/config is the workspace’s local configuration file.

Every west workspace contains exactly one manifest repository, which is a Git repository containing a manifest file. The location of the manifest repository is given by the manifest.path configuration option in the local configuration file.

For upstream Zephyr, zephyr is the manifest repository, but you can configure west to use any Git repository in the workspace as the manifest repository. The only requirement is that it contains a valid manifest file. See Topologies supported for information on other options, and West Manifests for details on the manifest file format.

The manifest file is a YAML file that defines projects, which are the additional Git repositories in the workspace managed by west. The manifest file is named west.yml by default; this can be overridden using the manifest.file local configuration option.

You use the west update command to update the workspace’s projects based on the contents of the manifest file. The manifest file controls things like where the project should be placed within the workspace, what URL to clone it from if it’s missing, and what Git revision should be checked out locally.

Projects can be located anywhere inside the workspace, but they may not “escape” it. Project repositories need not be located in subdirectories of the manifest repository or as immediate subdirectories of the topdir. However, projects must have paths inside the workspace. (You may replace a project’s repository directory within the workspace with a symbolic link to elsewhere on your computer, but west will not do this for you.)

A workspace can contain additional Git repositories or other files and directories not managed by west. West basically ignores anything in the workspace except .west, the manifest repository, and the projects specified in the manifest file.

For upstream Zephyr, tinycbor and net-tools are projects. They are specified in the manifest file zephyr/west.yml. This file specifies that tinycbor is located in the modules/lib/tinycbor directory beneath the workspace topdir. By default, the Zephyr build system uses west to get the locations of all the projects in the workspace, so any code they contain can be used as Modules (External projects).

Finally, any repository managed by a west workspace (either the manifest repository or any project repository) can define Extensions. Extensions are extra commands not built into west that you can run when using that workspace.

The zephyr repository uses this feature to provide Zephyr-specific commands like west build. Defining these as extensions keeps west’s core agnostic to the specifics of any workspace’s Zephyr version, etc.

Structure¶

West’s code is distributed via PyPI in a Python package named west. This distribution includes a launcher executable, which is also named west (or west.exe on Windows).

When west is installed, the launcher is placed by pip3 somewhere in the user’s filesystem (exactly where depends on the operating system, but should be on the PATH environment variable). This launcher is the command-line entry point to running both built-in commmands like west init, west update, along with any extensions discovered in the workspace.

In addition to its command-line interface, you can also use west’s Python APIs directly. See West APIs for details.

The `manifest-rev` branch¶

West creates and controls a Git branch named manifest-rev in each project. This branch points to the revision that the manifest file specified for the project at the time west update was last run. Other workspace management commands may use manifest-rev as a reference point for the upstream revision as of this latest update. Among other purposes, the manifest-rev branch allows the manifest file to use SHAs as project revisions.

Although manifest-rev is a normal Git branch, west will recreate and/or reset it on the next update. For this reason, it is dangerous to check it out or otherwise modify it yourself. For instance, any commits you manually add to this branch may be lost the next time you run west update. Instead, check out a local branch with another name, and either rebase it on top of a new manifest-rev, or merge manifest-rev into it.

Note

West does not create a manifest-rev branch in the manifest repository, since west does not manage the manifest repository’s branches or revisions.

Built-in Commands¶

This section gives an overview of west’s built-in commands.

Some commands are related to Git commands with the same name, but operate on the entire workspace. For example, west diff shows local changes in multiple Git repositories in the workspace.

Some commands take projects as arguments. These arguments can be project names as specified in the manifest file, or (as a fallback) paths to them on the local file system. Omitting project arguments to commands which accept them (such as west list, west forall, etc.) usually defaults to using all projects in the manifest file plus the manifest repository itself.

The following documentation does not exhaustively describe all commands. For additional help, run west <command> -h (e.g. west init -h).

`west init`¶

This command creates a west workspace. It can be used in two ways:

Cloning a new manifest repository from a remote URL
Creating a workspace around an existing local manifest repository

Option 1: to clone a new manifest repository from a remote URL, use:

west init [-m URL] [--mr REVISION] [--mf FILE] [directory]

The new workspace is created in the given directory, creating a new .west inside this directory. You can give the manifest URL using the -m switch, the initial revision to check out using --mr, and the location of the manifest file within the repository using --mf.

For example, running:

west init -m https://github.com/zephyrproject-rtos/zephyr --mr v1.14.0 zp

would clone the upstream official zephyr repository into zp/zephyr, and check out the v1.14.0 release. This command creates zp/.west, and set the manifest.path configuration option to zephyr to record the location of the manifest repository in the workspace. The default manifest file location is used.

The -m option defaults to https://github.com/zephyrproject-rtos/zephyr. The --mr option defaults to master. The --mf option defaults to west.yml. If no directory is given, the current working directory is used.

Option 2: to create a workspace around an existing local manifest repository, use:

west init -l [--mf FILE] directory

This creates .west next to directory in the file system, and sets manifest.path to directory.

As above, --mf defaults to west.yml.

The west init command does not clone any of the projects defined in the manifest file, regardless of whether -l is given. To do that, use west update.

`west update`¶

This command clones and updates the projects specified in the west manifest file.

west update [-h] [--stats] [-f {always,smart}] [-k] [-r] [PROJECT ...]

By default, this command:

Parses the manifest file, usually west.yml
Clones any project repositories that are not already present locally
Fetches any project revisions which are not already pulled from the remote
Sets each project’s manifest-rev branch to the revision specified for that project in the manifest file
Checks out each manifest-rev in local working trees, as detached HEADs

To operate on a subset of projects only, specify them using the PROJECT positional arguments, which can be either project names as given in the manifest file, or paths to the local project clones.

To force this command to fetch from project remotes even if the revisions appear to be available locally, either use --fetch always or set the update.fetch configuration option to "always".

For safety, west update uses git checkout --detach to check out a detached HEAD at the manifest revision for each updated project, leaving behind any branches which were already checked out. This is typically a safe operation that will not modify any of your local branches.

If you would rather rebase any locally checked out branches instead, use the -r (--rebase) option.

If you would like west update to keep local branches checked out as long as they point to commits that are descendants of the new manifest-rev, use the -k (--keep-descendants) option.

Note

west update --rebase will fail in projects that have git conflicts between your branch and new commits brought in by the manifest. You should immediately resolve these conflicts as you usually do with git, or you can use git -C <project_path> rebase --abort to ignore incoming changes for the moment.

With a clean working tree, a plain west update never fails because it does not try to hold on to your commits and simply leaves them aside.

west update --keep-descendants offers an intermediate option that never fails either but does not treat all projects the same:

in projects where your branch diverged from the incoming commits, it does not even try to rebase and leaves your branches behind just like a plain west update does;
in all other projects where no rebase or merge is needed it keeps your branches in place.

Other Repository Management Commands¶

West has a few more commands for managing the repositories in the workspace, which are summarized here. Run west <command> -h for detailed help.

west list: print a line of information about each project in the manifest, according to a format string
west manifest: manage the manifest file. See Manifest Command.
west diff: run git diff in local project repositories
west status: run git status in local project repositories
west forall: run an arbitrary command in local project repositories

Additional Commands¶

Finally, here is a summary of other built-in commands.

west config: get or set configuration options
west topdir: print the top level directory of the west workspace
west help: get help about a command, or print information about all commands in the workspace, including Extensions

Topologies supported¶

The following are example source code topologies supported by west.

T1: star topology, zephyr is the manifest repository
T2: star topology, a Zephyr application is the manifest repository
T3: forest topology, freestanding manifest repository

T1: Star topology, zephyr is the manifest repository¶

The zephyr repository acts as the central repository and specifies its Modules (External projects) in its west.yml
Analogy with existing mechanisms: Git submodules with zephyr as the super-project

This is the default. See Introduction for how mainline Zephyr is an example of this topology.

T2: Star topology, application is the manifest repository¶

Useful for those focused on a single application
A repository containing a Zephyr application acts as the central repository and names other projects required to build it in its west.yml. This includes the zephyr repository and any modules.
Analogy with existing mechanisms: Git submodules with the application as the super-project, zephyr and other projects as submodules

A workspace using this topology looks like this:

west-workspace/
│
├── application/         # .git/     │
│   ├── CMakeLists.txt               │
│   ├── prj.conf                     │  never modified by west
│   ├── src/                         │
│   │   └── main.c                   │
│   └── west.yml         # main manifest with optional import(s) and override(s)
│                                    │
├── modules/
│   └── lib/
│       └── tinycbor/    # .git/ project from either the main manifest or some import.
│
└── zephyr/              # .git/ project
    └── west.yml         # This can be partially imported with lower precedence or ignored.
                         # Only the 'manifest-rev' version can be imported.

Here is an example application/west.yml which uses Manifest Imports, available since west 0.7, to import Zephyr v2.2.0 and its modules into the application manifest file:

# Example T2 west.yml, using manifest imports.
manifest:
  remotes:
    - name: zephyrproject-rtos
      url-base: https://github.com/zephyrproject-rtos
  projects:
    - name: zephyr
      remote: zephyrproject-rtos
      revision: v2.2.0
      import: true
  self:
    path: application

You can still selectively “override” individual Zephyr modules if you use import: in this way; see Example 1.3: Downstream of a Zephyr release, with module fork for an example.

Another way to do the same thing is to copy/paste zephyr/west.yml to application/west.yml, adding an entry for the zephyr project itself, like this:

# Equivalent to the above, but with manually maintained Zephyr modules.
manifest:
  remotes:
    - name: zephyrproject-rtos
      url-base: https://github.com/zephyrproject-rtos
  defaults:
    remote: zephyrproject-rtos
  projects:
    - name: zephyr
      revision: v2.2.0
      west-commands: scripts/west-commands.yml
    - name: net-tools
      revision: some-sha-goes-here
      path: tools/net-tools
    # ... other Zephyr modules go here ...
  self:
    path: application

(The west-commands is there for Building, Flashing and Debugging and other Zephyr-specific Extensions. It’s not necessary when using import.)

The main advantage to using import is not having to track the revisions of imported projects separately. In the above example, using import means Zephyr’s module versions are automatically determined from the zephyr/west.yml revision, instead of having to be copy/pasted (and maintained) on their own.

T3: Forest topology¶

Useful for those supporting multiple independent applications or downstream distributions with no “central” repository
A dedicated manifest repository which contains no Zephyr source code, and specifies a list of projects all at the same “level”
Analogy with existing mechanisms: Google repo-based source distribution

A workspace using this topology looks like this:

west-workspace/
├── app1/               # .git/ project
│   ├── CMakeLists.txt
│   ├── prj.conf
│   └── src/
│       └── main.c
├── app2/               # .git/ project
│   ├── CMakeLists.txt
│   ├── prj.conf
│   └── src/
│       └── main.c
├── manifest-repo/      # .git/ never modified by west
│   └── west.yml        # main manifest with optional import(s) and override(s)
├── modules/
│   └── lib/
│       └── tinycbor/   # .git/ project from either the main manifest or
│                       #       frome some import
│
└── zephyr/             # .git/ project
    └── west.yml        # This can be partially imported with lower precedence or ignored.
                        # Only the 'manifest-rev' version can be imported.

Here is an example T3 manifest-repo/west.yml which uses Manifest Imports, available since west 0.7, to import Zephyr v2.2.0 and its modules, then add the app1 and app2 projects:

manifest:
  remotes:
    - name: zephyrproject-rtos
      url-base: https://github.com/zephyrproject-rtos
    - name: your-git-server
      url-base: https://git.example.com/your-company
  defaults:
    remote: your-git-server
  projects:
    - name: zephyr
      remote: zephyrproject-rtos
      revision: v2.2.0
      import: true
    - name: app1
      revision: SOME_SHA_OR_BRANCH_OR_TAG
    - name: app2
      revision: ANOTHER_SHA_OR_BRANCH_OR_TAG
  self:
    path: manifest-repo

You can also do this “by hand” by copy/pasting zephyr/west.yml as shown above for the T2 topology, with the same caveats.

Private repositories¶

You can use west to fetch from private repositories. There is nothing west-specific about this.

The west update command essentially runs git fetch YOUR_PROJECT_URL when a project’s manifest-rev branch must be updated to a newly fetched commit. It’s up to your environment to make sure the fetch succeeds.

You can either enter the password manually or use any of the credential helpers built in to Git. Since Git has credential storage built in, there is no need for a west-specific feature.

The following sections cover common cases for running west update without having to enter your password, as well as how to troubleshoot issues.

Fetching via HTTPS¶

On Windows when fetching from GitHub, recent versions of Git prompt you for your GitHub password in a graphical window once, then store it for future use (in a default installation). Passwordless fetching from GitHub should therefore work “out of the box” on Windows after you have done it once.

In general, you can store your credentials on disk using the “store” git credential helper. See the git-credential-store manual page for details.

To use this helper for all the repositories in your workspace, run:

west forall -c "git config credential.helper store"

To use this helper on just the projects foo and bar, run:

west forall -c "git config credential.helper store" foo bar

To use this helper by default on your computer, run:

git config --global credential.helper store

On GitHub, you can set up a personal access token to use in place of your account password. (This may be required if your account has two-factor authentication enabled, and may be preferable to storing your account password in plain text even if two-factor authentication is disabed.)

If you don’t want to store any credentials on the file system, you can store them in memory temporarily using git-credential-cache instead.

Fetching via SSH¶

If your SSH key has no password, fetching should just work. If it does have a password, you can avoid entering it manually every time using ssh-agent.

On GitHub, see Connecting to GitHub with SSH for details on configuration and key creation.

Troubleshooting¶

One good way to troubleshoot fetching issues is to run west update in verbose mode, like this:

west -v update

The output includes Git commands run by west and their outputs. Look for something like this:

=== updating your_project (path/to/your/project):
west.manifest: your_project: checking if cloned
[...other west.manifest logs...]
--- your_project: fetching, need revision SOME_SHA
west.manifest: running 'git fetch ... https://github.com/your-username/your_project ...' in /some/directory

The git fetch command example in the last line above is what needs to succeed. Go to /some/directory, copy/paste and run the entire git fetch command, then debug from there using the documentation for your credential storage helper.

If you can get the git fetch command to run successfully without prompting for a password when you run it directly, you will be able to run west update without entering your password in that same shell.