zephyr/scripts/west_commands
Martí Bolívar 3124c02987 cmake: flash/debug: refactor runner configuration
This commit message is a bit of a novel mostly:

- because the issues involved are longstanding
- as evidence this is not a capricious refactoring

The runners.core.RunnerConfig Python class holds common configuration
values used by multiple runners, such as the location of the build
outputs and board directory.

The runners code, first written in 2017-ish, replaced various shell
scripts that got this information from the environment. Avoiding
environment variables was a requirement, however. It's ghastly to set
environment variables for a single command invocation on Windows, and
the whole thing was part of a larger push to make Zephyr development
on Windows better.

I had a hammer (the argparse module). Finding a replacement naturally
looked like a nail, so the information that ends up in RunnerConfig
got shunted from the build system to Python in the form of 'west
flash' / 'west debug' command line options like '--board-dir',
'--elf-file', etc.

I initially stored the options and their values in the CMake cache.
This was chosen in hopes the build system maintainer would like
the strategy (which worked).

I knew the command line arguments approach was a bit hacky (this
wasn't a nail), but I also honestly didn't have a better idea at the
time.

It did indeed cause issues:

- users don't know that just because they specify --bin-file on the
  command line doesn't mean that their runner respects the option, and
  have gotten confused trying to flash alternate files, usually for
  chain-loading by MCUboot (for example, see #15961)

- common options weren't possible to pass via board.cmake files
  (#22563, fixed partly via introduction of runners.yaml and the west
  flash/debug commands no longer relying on the cache)

- it is confusing that "west flash --help" prints information about
  openocd related options even when the user's board has no openocd
  support. The same could be said about gdb in potential future use
  cases where debugging occurs via some other tool.

Over time, they've caused enough users enough problems that
improvements are a priority.

To work towards this, put these values into runners.yaml using a new
'config: ...' key/value instead of command line options.

For example, instead of this in the generated runners.yaml file:

args:
  common:
  - --hex-file=.../zephyr.hex

we now have:

config:
  hex_file: zephyr.hex

and similarly for other values.

In Python, we still support the command line options, but they are not
generated by the build system for any in-tree boards. Further work is
needed to deprecate the confusing ones (like --hex-file) and move the
runner-specific host tool related options (like --openocd) to the
runners that need them.

Individual board.cmake files should now influence these values by
overriding the relevant target properties of the
runners_yaml_props_target.

For example, instead of:

  board_runner_args(foo "--hex-file=bar.hex")

Do this:

  set_target_properties(runners_yaml_props_target PROPERTIES
                        hex_file bar.hex)

This change additionally allows us to stitch cmake/mcuboot.cmake and
the runners together easily by having mcuboot.cmake override the
properties that set the hex or bin file to flash. (The command line
arguments are still supported as-is.)

Combined with 98e0c95d91ae16f14e4997fb64ccdf0956595712 ("build:
auto-generate signed mcuboot binaries"), this will allow users to
build and flash images to be chain loaded by mcuboot in a way that
avoids calling 'west sign' and passing 'west flash' its output files
entirely.

While we are here, rename runner_yml_write to runners_yaml_append().
This function doesn't actually write anything, and we're here
refactoring this file anyway, so we might as well improve the
situation while we're at it.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-09-03 16:49:09 -05:00
..
completion
runners scripts: runners: switch RunnerConfig to NamedTuple 2020-09-03 16:49:09 -05:00
tests nrfjprog.py: Fix snr globbing to allow leading 0s 2020-08-28 11:38:05 +02:00
boards.py
build_helpers.py scripts: clean up west build/flash/debug help 2020-06-29 08:47:04 -04:00
build.py cmake: west: invoke west using same python as rest of build system 2020-07-08 23:23:04 -04:00
completion.py
debug.py scripts: clean up west build/flash/debug help 2020-06-29 08:47:04 -04:00
export.py cmake: Zephyr CMake package clean-up and minor fix 2020-08-19 12:52:33 -04:00
flash.py scripts: clean up west build/flash/debug help 2020-06-29 08:47:04 -04:00
mypy.ini scripts: runners: add type checking for west_commands 2020-09-03 16:49:09 -05:00
README.txt scripts: runners: add type checking for west_commands 2020-09-03 16:49:09 -05:00
run_common.py cmake: flash/debug: refactor runner configuration 2020-09-03 16:49:09 -05:00
run_tests.py scripts: runners: add type checking for west_commands 2020-09-03 16:49:09 -05:00
sign.py scripts: west sign: add --quiet option 2020-09-01 13:33:46 +02:00
zcmake.py cmake: west: invoke west using same python as rest of build system 2020-07-08 23:23:04 -04:00
zephyr_ext_common.py scripts: west_commands: remove unused helper 2020-08-19 09:51:13 -04:00

This directory contains implementations for west commands which are
tightly coupled to the zephyr tree. This includes the build, flash,
and debug commands.

Before adding more here, consider whether you might want to put new
extensions in upstream west. For example, any commands which operate
on the multi-repo need to be in upstream west, not here. Try to limit
what goes in here to Zephyr-specific features.

When extending this code, please keep the unit tests (in tests/) up to
date. The mypy static type checker is also run on the runners package.

To run these tests locally on Windows, run:

   py -3 run_tests.py

On macOS and Linux:

   ./run_tests.py

Note that these tests are run as part of Zephyr's CI when submitting
an upstream pull request, and pull requests which break the tests
cannot be merged.

Thanks!