Go to file
Martí Bolívar 3124c02987 cmake: flash/debug: refactor runner configuration
This commit message is a bit of a novel mostly:

- because the issues involved are longstanding
- as evidence this is not a capricious refactoring

The runners.core.RunnerConfig Python class holds common configuration
values used by multiple runners, such as the location of the build
outputs and board directory.

The runners code, first written in 2017-ish, replaced various shell
scripts that got this information from the environment. Avoiding
environment variables was a requirement, however. It's ghastly to set
environment variables for a single command invocation on Windows, and
the whole thing was part of a larger push to make Zephyr development
on Windows better.

I had a hammer (the argparse module). Finding a replacement naturally
looked like a nail, so the information that ends up in RunnerConfig
got shunted from the build system to Python in the form of 'west
flash' / 'west debug' command line options like '--board-dir',
'--elf-file', etc.

I initially stored the options and their values in the CMake cache.
This was chosen in hopes the build system maintainer would like
the strategy (which worked).

I knew the command line arguments approach was a bit hacky (this
wasn't a nail), but I also honestly didn't have a better idea at the
time.

It did indeed cause issues:

- users don't know that just because they specify --bin-file on the
  command line doesn't mean that their runner respects the option, and
  have gotten confused trying to flash alternate files, usually for
  chain-loading by MCUboot (for example, see #15961)

- common options weren't possible to pass via board.cmake files
  (#22563, fixed partly via introduction of runners.yaml and the west
  flash/debug commands no longer relying on the cache)

- it is confusing that "west flash --help" prints information about
  openocd related options even when the user's board has no openocd
  support. The same could be said about gdb in potential future use
  cases where debugging occurs via some other tool.

Over time, they've caused enough users enough problems that
improvements are a priority.

To work towards this, put these values into runners.yaml using a new
'config: ...' key/value instead of command line options.

For example, instead of this in the generated runners.yaml file:

args:
  common:
  - --hex-file=.../zephyr.hex

we now have:

config:
  hex_file: zephyr.hex

and similarly for other values.

In Python, we still support the command line options, but they are not
generated by the build system for any in-tree boards. Further work is
needed to deprecate the confusing ones (like --hex-file) and move the
runner-specific host tool related options (like --openocd) to the
runners that need them.

Individual board.cmake files should now influence these values by
overriding the relevant target properties of the
runners_yaml_props_target.

For example, instead of:

  board_runner_args(foo "--hex-file=bar.hex")

Do this:

  set_target_properties(runners_yaml_props_target PROPERTIES
                        hex_file bar.hex)

This change additionally allows us to stitch cmake/mcuboot.cmake and
the runners together easily by having mcuboot.cmake override the
properties that set the hex or bin file to flash. (The command line
arguments are still supported as-is.)

Combined with 98e0c95d91ae16f14e4997fb64ccdf0956595712 ("build:
auto-generate signed mcuboot binaries"), this will allow users to
build and flash images to be chain loaded by mcuboot in a way that
avoids calling 'west sign' and passing 'west flash' its output files
entirely.

While we are here, rename runner_yml_write to runners_yaml_append().
This function doesn't actually write anything, and we're here
refactoring this file anyway, so we might as well improve the
situation while we're at it.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-09-03 16:49:09 -05:00
.buildkite ci: retry failed west update 2020-08-31 09:42:51 -04:00
.github scripts: runners: add type checking for west_commands 2020-09-03 16:49:09 -05:00
.known-issues doc: update known issues for Sphinx>=3.2.0 2020-08-31 09:57:33 -04:00
arch tracing: swap: bug fix and enhancement for ARC 2020-09-03 21:54:15 +02:00
boards boards: shields: add support for buydisplay 2.8" TFT touch display 2020-09-03 16:36:50 -05:00
cmake cmake: flash/debug: refactor runner configuration 2020-09-03 16:49:09 -05:00
doc Bluetooth: Mesh: Fix Health model shell doc 2020-09-03 21:50:41 +02:00
drivers i2s: cavs: cast device struct to void ptr for DMA user data 2020-09-03 17:33:11 -04:00
dts drivers: serial: uart_rtt converted to devicetree 2020-09-03 21:51:28 +02:00
include mgmt: hawkbit: Add Hawkbit FOTA Support 2020-09-03 22:13:52 +02:00
kernel tracing: swap: bug fix and enhancement for ARC 2020-09-03 21:54:15 +02:00
lib lib: os: prf: correct use of istype functions 2020-09-03 21:53:09 +02:00
misc
modules modules/Kconfig.nordic: Enable NRFX_USBREG together with NRFX_POWER 2020-09-03 21:52:52 +02:00
samples samples: display: lvgl: add information for buydisplay shield 2020-09-03 16:36:50 -05:00
scripts cmake: flash/debug: refactor runner configuration 2020-09-03 16:49:09 -05:00
share cmake: Zephyr CMake package clean-up and minor fix 2020-08-19 12:52:33 -04:00
soc drivers: clk_ctrl: npcx: fixed 'line length exceeds 80 columns' warning. 2020-09-03 21:53:35 +02:00
subsys mgmt: hawkbit: Add Hawkbit FOTA Support 2020-09-03 22:13:52 +02:00
tests tests: spinlock: fix occasional crash 2020-09-03 21:55:05 +02:00
.checkpatch.conf checkpatch: disable REPEATED_WORD 2020-08-21 13:57:02 -04:00
.clang-format
.codecov.yml
.editorconfig editorconfig: indent Kconfig files with tabs 2020-07-08 21:30:38 -04:00
.gitattributes
.gitignore
.gitlint
.mailmap
.uncrustify.cfg
CMakeLists.txt build: auto-generate signed mcuboot binaries 2020-09-01 13:33:46 +02:00
CODE_OF_CONDUCT.md
CODEOWNERS mgmt: hawkbit: Add Hawkbit FOTA Support 2020-09-03 22:13:52 +02:00
CONTRIBUTING.rst
Kconfig kconfig: Clean up header comments and make them consistent 2019-11-04 17:31:27 -05:00
Kconfig.zephyr build: auto-generate signed mcuboot binaries 2020-09-01 13:33:46 +02:00
LICENSE
MAINTAINERS.yml MAINTAINERS: remove bogus path for i2c 2020-08-13 17:03:47 +02:00
Makefile
README.rst doc: Replace shippable badge with buildkite badge 2020-07-14 07:21:44 -05:00
VERSION release: Update patch level to 99, post 2.3.0 release 2020-06-06 00:44:36 +02:00
version.h.in cmake: explain BUILD_VERSION and KERNEL_VERSION_* 2019-05-09 11:52:05 +02:00
west.yml west.yml: Snapshot update of mcumgr from upstream 2020-09-02 15:44:18 -05:00
zephyr-env.cmd scripts: remove west from scripts/ 2019-01-29 10:15:01 +01:00
zephyr-env.sh

.. raw:: html

   <a href="https://www.zephyrproject.org">
     <p align="center">
       <img src="doc/images/Zephyr-Project.png">
     </p>
   </a>

   <a href="https://bestpractices.coreinfrastructure.org/projects/74"><img
   src="https://bestpractices.coreinfrastructure.org/projects/74/badge"></a>
   <a href="https://buildkite.com/zephyr/zephyr">
   <img
   src="https://badge.buildkite.com/f5bd0dc88306cee17c9b38e78d11bb74a6291e3f40e7d13f31.svg?branch=master"></a>


The Zephyr Project is a scalable real-time operating system (RTOS) supporting
multiple hardware architectures, optimized for resource constrained devices,
and built with security in mind.

The Zephyr OS is based on a small-footprint kernel designed for use on
resource-constrained systems: from simple embedded environmental sensors and
LED wearables to sophisticated smart watches and IoT wireless gateways.

The Zephyr kernel supports multiple architectures, including ARM Cortex-M,
Intel x86, ARC, Nios II, Tensilica Xtensa, and RISC-V, and a large number of
`supported boards`_.

.. below included in doc/introduction/introduction.rst

.. start_include_here

Getting Started
***************

Welcome to Zephyr! See the `Introduction to Zephyr`_ for a high-level overview,
and the documentation's `Getting Started Guide`_ to start developing.

Community Support
*****************

Community support is provided via mailing lists and Slack; see the Resources
below for details.

.. _project-resources:

Resources
*********

Here's a quick summary of resources to help you find your way around:

* **Help**: `Asking for Help Tips`_
* **Documentation**: http://docs.zephyrproject.org (`Getting Started Guide`_)
* **Source Code**: https://github.com/zephyrproject-rtos/zephyr is the main
  repository; https://elixir.bootlin.com/zephyr/latest/source contains a
  searchable index
* **Releases**: https://github.com/zephyrproject-rtos/zephyr/releases
* **Samples and example code**: see `Sample and Demo Code Examples`_
* **Mailing Lists**: users@lists.zephyrproject.org and
  devel@lists.zephyrproject.org are the main user and developer mailing lists,
  respectively. You can join the developer's list and search its archives at
  `Zephyr Development mailing list`_. The other `Zephyr mailing list
  subgroups`_ have their own archives and sign-up pages.
* **Nightly CI Build Status**: https://lists.zephyrproject.org/g/builds
  The builds@lists.zephyrproject.org mailing list archives the CI
  (shippable) nightly build results.
* **Chat**: Zephyr's Slack workspace is https://zephyrproject.slack.com.  Use
  this `Slack Invite`_ to register.
* **Contributing**: see the `Contribution Guide`_
* **Wiki**: `Zephyr GitHub wiki`_
* **Issues**: https://github.com/zephyrproject-rtos/zephyr/issues
* **Security Issues**: Email vulnerabilities@zephyrproject.org to report
  security issues; also see our `Security`_ documentation. Security issues are
  tracked separately at https://zephyrprojectsec.atlassian.net.
* **Zephyr Project Website**: https://zephyrproject.org

.. _Slack Invite: https://tinyurl.com/y5glwylp
.. _supported boards: http://docs.zephyrproject.org/latest/boards/index.html
.. _Zephyr Documentation: http://docs.zephyrproject.org
.. _Introduction to Zephyr: http://docs.zephyrproject.org/latest/introduction/index.html
.. _Getting Started Guide: http://docs.zephyrproject.org/latest/getting_started/index.html
.. _Contribution Guide: http://docs.zephyrproject.org/latest/contribute/index.html
.. _Zephyr GitHub wiki: https://github.com/zephyrproject-rtos/zephyr/wiki
.. _Zephyr Development mailing list: https://lists.zephyrproject.org/g/devel
.. _Zephyr mailing list subgroups: https://lists.zephyrproject.org/g/main/subgroups
.. _Sample and Demo Code Examples: http://docs.zephyrproject.org/latest/samples/index.html
.. _Security: http://docs.zephyrproject.org/latest/security/index.html
.. _Asking for Help Tips: https://docs.zephyrproject.org/latest/guides/getting-help.html