Commit Graph

2784 Commits

Author SHA1 Message Date
Eugeniy Paltsev
0699f38f16 ARC: west: mdb runner: record MDB's CLD process pid to file
Record MDB's CLD process pid to file so this process can be
terminated by sanitycheck infrastructure.
Update mdb runners test to be able to handle changes.

Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
2020-11-11 08:18:38 -05:00
Eugeniy Paltsev
91d7ec5a35 ARC: west: split mdb runner for mdb-hw & mdb-nsim runners
mdb runner is quite special as it can be used to run Zephyr on
both simulator (nSIM) and real hardware.
However it is really misleading as same command (west flash)
will run Zephyr in simulation for one board and try to run it
on HW for another board. Things are getting worse for boards
supporting both runs in simulation and on real hardware.

Let's split mdb runner for mdb-hw (for runs on HW) and mdb-nsim
(for runs in simulation) runners.

Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
2020-11-11 08:18:38 -05:00
Jukka Rissanen
2f4b66967d net: scripts: Close zephyr network interface when test closes
The zeth interface was left hanging around when the test
finished but we can remove it here as it is not needed.

Signed-off-by: Jukka Rissanen <jukka.rissanen@linux.intel.com>
2020-11-09 14:42:34 +02:00
Jukka Rissanen
b68f7dfac2 net: scripts: Fix indentation and help usage
Fixing the indentation of run-sample-tests.sh script so that it
is easier to read.

Signed-off-by: Jukka Rissanen <jukka.rissanen@linux.intel.com>
2020-11-09 14:42:34 +02:00
Gerson Fernando Budke
3fe4c702e7 runners: bossac: Add speed argument
The current stty command uses a hard code value of 1200.  This is not
compliant with SAM-BA specs and may create compatibility problems.  Add
an optional speed argument with 115200 as default value following SAM-BA
specifications.  All boards that needs a different speed should define
board_runner_args(bossac "--speed=<value>") with value as required
speed.

Signed-off-by: Gerson Fernando Budke <nandojve@gmail.com>
2020-11-06 12:15:55 -06:00
Gerard Marull-Paretas
e066a00d5e runners: stm32cubeprogrammer: add tests
Add tests for the stm32cubeprogrammer runner.

Signed-off-by: Gerard Marull-Paretas <gerard@teslabs.com>
2020-11-04 14:47:35 -06:00
Gerard Marull-Paretas
f98dd24993 runners: add support for stm32cubeprogrammer
Add support for the official ST Microelectronics programming tool (CLI
version).

Signed-off-by: Gerard Marull-Paretas <gerard@teslabs.com>
2020-11-04 14:47:35 -06:00
Gerson Fernando Budke
0d216924d5 samples: net: cloud: Introduce TagoIO IoT cloud http post
Introduce TagoIO IoT Cloud HTTP post client example.  This explorer
Zephyr network resources to demonstrate an end to end application.
The TagoIO allows that any user can test on a easy way Ethernet, WIFI
and Modem (PPP) with BSD sockets.  The example provides overlays to
configure WIFI and Modem.

The application consists an a pseudo temperature sensor that sends
periodically data to TagoIO IoT Cloud platform.  The data can be
visualized on a web browser dashboard, cellphone or tablet.  The
steps to configure TagoIO are described on the example documentation.

Special Variables:
 - CONFIG_TAGOIO_DEVICE_TOKEN   DEVID  token generated by TagoIO
 - CONFIG_TAGOIO_HTTP_WIFI_SSID SSID   when using WIFI
 - CONFIG_TAGOIO_HTTP_WIFI_PSK  PASSWD when using WIFI
 - CONFIG_MODEM_GSM_UART_NAME   UART   label when using MODEM
 - CONFIG_MODEM_GSM_APN         APN    when using MODEM

Signed-off-by: Gerson Fernando Budke <nandojve@gmail.com>
2020-11-04 09:06:23 +02:00
Martí Bolívar
0d4dca10b2 scripts: edtlib: child binding compatibles match parents
Fixes: #29758

Commit 7165b77a81 ("scripts: edtlib:
refactor for first class bindings") introduced a Binding class.

Its child_binding instance attribute has a compatible which can be
None. Adjust this behavior so the child Binding object has the same
compatible as the parent binding which ultimately has a compatible.

Without this, sanitycheck's expr_parser is doing some matching on
compatibles in child nodes that is producing unexpected results.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-11-03 14:58:51 -06:00
Martí Bolívar
2522eec578 scripts: testedtlib: add more child-binding tests
The child_binding object should default to having a path and
compatible that matches the parent's. Mark it as xfail because the
compatible part is failing.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-11-03 14:58:51 -06:00
Peter Bigot
abc72694ab doc: bump sphinx>=3.3.0
The filters were updated to match the 3.3.0 generated text, so update
the sphinx requirement to exclude versions that use different text.

Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
2020-11-03 10:24:25 -06:00
Martí Bolívar
2c19cccd4b scripts: edtlib: bindings_from_paths() helper function
This is a convenience function for creating a bunch of Binding objects
from files in a directory.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-11-02 14:26:33 -05:00
Anas Nashif
be1025a736 sanitycheck: handle overflow skips
skips due to ram/rom overflow were captured as failures. Fix this and
count them correctly as skips.

Fixes #29412

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-11-02 14:21:55 -05:00
Anas Nashif
5a6e64f448 sanitycheck: count skipped tests due to overflow
Count tests that were skipped due to SRAM/RAM overdlow as skipped.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-11-02 14:21:55 -05:00
Anas Nashif
4cd63abafd ci: add more fields to versions.json
Add date and if a commit should be considered for weekly testing.
This will still work with the old format.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-10-30 15:56:52 +01:00
Kumar Gala
33db7b5b01 scripts: edtlib: Add support for 'deprecated'
Add the ability to mark a property as 'deprecated' to get a warning that
it will be removed in the future.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-10-29 11:06:32 -05:00
Fabio Utzig
8cd7ed1980 doc: bump breathe requirement to >=4.23.0
Breathe>=4.23.0 brings in basic xrefitem rendering support and fixes
issues parsing anonymous struct/unions. This version also adds a config
knob for showing or hiding enumerator values, so set it to generate
documentation compatible with previous versions.

Signed-off-by: Fabio Utzig <fabio.utzig@nordicsemi.no>
2020-10-27 13:09:24 +01:00
Maciej Perkowski
060e00d9c5 sanitycheck: Add checking of zephyr version
Adds function checking the version of zephyr, used for reporting

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2020-10-26 14:19:34 -04:00
Maciej Perkowski
725d19b79e sanitycheck: Add info about zephyr version to the xml reports
This commit adds posibility to save zephyr version in the xml reports.
It adds 'properties' attribute to testsuite attribute. 'properties'
can contain multiple 'property' attributes. Zephyr version is added
as such 'property'.

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2020-10-26 14:19:34 -04:00
Andy Ross
60fb850713 scripts/west_commands: Don't demand non-empty output formats
It's not clear why this error is here.  The "formats" array seems to
be limited to "bin" and "hex" only, but every signing tool is going to
have its own idea of what format to emit and what ingredients need to
be used to do that.

In particular, rimage (used for the Intel Audio DSPs) doesn't use nor
generate zephyr.bin (it's very large), so it trips over this failure.

Just present the possibly-empty list of output formats to the Signer
object and let it make the decision about whether an empty formats
list is an error.

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2020-10-23 22:22:52 -04:00
Anas Nashif
3ac163eae6 boards: rename up_squared_adsp intel_adsp_cavs15
The Audio DSP is not specific to up_squared, so make it more generic.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-10-23 12:56:03 +02:00
Erwan Gouriou
48b786b159 scripts: series-push-hook: Fix behavior after rebase
Avoids parsing the whole "before to now" commits after rebase.
Now, we just parse all commits since first commit of the branch
compared to master, each timer either at first push, forced push,
forced push after rebase.

Fixes #28509

Signed-off-by: Erwan Gouriou <erwan.gouriou@linaro.org>
2020-10-21 07:16:54 -05:00
Kumar Gala
fc8c0c073c scripts: edt: Fix comment in _prop_val
_prop_val comment referred to one of the arguments called "optional"
however the code has changed to call that argument "required" now.  Fix
up the comment block to use the correct argument name and semantics of
that argument.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-10-21 06:44:32 -05:00
Kumar Gala
83be5cb9b3 scripts: edtlib: cleanup required arg to _check_prop_type_and_default
_check_prop_type_and_default doesn't use the required argument so remove
passing it to the function.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-10-21 06:44:32 -05:00
Kumar Gala
c7baf2f663 scripts: edtlib: Remove deprecated "category" support
"category" support has been deprecated for over 2 releases.  Removing
support for the keyword.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-10-21 06:44:32 -05:00
Martí Bolívar
876b961a71 scripts: edtlib: cosmetic variable name change
Change "elm" to "elem" for readability.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-10-21 06:44:32 -05:00
Martí Bolívar
7165b77a81 scripts: edtlib: refactor for first class bindings
Add two new types: Binding and PropertySpec.

- Binding is a first-class abstraction for a devicetree binding
  file as defined by a YAML file in the Zephyr syntax.

- PropertySpec is a helper type which represents a property
  definition within a Binding.

Make the Binding constructor a new entry point to the library. This
enables users to deal with bindings as standalone entities, apart from
how they characterize a particular devicetree.

Rework the EDT and Node internals that deal with bindings as dict
values to use the Binding type instead. To make this less ambiguous,
use the variable name 'raw' when we're dealing with a binding as it's
parsed from YAML, and 'binding' when we're dealing with a Python
Binding object.

This commit drops support for the following legacy bindings keys

- '#cells': use '*-cells' instead (e.g. 'gpio-cells', 'pwm-cells')
- "child-bus: foo" and "child: bus: foo": use "bus:" instead
- "parent-bus" and "parent: bus: ": use "on-bus:" instead

Officially, legacy bindings have been gone since
6bf761fc0a ("dts: Remove support for deprecated DTS binding
syntax"), so this is vestigial code, and I couldn't find any in-tree
users.

It also drops the convention in this file that ""-strings are
preferred.

I honestly don't understand why this was ever enforced; the file
itself admits single quotes are common in Python and we use them
elsewhere in Zephyr's Python code.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-10-21 06:44:32 -05:00
Martí Bolívar
8640a8d666 scripts: edtlib: tweak module docstring
Add a bit more information and clarification.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-10-21 06:44:32 -05:00
Anas Nashif
f751dd46b2 west: commands: sign: Add signing support for other CAVS targets
Sign for CAVS15, CAVS18, CAVS20 and CAVS25 SOCs

Originally-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-10-21 06:38:53 -04:00
Kumar Gala
eb2e89aecf sanitycheck: Fix bug with -M option
If we get a build failure with the -M option we get the following:

TypeError: 'NoneType' object is not iterable

This was due to having the default value of additional_keep in
cleanup_artifacts being None instead of an empty array.

Fixes #29376

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-10-20 19:30:43 -05:00
Kumar Gala
b2c07e439e sanitycheck: sanitize paths for device testing artifacts
To make the artifacts we keep around for --device-testing -M re-usable
we need to sanitize any aboslute build paths that might exist in
CMakeCache.txt or zephyr/runners.yaml.

This allows us to build the tests to run on one CI system and run the
tests on another system that has the board(s) connected to it.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-10-20 07:24:30 -05:00
Kumar Gala
1285c1f80a sanitycheck: make artifact cleanup for device-testing usable
If we specify --device-testing and -M make it so that the resulting
artifacts are usable to run with device-testing again.  This means
keeping around the zephyr binary images (zephyr.{bin, hex, elf}) and
a few files so 'west' can function for flashing (CMakeCache.txt and
zephyr/runners.yaml).

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-10-20 07:24:30 -05:00
Kumar Gala
037e1274d0 sanitycheck: qualify device testing output
If --device-testing and --build-only are specified surpress some board
related output

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-10-20 07:24:30 -05:00
Eugeniy Paltsev
ea11784919 sanitycheck: actually allow empty west-flash without parameter
According to west-flash option description it's allowed to use
west-flash without parameter:
> There are three ways this option is used.
> 1) bare: --west-flash
> 2) with a value: --west-flash="--board-id=42"
> 3) Multiple values: --west-flash="--board-id=42,--erase"

However, we don't allow to west-flash to be without parameter
when we verify sanitycheck arguments. Fix that.

Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
2020-10-16 06:26:21 -04:00
Torsten Rasmussen
fef7879583 scripts: ensuring posix path on module defined roots.
Fixes: #29235

This commit ensures that roots are being converted to posix paths.
This fixes the issue where windows style path (containing `\`) would
result in DTS dependency file to contain mixed style path separator and
thus causing Ninja to re-invoke CMake in an endless loop.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2020-10-15 22:07:11 -04:00
Martí Bolívar
305379e944 devicetree: add first round of dependency ordinal info
Add the first API functions that directly deal with node dependency
ordinals as determined by edtlib:

- DT_DEP_ORD(node_id): node_id's ordinal
- DT_REQUIRES_DEP_ORDS(node_id): list of dep ordinals for node_id's
  direct dependencies
- DT_SUPPORTS_DEP_ORDS(node_id): list of dep ordinals for nodes
  depending directly on node_id
- DT_INST_ equivalents

This is not meant to be an exhaustive set of macros related to
dependency ordinals; rather, it's a starting out point meant to enable
initial struct device dependency tracking work. We can add more if
needed.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-10-14 13:18:44 -05:00
Martí Bolívar
f752c5eeec west build: fix --pristine on early build system failure
The west build command has historically tried not to rm -rf
directories that don't look like zephyr build directories. The way it
does this is by checking for the presence of a CMake cache
with a Zephyr-specific variable (ZEPHYR_TOOLCHAIN_VARIANT) in it.

The problem with this approach is that if the build system fails
before this cache variable is set, the directory doesn't look like a
zephyr build directory, and therefore west build won't make it
pristine even with --pristine=always, even though build directories
resulting from failed runs like that are almost certainly
irrecoverably broken and need to be made pristine before anything will
work.

This leads to users having to rm -rf their directories manually, which
is not so nice.

To avoid this from happening, just check for ZEPHYR_BASE, which is
set early on in ZephyrConfig.cmake in 'modern' zephyr build systems.
Keep the ZEPHYR_TOOLCHAIN_VARIANT check in place for compatibility.

We could consider being less selective and just using shutil.rmtree()
whenever we have --pristine=always, but that would be a bigger
behavioral change than I'm comfortable doing without a good reason.

Fixes: #28876
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-10-14 07:22:32 -05:00
Kumar Gala
659b24bdcf sanitycheck: Fix results for --cmake-only
If we run with --cmake-only we get the following:

`ERROR   - Unknown status None`

Fix this by treating no status set as "passed" for the --cmake-only
case.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-10-13 13:13:21 +02:00
Martí Bolívar
8165008f44 dts: remove legacy macro support
The legacy macros were first deprecated in Zephyr v2.3. Now that
Zephyr v2.4 has been released, that makes two releases where these
macros have been deprecated, so it's OK to remove them.

This leaves support for legacy binding syntax in place. Removing that
is left to future work.

We need to update various pieces of documentation related to flash
partitions that never got updated when the new API was introduced.
Consolidate this information in the flash_map.h API reference page,
since that's really where users will run into it. This also gives us
the opportunity to improve this documentation.

Adjust a couple of kconfigfunctions.py and sanitycheck bits to use
non-legacy edtlib APIs.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-10-09 08:45:38 -05:00
Michael Hope
bf04c29c26 runners: bossac: handle --help on the current git version of BOSSA
BOSSA 1.9.1 and before an errror on `--help`, but the current git
version returns 0.  Handle both cases.

Signed-off-by: Michael Hope <mlhx@google.com>
2020-10-09 08:22:38 -05:00
Michael Hope
c0055ed392 runners: bossac: calculate the offset instead of absolute address
BOSSA takes the offset within flash memory, not the absolute address.
This doesn't matter on most boards as the flash starts at zero but
does matter on the Due as the flash starts at +512 KiB.

Signed-off-by: Michael Hope <mlhx@google.com>
2020-10-09 08:22:38 -05:00
Martí Bolívar
1ea7bf0db0 scripts: dts: fix handling for phandle-arrays with *-cells=0
Each controller node in a phandle-array may set the number of cells in
a specifier as any nonnegative integer. Currently, we don't allow
this in edtlib in the case where there are multiple controllers in a
phandle-array property all of which have 0 cells in the relevant
specifier, which is not correct. Fix this, add a regression test, and
improve the error message while we are here.

Fixes: #28709
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-10-06 12:05:37 -05:00
Peter Bigot
44394e3d8e edtlib: add EDT.dep_ord2node
Add a lookup table for finding a node by its dependency ordinal.

Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-10-06 09:50:39 -05:00
Martí Bolívar
b6db2019dd edtlib: propertize EDT.scc_order, set up graph earlier
Make the scc_order method a property instead. This is in keeping with
the "General biased advice" at the top of file.

The actual order is therefore lazily initialized in this commit and
the order is not computed by the time __init__() returns. The next
commit will invoke scc_order by the time the constructor returns.

This is preparation work for adding a lookup table from dependency
ordinals to nodes. The combination of these two changes will make
intializing that lookup table a bit easier.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-10-06 09:50:39 -05:00
Martí Bolívar
3a2dc77743 dts: break cycles in test scripts
We don't want to support cyclic dependency structures, because it
means that Node objects cannot have dep_ordinal attributes as they are
currently documented to possess unconditionally.

Nevertheless, we have some in our tests. Remove them by extracting the
/props/ctrl-X nodes to the same level as the /props nodes. This breaks
a cycle caused by:

- /props/ctrl-X nodes depend on /props because of the parent/child
  relationship
- /props depends on /props/ctrl-X because it refers to them by phandle

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-10-06 09:50:39 -05:00
Martí Bolívar
09b8db4088 nrfjprog.py: handle ImportError
We need to handle ImportErrors on all non-stdlib imports in the
runners package.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-10-06 07:38:58 -04:00
Maciej Perkowski
2ec5ec2153 sanitycheck: Fix ZeroDivisionError during reporting
This patch fixes the ZeroDivisionError which can
occur during percantage of test execution reporting.

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2020-10-02 07:56:17 -04:00
Martí Bolívar
a6856811a3 scripts: dts: pass tests on windows
Doing this just requires a bit of os.fspath()-ery.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-10-02 11:51:15 +02:00
Martí Bolívar
a8612f75c5 scripts: dts: convert test suites to pytest
Use the pytest test framework in the dtlib.py and edtlib.py test
suites (testdtlib.py and testedtlib.py respectively).

The goal here is not to change what is being tested. The existing test
suite is excellent and very thorough.

However, it is made up of executable scripts where all of the tests
are run using a hand-rolled framework in a single function per file.
This is a bit all-or-nothing and prevents various nice features
available in the de-facto standard pytest test framework from being
used.

In particular, pytest can:

- drop into a debugger (pdb) when there is a problem
- accept a pattern which specifies a subset of tests to run
- print very detailed error messages about the actual and expected
  results in various traceback formats from brief to very verbose
- gather coverage data for the python scripts being tested (via plugin)
- run tests in parallel (via plugin)
- It's easy in pytest to run tests with temporary directories
  using the tmp_path and other fixtures. This us avoid
  temporarily dirtying the working tree as is done now.

Moving to pytest lets us leverage all of these things without any loss
in ease of use (in fact, some things are nicer in pytest):

- Any function that starts with "test_" is automatically picked up and
  run. No need for rolling up lists of functions into a test suite.
- Tests are written using ordinary Python 'assert'
  statements.
- Pytest magic unpacks the AST of failed asserts to print details on
  what went wrong in really nice ways. For example, it will show you
  exactly what parts of two strings that are expected to be equal
  differ.

For the most part, this is a pretty mechanical conversion:

- extract helpers and test cases into separate functions
- insert temporary paths and adjust tests accordingly to not match
  file names exactly
- use 'assert CONDITION' instead of 'if not CONDITION: fail()'

There are a few cases where making this happen required slightly
larger changes than that, but they are limited.

Move the checks from check_compliance.py to a new GitHub workflow,
removing hacks that are no longer needed.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-10-02 11:51:15 +02:00
Marcin Niestroj
fc67409291 sanitycheck: ignore coverage directories by matching any in path hierarchy
So far 'tests' and 'samples' directories were matched only when they
were first directories in path hierarchy. This doesn't work when running
sanitycheck from directory other than Zephyr source code root.

Match any directory in path hierarchy, similar how file is matched
currently. That way sanitycheck can be executed from outside of Zephyr
source code directory.

Signed-off-by: Marcin Niestroj <m.niestroj@grinn-global.com>
2020-10-01 17:11:34 -04:00