By default, subprocess.Popen commands are supplied as a list of strings.
Using split() allows to use command with arguments, for example it is
possible to use following:
sanitycheck ... \
--device-serial-pty="set_ambient read_terminal.py" \
...
Signed-off-by: Andrei Emeltchenko <andrei.emeltchenko@intel.com>
This commit message is a bit of a novel mostly:
- because the issues involved are longstanding
- as evidence this is not a capricious refactoring
The runners.core.RunnerConfig Python class holds common configuration
values used by multiple runners, such as the location of the build
outputs and board directory.
The runners code, first written in 2017-ish, replaced various shell
scripts that got this information from the environment. Avoiding
environment variables was a requirement, however. It's ghastly to set
environment variables for a single command invocation on Windows, and
the whole thing was part of a larger push to make Zephyr development
on Windows better.
I had a hammer (the argparse module). Finding a replacement naturally
looked like a nail, so the information that ends up in RunnerConfig
got shunted from the build system to Python in the form of 'west
flash' / 'west debug' command line options like '--board-dir',
'--elf-file', etc.
I initially stored the options and their values in the CMake cache.
This was chosen in hopes the build system maintainer would like
the strategy (which worked).
I knew the command line arguments approach was a bit hacky (this
wasn't a nail), but I also honestly didn't have a better idea at the
time.
It did indeed cause issues:
- users don't know that just because they specify --bin-file on the
command line doesn't mean that their runner respects the option, and
have gotten confused trying to flash alternate files, usually for
chain-loading by MCUboot (for example, see #15961)
- common options weren't possible to pass via board.cmake files
(#22563, fixed partly via introduction of runners.yaml and the west
flash/debug commands no longer relying on the cache)
- it is confusing that "west flash --help" prints information about
openocd related options even when the user's board has no openocd
support. The same could be said about gdb in potential future use
cases where debugging occurs via some other tool.
Over time, they've caused enough users enough problems that
improvements are a priority.
To work towards this, put these values into runners.yaml using a new
'config: ...' key/value instead of command line options.
For example, instead of this in the generated runners.yaml file:
args:
common:
- --hex-file=.../zephyr.hex
we now have:
config:
hex_file: zephyr.hex
and similarly for other values.
In Python, we still support the command line options, but they are not
generated by the build system for any in-tree boards. Further work is
needed to deprecate the confusing ones (like --hex-file) and move the
runner-specific host tool related options (like --openocd) to the
runners that need them.
Individual board.cmake files should now influence these values by
overriding the relevant target properties of the
runners_yaml_props_target.
For example, instead of:
board_runner_args(foo "--hex-file=bar.hex")
Do this:
set_target_properties(runners_yaml_props_target PROPERTIES
hex_file bar.hex)
This change additionally allows us to stitch cmake/mcuboot.cmake and
the runners together easily by having mcuboot.cmake override the
properties that set the hex or bin file to flash. (The command line
arguments are still supported as-is.)
Combined with 98e0c95d91ae16f14e4997fb64ccdf0956595712 ("build:
auto-generate signed mcuboot binaries"), this will allow users to
build and flash images to be chain loaded by mcuboot in a way that
avoids calling 'west sign' and passing 'west flash' its output files
entirely.
While we are here, rename runner_yml_write to runners_yaml_append().
This function doesn't actually write anything, and we're here
refactoring this file anyway, so we might as well improve the
situation while we're at it.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
The 'runner_config' variable name is particularly misleading because
there is a class called RunnerConfig, and that variable does not
contain one.
Rename it to 'runners_yaml' since it contains the parsed contents of
the runners.yaml file. Rename the variable that refers to the path
itself to 'runners_yaml_path'. No functional changes expected.
This is prep work for redoing how actual RunnerConfig objects get
made.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Before venturing off into refactoring land, let's make sure we have a
type checker running on the main APIs used by the various runners so
we don't miss anything obvious.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Use mypy to type check the runners package.
The test procedure is now annoying enough to replicate locally that
I'm going to wrap it in a script. Do this for both UNIX and Windows
environments by writing that script in Python.
Keep the GitHub workflow up to date so we now get mypy results in CI.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Colorama needs to be initialized, and for colors to display correctly on
Windows one needs to use the `USES_TERMINAL` option in CMake.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Certain symbols do not contain a valid path, instead showing
`<built-in>` in the filename attribute. This leads to the resolve() call
failing on Windows, since the `<>` characters are not allowed in
filenames there. Fix this by catching the exception and skipping the
call in that case.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2 scripts are provided.
- find_functions.cocci (name probably sucks...)
- find_dev_usage.cocci (ditto...)
find_functions.cocci can patch files where device instance are not
const.
Then it is used to generate the function database:
./scripts/coccicheck --mode=report --jobs=1 \
--cocci=scripts/coccinelle/find_functions.cocci \
--sp-flag="--include-headers" ./
Then, find_dev_usage.cocci will check if the const qualifier is, or
might be lost in a function call.
For instance:
./scripts/coccicheck --mode=report --jobs=1 \
--cocci=scripts/coccinelle/find_dev_usage.cocci \
--sp-flag="--include-headers" drivers/i2c
Which will output a WARNING on non-zephyr functions calls with a device:
./drivers/i2c/i2c_cc13xx_cc26xx.c:393:5-8: WARNING: in i2c_cc13xx_cc26xx_pm_control calling cb param with dev, check if const qualifier is not lost
./drivers/i2c/i2c_mcux_lpi2c.c:205:40-43: WARNING: in mcux_lpi2c_init calling LPI2C_MasterTransferCreateHandle param with dev, check if const qualifier is not lost
./drivers/i2c/i2c_nrfx_twi.c:258:5-8: WARNING: in twi_nrfx_pm_control calling cb param with dev, check if const qualifier is not lost
./drivers/i2c/i2c_nrfx_twi.c:202:22-25: WARNING: in init_twi calling nrfx_twi_init param with dev, check if const qualifier is not lost
./drivers/i2c/i2c_mcux.c:187:38-41: WARNING: in i2c_mcux_init calling I2C_MasterTransferCreateHandle param with dev, check if const qualifier is not lost
./drivers/i2c/i2c_mcux_flexcomm.c:184:43-46: WARNING: in mcux_flexcomm_init calling I2C_MasterTransferCreateHandle param with dev, check if const qualifier is not lost
./drivers/i2c/i2c_nrfx_twim.c:232:5-8: WARNING: in twim_nrfx_pm_control calling cb param with dev, check if const qualifier is not lost
./drivers/i2c/i2c_nrfx_twim.c:174:8-11: WARNING: in init_twim calling nrfx_twim_init param with dev, check if const qualifier is not lost
./drivers/i2c/i2c_rv32m1_lpi2c.c:246:6-9: WARNING: in rv32m1_lpi2c_init calling LPI2C_MasterTransferCreateHandle param with dev, check if const qualifier is not lost
Or:
./scripts/coccicheck --mode=report --jobs=1 \
--cocci=scripts/coccinelle/find_dev_usage.cocci \
--sp-flag="--include-headers" drivers/ieee802154
Which will output an ERROR on using a zephyr function that looses the
const qualifier:
drivers/ieee802154/ieee802154_rf2xx.c:778:3-6: ERROR: in rf2xx_init calling k_thread_create param with dev, loosing const qualifier, please wrap
drivers/ieee802154/ieee802154_nrf5.c:477:19-22: ERROR: in nrf5_init calling k_thread_create param with dev, loosing const qualifier, please wrap
drivers/ieee802154/ieee802154_cc1200.c:819:3-6: ERROR: in cc1200_init calling k_thread_create param with dev, loosing const qualifier, please wrap
drivers/ieee802154/ieee802154_mcr20a.c:1443:3-6: ERROR: in mcr20a_init calling k_thread_create param with dev, loosing const qualifier, please wrap
drivers/ieee802154/ieee802154_cc2520.c:1116:3-6: ERROR: in cc2520_init calling k_thread_create param with dev, loosing const qualifier, please wrap
drivers/ieee802154/ieee802154_cc13xx_cc26xx.c:439:32-35: ERROR: in ieee802154_cc13xx_cc26xx_data_init calling k_thread_create param with dev, loosing const qualifier, please wrap
ISSUE:
- Is it possible to run a set of rules first on all the code, and then
another set, both sets being in the same .cocci file?
Would be nice to have all at once.
Fixes#27399
Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
The k_object API associates mutable state structures with known kernel
objects to support userspace. The kernel objects themselves are not
modified by the API, and in some cases (e.g. device structures) may be
const-qualified. Update the API so that pointers to these const
kernel objects can be passed without casting away the const qualifier.
Fixes#27399
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
Fix the case where we have no ztest test_* testcases and reports were
missed. This now makes sure we do not report the parent testcase when
there are individual test_* results.
Fixes#27765
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
add argument --pre-script to specify a pre script. This will be
executed before device handler open serial port and invoke runner.
Currently, pre_script can only get from hardware.map file. just like:
./scripts/sanitycheck --hardware-map xxx.map
and set pre_script in xxx.map file.
Signed-off-by: Watson Zeng <zhiwei@synopsys.com>
In some cases it might be a good idea to reset the board for real to
make sure it is completely recovered from some failed state (simple
re-loading of the application binary or even Elf file contents doesn't
affect most of internal CPU states so doesn't help in recovery,
see https://github.com/zephyrproject-rtos/zephyr/issues/25022 &
https://github.com/zephyrproject-rtos/zephyr/issues/26665).
And so we may want to utilize some external utility which triggers the
hard reset (in case of ARC boards it is
https://github.com/foss-for-synopsys-dwc-arc-processors/rff-ftdi-reset).
So we need to have a way to execute an external command before each and
every test.
Now given we already have quite some call-backs we try to use them
before re-inventing the wheel. And pre_script seem to be a good option
with just on minor note - it is called after serial port gets open.
And while in some cases it might be OK if serial port on the board is
not affected by the board's reset, if it is affected we'll be losing
connection on reset (and that's the case with ARC boards BTW as the
FTDI USB-to-Serial IC is also wired to the reset signal on most of
the boards). That said we just move invocation of pre_script before
opening the serial port and everything should be good now.
Signed-off-by: Watson Zeng <zhiwei@synopsys.com>
By default show reports based on last release. Fix a few other issues
where we had 0 values and were dividing by zero.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
This commit improves Zephyr modules support in CMake.
All Zephyr modules will now have a corresponding
`ZEPHYR_${module_name}_MODULE_DIR` set, regardless of whether they
define a cmake folder in module.yml.
This results in:
`ZEPHYR_${module_name}_MODULE_DIR` defined for all modules and refers to
the root of the modle.
`ZEPHYR_${module_name}_CMAKE_DIR` defined for modules that specifies
cmake in module.yml, or is having both a CMakeLists.txt and Kconfig file
in the zephyr folder.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
This commit has no functional changes but is simply a cleanup so that
flake8 will pass without warnings.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
This will only build/run on emulation platforms.
The decision is made based on the value of the 'simulation' key in the
platform yaml file.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
This is needed for sanitycheck hardware maps which take the serial
number directly from USB metadata.
Signed-off-by: Øyvind Rønningstad <oyvind.ronningstad@nordicsemi.no>
Adds some code to allow QEMU to crash without failing the test.
This is required for testing coredump code as it will certainly
cause QEMU to crash.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
There is a potential that QEMUHandler.handle() returns before
the testing harness finishes processing all the output from QEMU.
Simply wait for the harness thread to finish before returning.
Also, fix the return code in the debug message as it should be
the return code from Popen().
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This adds the necessary bits in arch code, and Python scripts
to enable coredump support for ARM Cortex-M.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This adds a very primitive coredump mechanism under subsys/debug
where during fatal error, register and memory content can be
dumped to coredump backend. One such backend utilizing log
module for output is included. Once the coredump log is converted
to a binary file, it can be used with the ELF output file as
inputs to an overly simplified implementation of a GDB server.
This GDB server can be attached via the target remote command of
GDB and will be serving register and memory content. This allows
using GDB to examine stack and memory where the fatal error
occurred.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Commit 8d7bb8ffd8 refactored
device structures which changed some of the linker sections:
"devconfig" was removed, and "devices" was added. However,
the list in sanitycheck's size calculator was not updated,
which results in sanitycheck complaining about unrecognized
sections when doing footprint.
Also, a few sections have been renamed (with added suffix
"_area") due to introduction of Z_ITERABLE_SECTION_RAM/ROM
macros. There are also some missing section names.
Fixes the issue by adding the missing sections names, and
updating existing ones.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
When MCUBoot is built using the swap-move strategy, the secondary slot
can be one sector smaller than the primary slot, because the primary
slot's last "useful" image sector must be "reserved" for the move
operation; this impacts the generation of padded images, so when
signing an image, the proper size to use is the secondary slot's,
unless a secondary image is not defined which is the case when using
single image DFU.
Signed-off-by: Fabio Utzig <fabio.utzig@nordicsemi.no>
The extra empty line broke the "ninja hardenconfig" on my machine with
Python 3.7.5, it complains:
"
... ...
File "/home/zephyrproject/zephyr/scripts/kconfig/hardenconfig.py",
line 46, in compare_with_hardened_conf
name = row[0]
IndexError: list index out of range
FAILED: CMakeFiles/hardenconfig
"
The csv.reader reads this empty line and gets an empty list which will
not be successfully "de-referenced". Removing it to improve the
out-of-box experience when pepople try out the hardening options.
Signed-off-by: Wenbo Yang <wenbo.yangcn@gmail.com>
This squashes and applies the Linux diffs to scripts/checkpatch.pl
between Linux commit 16fbf79b0f83bc75 ("Linux 5.6-rc7") and
9123e3a74ec7b93 ("Linux 5.9-rc1") except for commits identified below.
The last 1000 commits to Zephyr master were compared for checkpatch
output differences between the previous Zephyr version and this
version. One new diagnostic about function declarations with an empty
parameter-list was introduced (FUNCTION_WITHOUT_ARGS). The text of an
existing diagnostic was changed (DT_SPLIT_BINDING_PATCH). The text of
LONG_LINE diagnostics was enhanced to provide the actual line length.
Linux commit dfa05c28ca7ffc0a ("checkpatch: remove email address
comment from email address comparisons") was removed because
differences in the scripts resulted in false signed-off-by check
diagnostics when a full name included characters not in Basic Latin,
due to changes in how the author name was extracted. Earlier upstream
changes not integrated into Zephyr may be required.
Linux commit b95692f8b3000166 ("checkpatch: prefer fallthrough; over
fallthrough comments") was removed because Zephyr doesn't support the
upstream pseudo keyword.
Linux commit bdc48fa11e46f867 ("checkpatch/coding-style: deprecate
80-column warning") was edited to not actually change the 80-column
maximum line limit as this change has not been mooted for Zephyr.
Linux commit ced69da1db0b57bb ("checkpatch: fix CONST_STRUCT when
const_structs.checkpatch is missing") was edited to the CONST_STRUCT
file as that's not supported in Zephyr.
Signed-off-by: Peter A. Bigot <pab@pabigot.com>
Add support for the J-Link Commander "-NoGui 1" command line parameter
in the West J-Link runner.
This command line parameter suppresses GUI dialogs (except for license
dialogs) in J-Link Commander starting from v6.80.
Signed-off-by: Henrik Brix Andersen <hebad@vestas.com>
If a runner supports the --erase option, make it so that the user can
add --no-erase to the command line to explicitly disable it.
Add a diagnostic at info() level whenever this option is requested.
The intent of this commit is to make it more obvious when a mass erase
was requested, especially in situations (like MCUboot with
CONFIG_MCUBOOT_TRY_MASS_ERASE=y) where this option may be turned on by
default.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Fixes: #27375
This is a cleanup of the Zephyr CMake package export.
The code has been simplified so that the export now happens through a
CMake script. This avoids several generated CMake build files compared
to previous export mode, and thus removes the need for a CMake pristine
script.
A benefit of this cleanup is that it also fixes#27375.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
This patch should improve couting and reporting of the number
of passed/skipped tests.
Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
Avoid use of the cached_runner_config() helper, which we have a TODO
item to get rid of. Adjust the output and do some other minor
improvements.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Rolling yet another parser turns out to be the best way to let west
extensions respond to Kconfig values. See source code comments in the
patch for details.
The sanitylib library has some similar functionality but it isn't
exactly the same, and it gets strings wrong. For example, that parser
can't handle this option:
CONFIG_FOO="he said \"no\" to me"
This one can, and it has a couple of other features we'll find useful
for west extensions eventually besides.
(Not to mention that sanitylib also rolled its own CMake cache parser,
which also exists in west_commands.)
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Use anytree module to display tree and to allow easy exporting into
json.
Add option to export results into a json file.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
In DWARF 4, e.g. ARC Metaware toolchain, DW_AT_count is
used not DW_AT_upper_bound. We should consider this corner
case.
Signed-off-by: Wayne Ren <wei.ren@synopsys.com>
When running with --test-only we get:
Traceback (most recent call last):
File "./scripts/sanitycheck", line 1168, in <module>
main()
File "./scripts/sanitycheck", line 1160, in main
options.only_failed)
File "scripts/sanity_chk/sanitylib.py", line 2543, in save_reports
self.xunit_report(filename + ".xml", full_report=False, append=only_failed)
File "scripts/sanity_chk/sanitylib.py", line 3220, in xunit_report
return fails, passes, errors, skips
UnboundLocalError: local variable 'fails' referenced before assignment
This is due to the fact that selected_platforms was not set.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
In the junit output the attribute containing the number of skipped tests
must be named "skipped" instead of "skip".
See e.g. https://github.com/junit-team/junit5/blob/main/platform-tests/
src/test/resources/jenkins-junit.xsd#L95
Signed-off-by: Christian Taedcke <christian.taedcke@lemonbeat.com>
The extra empty line broke the "ninja hardenconfig" on my machine with
Python 3.7.5, it complains:
"
... ...
File "/home/zephyrproject/zephyr/scripts/kconfig/hardenconfig.py",
line 46, in compare_with_hardened_conf
name = row[0]
IndexError: list index out of range
FAILED: CMakeFiles/hardenconfig
"
The csv.reader reads this empty line and gets an empty list which will
not be successfully "de-referenced". Adding extra check to skip the
empty lines.
Signed-off-by: Wenbo Yang <wenbo.yangcn@gmail.com>
Running check_compliance on a PR before submitting it can avert
embarrassing mistakes. Ensure the packages needed to do so are
installed along with all the others.
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
Currently '--coverage-tool gcovr' results in using lcov. Fix that to use
gcovr, as requested by user.
Fixes: f6462a3a8c ("sanitycheck: get rid of global VERBOSE")
Signed-off-by: Marcin Niestroj <m.niestroj@grinn-global.com>
Sanitycheck discards test instances if a set of given coditions
is not fulfilled. This leads to empty test results for these
instances. This can introduce ambiguity with results that are
missing due to some bugs in the framework. This commit fills
the results for skipped tests with 'skipped' states and provides
the reason for filtering them out in the msg field. The commit
also fixes the way sanitycheck counts and reports tests/test cases
Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
Fixes: #25215
This commit introduces the possibility of defining ROOTs in a Zephyr
module and have it automatically appended to list of other ROOTs.
Supported with this commit:
- BOARD_ROOT
- SOC_ROOT
- DTS_ROOT
- ARCH_ROOT
In order to support this in Zephyr module files, the detection of west
has been moved to dedicated west.cmake file and included immediately
after python.cmake.
Also the inclusion of zephyr_modules.cmake has moved before first use
of BOARD_ROOT.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
This commit introduces support for multiple SOC_ROOT.
This means that additional SOC_ROOTs specified using -DSOC_ROOT as
argument to CMake will be forming a list together with ${ZEPHYR_BASE}.
This allows for greater flexibility, as developers can now specify
multiple out-of-tree SoCs and not worry about the SoC used for the
board they compile for.
Also it avoid code, such as:
if(BOARD STREQUAL my_board_using_out_of_tree_soc)
set(SOC_ROOT some/out/of/tree/soc/path)
endif()
in application CMakeLists.txt.
Finally, allowing multiple SOC_ROOTs prepares for specifying SOC_ROOTs
in Zephyr modules.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
When passing a using --hardware-map, sanitycheck would ignore extra
args passed via --west-flash, such as --west-flash="--erase".
This is because the command with the extra args was overwritten by the
command with the runner info from the hardware map.
This patch merges those code paths so they are aware of each other and
of --west-runner.
Signed-off-by: Øyvind Rønningstad <oyvind.ronningstad@nordicsemi.no>
Before sanitycehck was removing all `test_` chars in test cases'
names. This test has to be modify to work with the improved behavior
Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
This commit limits the removal of `test_` from test case name
to only the first occurance. There are test cases with `test_`
also in the middle of their names and removing it couses mismatches
between extracted testcases and the names obtained when passing
Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
These stacks are appropriate for threads that run purely in
supervisor mode, and also as stacks for interrupt and exception
handling.
Two new arch defines are introduced:
- ARCH_KERNEL_STACK_GUARD_SIZE
- ARCH_KERNEL_STACK_OBJ_ALIGN
New public declaration macros:
- K_KERNEL_STACK_RESERVED
- K_KERNEL_STACK_EXTERN
- K_KERNEL_STACK_DEFINE
- K_KERNEL_STACK_ARRAY_DEFINE
- K_KERNEL_STACK_MEMBER
- K_KERNEL_STACK_SIZEOF
If user mode is not enabled, K_KERNEL_STACK_* and K_THREAD_STACK_*
are equivalent.
Separately generated privilege elevation stacks are now declared
like kernel stacks, removing the need for K_PRIVILEGE_STACK_ALIGN.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
We have been doing an AND comparison instead of an OR. AND does exclude
way to many testcases where multiple tags are being used.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Use --integration testing for CI to limit execution and builds on
platforms that actually provide most of the coverage instead of blindly
building/running on all available platforms.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
On some NXP LPC MCUs (such as LPC11U6x), a valid firmware image must
include a checksum in the 7th entry of the ARM Cortex-M0 exception
vector table. This checksum is verified by the bootloader.
The lpc_checksum.py python script allows to insert this checksum into
the firmware images (BIN and HEX formats).
Project page: https://github.com/basilfx/lpc_checksum
Signed-off-by: Simon Guinot <simon.guinot@seagate.com>
Make net_if a kernel object with type K_OBJ_NET_IF so that we
can restrict access to it.
Signed-off-by: Jukka Rissanen <jukka.rissanen@linux.intel.com>
instances that do not run will have no results (beside the fact they
built successfully), so log those as such.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
This is a cargo-culted attempt to make an error observed in poorly
understood CI circumstances go away when loading edt.pickle, by making
edtlib visible in sys.modules before loading the pickle file.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Adding the get_maintainer.py script
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
Consolidate creation of edtlib.EDT objects from a build directory's
devicetree into one place by loading it from build/zephyr/edt.pickle
everywhere. A previous commit creates edt.pickle from gen_defines.py.
In addition to probably speeding things up slightly by not reparsing
the devicetree, the main benefit of this approach is creating a single
point of truth for the bindings directories and warnings
configuration, meaning we don't have to worry about them getting out
of sync while being passed around between devicetree creation and
usage time.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Increased minimal pyelftools version to 0.26 else
scripts/footprint/size_report will fail to import LocationExpr from
elftools.dwarf.locationlists
Signed-off-by: Jan Van Winkel <jan.van_winkel@dxplore.eu>
I can never remember what the contents of the kconfigfunctions dict is
and have to look it up every time. I suspect I'm not alone, and not
everyone who uses these will know how to look up the answer. Add a
comment explaining the value.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
There is already a way to check if a node is enabled based on its node
label, but we don't have an equivalent way to do that for a path or an
alias. Add them. These rely on the same underlying edtlib machinery to
get their jobs done, but are being treated differently within Kconfig
in order to match distinctions between paths and aliases made in the
devicetree.h API.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
This adds CONFIG_BT_CONN_DISABLE_SECURITY which can be used to disable
security checks for incoming requests enabling to test accessing GATT
attributes and L2CAP channels that would otherwise require
encryption/authentication in order to be accessed.
It depends on BT_TESTING to indicate to the users that this is a
testing feature which shall not be used in production.
Signed-off-by: Luiz Augusto von Dentz <luiz.von.dentz@intel.com>
serial-pty is an extension that is not supported in the hardware map, so
we need to work around the fact that it is not part of the map when
dealing with normal serial.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Add option to provide to sanitycheck argument for creating
pseudoterminal. Used with hardware without serial console connected.
A pseudoterminal is used to make a sanitycheck believe that it
interacts with a terminal although it actually interacts with the
script.
E.g "sanitycheck --device-testing --device-serial-pty <script>"
Signed-off-by: Andrei Emeltchenko <andrei.emeltchenko@intel.com>
Support running/building only specific tags and ignoring everything
else.
Adding this to a platform yaml file will enable tests with one of of
tags listed to be executed on the platform.
This is useful for special platform configurations used for testing
specific features for example.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Refactor subfolder_list.py to enable unit testing. This means split the
implementation into separate logical functions. Additionally, modify the
code to improve PEP 8 compatibility and pylint result.
Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
The projects variable needs to be initialized with the list of modules
provided via cmd-line arguments.
This is a regression introduced by
ef3c5e5516.
Fixes#26948.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
--warnings-as-errors was basically doing nothing, it is the default.
Replace this with an option to disable erroring on warning:
-W, --disable-warnings-as-errors
Fixes#26910
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
We still have cases where qemu crashes are not being correctly captured
in the on-screen summary, however they are captured in the final
generated report.
Fixes#26679
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
...instead of GNU binutils to extract DWARF information.
This is now a bit more portable across OS and toolchain.
One bouns is that this nows with on qemu_x86_64.
A few differences are:
() z_mrsh_* which are aliases to handler_no_syscalls() are now
dropped as they are mapped to the same address, so that they
are not counted multiple times.
() Static functions and variables with same names should now be
attributed to the correct source files instead of being
accumulated into the same symbol of one file (e.g. multiple
thread_entry() in kernel tests).
() The totals for ROM and RAM are calculated from the
corresponding sections. Previous script includes the debug
sections as total ROM size which is not entirely correct.
Fixes#22996
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
restore how --only-failed works by allow rebuilds in case of build
errors, however, do not rebuild when --retry-failed is used, which is a
CI usecase and nothing would change in the second iteration.
Fixes#26685
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
The `TEXT_SECTION_OFFSET` symbol is used to specify the offset between
the beginning of the ROM area and the address of the first ROM section.
This commit renames `TEXT_SECTION_OFFSET` to `ROM_START_OFFSET` because
the first ROM section is not always the `.text` section.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
Updating mock module version to >=4.0.1 as sanitycheck testsuite
might fail in some systems due to old version.
Signed-off-by: Aastha Grover <aastha.grover@intel.com>
When running CMake, then Python3 will be used.
This is detected through FindPython3, with a preference for using the
python or python3 in path, if any of those matches the required Python
minimal version in Zephyr.
It is also possible for users to specify a different Python, as example
by using:
`cmake -DPYTHON_PREFER=/usr/bin/python3.x`
However, when running `west` as native command, then west will be
invoked on linux based on the python defined in:
`west` launcher, which could be: `#!/usr/bin/python3.y`
Thus there could be mismatch in Pythons used for `west` and the python
used for other scripts.
This is even worse on windows, where a user might experience:
```
>.\opt\bin\Scripts\west.exe --version
Traceback (most recent call last):
File "C:\Python37\lib\runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
...
File "C:\Python37\lib\socket.py", line 49, in <module>
import _socket
ImportError: Module use of python38.dll conflicts with this version of
Python.
```
when testing out a newer Python, but the python in path is still a 3.7.
By importing `west` into zephyr_module.py and by using, as example
`python -c "from west.util import west_topdir; print(topdir())"`
we ensure the same python is used in all python scripts.
Also it allows the user to control the python to use for west.
It also ensures that the west version being tested, is also the version
being used, where old code would test the version imported by python,
but using the west in path (which could be a different version)
If the west version installed in the current Python, and west invocation
is using a different Python interpreter, then an additional help text
is printed, to easier assist users with debugging.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
We only have one use of _binding_compat and it doesn't need self, so
just fold it into _init_compat2binding.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
We deprecated a number of aspects of the DTS binding syntax in Zephyr
2.1. Remove the support for the deprecated syntax. Remove from docs
about the deprecated syntax as well.
Removed reference in release-notes-2.1.rst to legacy_binding_syntax
since that anchor doesn't exist anymore.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
According to our compliance checks, we have 'useless-suppression'
and 'unused-import' on our hands:
W0611: Unused CLoader imported from yaml as Loader
W0611: Unused Loader imported from yaml
I0021: Useless suppression of 'unsubscriptable-object'
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Now that kconfigfunctions.py is loading the EDT object from a pickle
file, I can't find any other reads of the environment variables it was
previously using to do that.
(The CMake variables with the same names are read in different places;
I'm specifically talking about environment variables here.)
Remove the dead stores.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
This saves time and avoids the need to pass additional state around in
the environment to recreate the edt exactly.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
We need to save and restore the devicetree data to generate optimized
dependency information later on in the build, in particular during the
final application link.
Make this happen by pickling the EDT object in BUILD_DIR/edt.pickle.
The existence of this file is an implementation detail, so do not add
it to the documentation.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
We have a use case for saving the EDT object to be able to open it up
again later. It would be convenient to be able to do this with the
pickle module from stdlib.
The only thing stopping us from doing that appears to be the open
reference to sys.stderr that's held the edt object even after
EDT.__init__ exits. However, there doesn't seem to be a need to keep
holding on to this object, and in fact it would be a little bit nicer
to drop the reference in case something else (even in the same Python
process that created it originally) wants the EDT object around, but
might want the warn file closed if its refcount zeroes out.
Just drop the reference at the end of __init__ and make EDT._warn()
throw an exception if it's attempted to be used after the constructor
exits.
Make pickle-ability an API guarantee so we can treat any regressions
as bugs going forward.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
I can't see a good reason to be doing this in the Node class's
unit_addr accessor. Move it up to the edtlib initialization so it only
happens once.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Use the C LibYAML parser if available, rather than the Python parser. It
is much faster.
This is a clean and rebased version of PR #20210 by
Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Co-authored-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Deal with qemu crashes and do not register pass status as the default.
In case we do not have a status, report unknown.
This now captures different states from qemu that were not captured
before.
Fixes#26679
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Recent ST boards embed the new ST-Link probe V3.
It is advertised as "STLINK-V3", update sanitylmib to take it
into account.
In handle function, it is proposed to treat it separately as current
"STM32 STLink" as processing might differ in next future (hla_serial
deprecation).
Signed-off-by: Erwan Gouriou <erwan.gouriou@linaro.org>
Some devices have multiple serial ports, which show as duplicates in the
hardware map. This doesn't work well when using --generate-hardware-map
to regenerate an existing map. E.g. nrf5340pdk_nrf5340_cpuapp by default
prints to the 3rd of 3 devices. If it shows up on /dev/ttyACM0,
/dev/ttyACM1, and /dev/ttyACM2, printout will come on /dev/ttyACM2.
nrf9160dk_nrf9160 also provides 3 devices, but prints to the 1st.
This patch sorts the devices by the serial port and matches multiple
duplicate entries one-to-one to retain the ordering. This way, the
correct device can be given the platform name and the others can be kept
as "unknown" so that --device-testing understands correctly
1) that there is only one device (not 3)
2) where the serial output will come.
Signed-off-by: Øyvind Rønningstad <oyvind.ronningstad@nordicsemi.no>
test_reporting_testsuite.py: Adding testcases for reporting
functions of Class testSuite.
test_testsuite_class.py: Testcase for add_instances function
conftest.py: Added fixture for instances_fixture
Signed-off-by: Aastha Grover <aastha.grover@intel.com>
Modifying discard_report function to fix the check for
self.discards dictionary in try block. self.discards
is None is dead code which didnot check if the dictionary was
empty. Changed it to if not self.discards.
Signed-off-by: Aastha Grover <aastha.grover@intel.com>
- Report build errors as errors, not test failures
- Do not try and build/run tests with build failures
- Fix issue with empty reports when running --only-failed
- Report build errors in the detailed and target reports
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
BOSSAC uses a bootloader, so pull the flash address from DeviceTree or
the config and use if the version of BOSSAC supports offsets.
Signed-off-by: Michael Hope <mlhx@google.com>
BOSSA 1.7 has built-in bootloader offset handling while 1.9.1
requires the user to supply the offset. Add support for both by
sniffing the help output and warn the user if the flags needs adding.
Related to #22062
Signed-off-by: Michael Hope <mlhx@google.com>
This commit creates a list of a phony targets for each runner, that is:
`west_flash_depends`, `west_debug_depends`, and so on.
Those targets has identical dependencies as CMake runner target.
flash, debug, debugserver, attach targets.
As example `ninja flash` correctly ensures dependencies are taken into
consideration before calling `west flash`.
Unfortunately, calling `west flash` directly would not re-run the flash
dependencies, cause `west flash` would only build the default CMake
target.
Now, `west flash` calls the phony `west_flash_depends` target, ensuring
all deps are up-to-date before flashing (unless --skip-rebuild is given)
The same is true for the other mentioned runners.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Missed the case we use in CI where we pre-generate the testcase list and
load it, this was resulting in empty test reports in CI.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
test_testinstance.py: Add testcases to scan file and path for
sub testcases
test_data/testcases: Also added the ztest test files
test_testsuite_class.py: changed get_all_tests() to match
count of sub testcases in ztest files
Signed-off-by: Spoorthy Priya Yerabolu <spoorthy.priya.yerabolu@intel.com>
While testing the warnings in scan_file() format is not the same
everytime we run tests
Added sorted with reverse flag so that warnings are same for
every run
Signed-off-by: Spoorthy Priya Yerabolu <spoorthy.priya.yerabolu@intel.com>
BT simulator tests run standalone and independent of sanitycheck and
might be triggered by tests that are not managed by sanitycheck, so
leave it alone.
Fixes#26508
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Just changes to the west help output; no functional changes expected.
Make option descriptions lowercase to match the argparse module's
conventions. When multiple sentences are required, move them to parser
prolog/epilog or argument group description sections.
Clarify some points that have confused multiple people.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
If we pass --west-flash with no additional argument that was getting
treated the same as if it wasn't passed at all.
Change the check to be based on 'None' to handle the bare --west-flash
case properly.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
classname in the junit report was not being used correctly since the
beginning. We had the targetname in the classname and put all results in
1 single testsuite.
The new output add 1 single testsuite for each platform and simplifies
the classname so that tests of the same classname can be grouped in
output, producing readable reports.
The current output can be seen here:
https://testing.zephyrproject.org/daily_tests/zephyr-v2.2.0-2533-gdbbf834605/report/index.html
This should much better once we have this change in.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
In some cases we were not handling return code correctly and evaluating
timeouts as a pass. Report failure correctly.
Fixes#26065
Signed-off-by: Wentong Wu <wentong.wu@intel.com>
Move the --erase option to core.py from nrfjprog.py and jlink.py,
where it is currently supported.
Using the RunnerCaps option enforcement mechanism introduced earlier,
enforce that it will only be given to runners that support it.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Require all implementations to provide a do_create(), a new
ZephyrBinaryRunner abstract class method, and make create() itself
concrete.
This allows us to enforce common conventions related to individual
runner capabilities as each runner provides to the core via
RunnerCaps.
For now, just enforce that:
- common options related to capabilities are always added, so runners
can't reuse them for different ends
- common options provided for runners which don't support them emit
sensible error messages that should be easy to diagnose and support
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
We can just call super() instead of super(MyClassName, self). The
original extra verbosity is likely due to old habits of mine from
Python 2 which are no longer necessary, but got copy/pasted around.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
For iterable areas defined with Z_STRUCT_SECTION_ITERABLE(),
the corresponding output section in the linker script is just
boilerplate. Add macros to make these definitions simpler.
Unfortunately, we have a fair number of iterable sections not
defined with Z_STRUCT_SECTION_ITERABLE(), this patch does not
address this.
The output sections are all named <struct name>_area, update
sanitylib.py with this.
sys_sem with no userspace, and k_lifo/k_fifo are special cases
where different data types that are all equivalent need to be
put in the same iterable area. Add
Z_STRUCT_SECTION_ITERABLE_ALTERNATE() for this special case.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
The argparse module, by default, complains about non-defined
arguments. This prevents passing arguments prefixed with '-'
or '--' to the target script (e.g. calling another Python
script using argparse). This changes the misc-flasher script
so that any arguments not recognized by west will be passed
to the target script.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
The start_addr attribute of Stm32flashBinaryRunner is an integer. It
must be converted to a string before being concatenated with a colon and
the (already converted to a string) size to erase or write.
Signed-off-by: Grzegorz Szymaszek <gszymaszek@short.pl>
Disables the Debug-Access-Port of the microcontroller after flashing.
If not disabled, the DAP consumes ~1.6mA until the debugger disables it
or a hard power cycle is applied.
The DAP is typically automatically disabled after flashing, but if other
instances of JLink software are running (not connected), it will not be.
The added command resets the value of the CTRL/STAT register of the DAP.
This clears the CSYSPWRUPREQ and CDBGPRWUPREQ bits, leaving the debug
hardware free to power off the appropriate hardware. In no way does it
hinder the ability to later connect to the device for debugging.
This resolves the jlink portion of #26139
Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
Add warning about enabling the options below so that users are
aware that this is a security risk.
- CONFIG_BT_DEBUG_SMP
- CONFIG_BT_DEBUG_KEYS
- CONFIG_BT_OOB_DATA_FIXED
- CONFIG_BT_USE_DEBUG_KEYS
- CONFIG_BT_STORE_DEBUG_KEYS
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
test_testsuite_class.py: Added testcases for load_From_file and
apply_filters functions of Class Testsuite.
conftest.py: Modified class_testsuite fixture to create the
outdir as temporary directory which gets deleted after execution
of testcases.
test_data/sanitycheck.csv: load_from_file function uses this
customized file to load the failed or last run testcases.
test_data/sanitycheck_keyerror.csv: file used by
test_load_from_file function to raise the appropriate error
if there is a keyerror.
scripts/requirements-build-test.txt: added mock & csv python libraries
Signed-off-by: Aastha Grover <aastha.grover@intel.com>
test_testsuite_class.py: Added testcases for add_configurations(),
get_all_testcases(), get_toolchain() and get_platforms() functions
test_data/board_config: board root directory for all board
configurations. This is also to add the platform configurations
in a list which is used for further testing.
conftest.py: Added fixtures for all_testcases_dict (returns a
dictionary of testcases) and platforms_list (returns a list of
all platforms using add_configurations function).
Signed-off-by: Aastha Grover <aastha.grover@intel.com>
Convert the description comment at the top of the file to a
documentation string. That string then maybe viewed with the --help
argument. Also, rework a bit the documentation string: remove usage,
since that is provided by the help message of the argparse module.
Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
MCMUMGR file system management is discouraged in production, just
adding it to the recommendation list.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
Remove Kconfig, linker script, and related bits associated with
CUSTOM_RODATA_LD, CUSTOM_RWDATA_LD, CUSTOM_SECTIONS_LD,
SOC_NOINIT_LD, SOC_RODATA_LD, and SOC_RWDATA_LD options that have been
deprecated since Zephyr 2.2.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Convert the description comment at the top of the file to a
documentation string. That string then maybe viewed with the --help
argument. Also, rework a bit the documentation string.
Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
This commit introduces the possibility of adding dependencies between
Zephyr modules to ensure that a Zephyr module on which other modules
depends is processed first.
The dependency chain is ordered using a topological sort.
This allows to add dependencies to a zephyr/module.yml as:
build:
cmake: .
depends:
- fatfs
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
