Commit Graph

2847 Commits

Author SHA1 Message Date
Watson Zeng
0079cecc19 sanitycheck: add argument --pre-script
add argument --pre-script to specify a pre script. This will be
executed before device handler open serial port and invoke runner.

Currently, pre_script can only get from hardware.map file. just like:
./scripts/sanitycheck  --hardware-map xxx.map
and set pre_script in xxx.map file.

Signed-off-by: Watson Zeng <zhiwei@synopsys.com>
2020-08-31 15:54:32 +02:00
Watson Zeng
3b43d946fd sanitycheck: run custom pre script before open the serial port
In some cases it might be a good idea to reset the board for real to
make sure it is completely recovered from some failed state (simple
re-loading of the application binary or even Elf file contents doesn't
affect most of internal CPU states so doesn't help in recovery,
see https://github.com/zephyrproject-rtos/zephyr/issues/25022 &
https://github.com/zephyrproject-rtos/zephyr/issues/26665).
And so we may want to utilize some external utility which triggers the
hard reset (in case of ARC boards it is
https://github.com/foss-for-synopsys-dwc-arc-processors/rff-ftdi-reset).
So we need to have a way to execute an external command before each and
every test.

Now given we already have quite some call-backs we try to use them
before re-inventing the wheel. And pre_script seem to be a good option
with just on minor note - it is called after serial port gets open.
And while in some cases it might be OK if serial port on the board is
not affected by the board's reset, if it is affected we'll be losing
connection on reset (and that's the case with ARC boards BTW as the
FTDI USB-to-Serial IC is also wired to the reset signal on most of
the boards). That said we just move invocation of pre_script before
opening the serial port and everything should be good now.

Signed-off-by: Watson Zeng <zhiwei@synopsys.com>
2020-08-31 15:54:32 +02:00
Anas Nashif
ad44bed754 sanitycheck: fix footprint reports
By default show reports based on last release. Fix a few other issues
where we had 0 values and were dividing by zero.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-08-31 09:20:52 -04:00
Martí Bolívar
b5ad84afe9 scripts: west_commands: handle EOFError in nrfjprog.py
We should simply exit if the user hits control-D during the prompt.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-08-31 12:56:20 +02:00
Torsten Rasmussen
ab7ec17b86 cmake: zephyr module processing improvements.
This commit improves Zephyr modules support in CMake.
All Zephyr modules will now have a corresponding
`ZEPHYR_${module_name}_MODULE_DIR` set, regardless of whether they
define a cmake folder in module.yml.

This results in:
`ZEPHYR_${module_name}_MODULE_DIR` defined for all modules and refers to
the root of the modle.
`ZEPHYR_${module_name}_CMAKE_DIR` defined for modules that specifies
cmake in module.yml, or is having both a CMakeLists.txt and Kconfig file
in the zephyr folder.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2020-08-31 12:15:02 +02:00
Torsten Rasmussen
25c3f8e12f scripts: zephyr_modules.py cleanup.
This commit has no functional changes but is simply a cleanup so that
flake8 will pass without warnings.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2020-08-31 12:15:02 +02:00
Anas Nashif
9eb9c4cce9 sanitycheck: add option --emulation-only
This will only build/run on emulation platforms.
The decision is made based on the value of the 'simulation' key in the
platform yaml file.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-08-28 14:22:07 -07:00
Øyvind Rønningstad
7012fdf0b7 nrfjprog.py: Fix snr globbing to allow leading 0s
This is needed for sanitycheck hardware maps which take the serial
number directly from USB metadata.

Signed-off-by: Øyvind Rønningstad <oyvind.ronningstad@nordicsemi.no>
2020-08-28 11:38:05 +02:00
Anas Nashif
dca317c730 sanitycheck: inclusive language
change whitelist -> allow.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-08-27 07:04:07 -04:00
Daniel Leung
faae15d838 sanitycheck: allow QEMU to crash without failing
Adds some code to allow QEMU to crash without failing the test.
This is required for testing coredump code as it will certainly
cause QEMU to crash.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2020-08-24 20:28:24 -04:00
Daniel Leung
5b1b4a3755 sanitycheck: need to wait for harness to process QEMU outputs
There is a potential that QEMUHandler.handle() returns before
the testing harness finishes processing all the output from QEMU.
Simply wait for the harness thread to finish before returning.

Also, fix the return code in the debug message as it should be
the return code from Popen().

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2020-08-24 20:28:24 -04:00
Daniel Leung
181d07321f coredump: add support for ARM Cortex-M
This adds the necessary bits in arch code, and Python scripts
to enable coredump support for ARM Cortex-M.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2020-08-24 20:28:24 -04:00
Daniel Leung
8fbb14ef50 coredump: add support for x86 and x86_64
This adds the necessary bits to enable coredump for x86
and x86_64.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2020-08-24 20:28:24 -04:00
Daniel Leung
49206a86ff debug/coredump: add a primitive coredump mechanism
This adds a very primitive coredump mechanism under subsys/debug
where during fatal error, register and memory content can be
dumped to coredump backend. One such backend utilizing log
module for output is included. Once the coredump log is converted
to a binary file, it can be used with the ELF output file as
inputs to an overly simplified implementation of a GDB server.
This GDB server can be attached via the target remote command of
GDB and will be serving register and memory content. This allows
using GDB to examine stack and memory where the fatal error
occurred.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2020-08-24 20:28:24 -04:00
Daniel Leung
203556cd8c sanitycheck: update section names in size calculator
Commit 8d7bb8ffd8 refactored
device structures which changed some of the linker sections:
"devconfig" was removed, and "devices" was added. However,
the list in sanitycheck's size calculator was not updated,
which results in sanitycheck complaining about unrecognized
sections when doing footprint.

Also, a few sections have been renamed (with added suffix
"_area") due to introduction of Z_ITERABLE_SECTION_RAM/ROM
macros. There are also some missing section names.

Fixes the issue by adding the missing sections names, and
updating existing ones.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2020-08-24 20:18:06 -04:00
Anas Nashif
5ab117ffd2 tracing: scripts: add scripts for parsing ctf data
Generate trace using samples/subsys/tracing for example:

    west build -b qemu_x86 samples/subsys/tracing  -t run \
      -- -DCONF_FILE=prj_uart_ctf.conf

    mkdir ctf
    cp build/channel0_0 ctf/
    cp subsys/tracing/ctf/tsdl/metadata ctf/
    ./scripts/tracing/parse_ctf.py -t ctf

1969-12-31 19:00:00.001779 (+0.000000 s): thread_create: thread_a
1969-12-31 19:00:00.001915 (+0.000136 s): thread_info (Stack size: 1024)
1969-12-31 19:00:00.002021 (+0.000107 s): thread_ready: thread_a
1969-12-31 19:00:00.002287 (+0.000265 s): thread_switched_out: main
1969-12-31 19:00:00.002320 (+0.000033 s): thread_switched_in: main
1969-12-31 19:00:00.002585 (+0.000265 s): thread_abort: main
1969-12-31 19:00:00.002730 (+0.000145 s): thread_switched_out: main
1969-12-31 19:00:00.002762 (+0.000032 s): thread_switched_in: thread_a
1969-12-31 19:00:00.002829 (+0.000067 s): thread_create: 1130656
1969-12-31 19:00:00.002860 (+0.000031 s): thread_info (Stack size: 1024)
1969-12-31 19:00:00.002911 (+0.000052 s): thread_ready: 1130656
1969-12-31 19:00:00.003033 (+0.000121 s): thread_name_set
1969-12-31 19:00:00.003132 (+0.000100 s): semaphore_take (1140992)
1969-12-31 19:00:00.003201 (+0.000069 s): end_call 38 (SEMA_TAKE)
1969-12-31 19:00:00.003330 (+0.000128 s): start_call 39 (SLEEP)
1969-12-31 19:00:00.003721 (+0.000391 s): thread_suspend: thread_a
1969-12-31 19:00:00.003802 (+0.000081 s): thread_switched_out: thread_a
1969-12-31 19:00:00.003834 (+0.000032 s): thread_switched_in: thread_b
1969-12-31 19:00:00.003872 (+0.000038 s): semaphore_take (1141016)
1969-12-31 19:00:00.003976 (+0.000103 s): thread_pending: thread_b
1969-12-31 19:00:00.004095 (+0.000119 s): thread_switched_out: thread_b
1969-12-31 19:00:00.004127 (+0.000032 s): thread_switched_in: idle 00
1969-12-31 19:00:00.004232 (+0.000105 s): idle
1969-12-31 19:00:00.510848 (+0.506616 s): isr_enter
1969-12-31 19:00:00.511664 (+0.000816 s): thread_resume: thread_a
1969-12-31 19:00:00.511947 (+0.000283 s): thread_ready: thread_a
1969-12-31 19:00:00.512385 (+0.000438 s): isr_exit
1969-12-31 19:00:00.512570 (+0.000186 s): thread_switched_out: idle 00
1969-12-31 19:00:00.512689 (+0.000119 s): thread_switched_in: thread_a
1969-12-31 19:00:00.512869 (+0.000180 s): end_call 39 (SLEEP)

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-08-24 13:21:12 +02:00
Fabio Utzig
716ab47dcc west: try image-1's size as slot size in imgtool
When MCUBoot is built using the swap-move strategy, the secondary slot
can be one sector smaller than the primary slot, because the primary
slot's last "useful" image sector must be "reserved" for the move
operation; this impacts the generation of padded images, so when
signing an image, the proper size to use is the secondary slot's,
unless a secondary image is not defined which is the case when using
single image DFU.

Signed-off-by: Fabio Utzig <fabio.utzig@nordicsemi.no>
2020-08-24 10:18:56 +02:00
Wenbo Yang
17216128ab hardening: remove the empty line before EOF
The extra empty line broke the "ninja hardenconfig" on my machine with
Python 3.7.5, it complains:

"
... ...
File "/home/zephyrproject/zephyr/scripts/kconfig/hardenconfig.py",
line 46, in compare_with_hardened_conf

name = row[0]
IndexError: list index out of range
FAILED: CMakeFiles/hardenconfig
"

The csv.reader reads this empty line and gets an empty list which will
not be successfully "de-referenced".  Removing it to improve the
out-of-box experience when pepople try out the hardening options.

Signed-off-by: Wenbo Yang <wenbo.yangcn@gmail.com>
2020-08-21 15:09:07 -04:00
Peter A. Bigot
e49e3c8fef scripts: update checkpatch based on current linux kernel version
This squashes and applies the Linux diffs to scripts/checkpatch.pl
between Linux commit 16fbf79b0f83bc75 ("Linux 5.6-rc7") and
9123e3a74ec7b93 ("Linux 5.9-rc1") except for commits identified below.

The last 1000 commits to Zephyr master were compared for checkpatch
output differences between the previous Zephyr version and this
version.  One new diagnostic about function declarations with an empty
parameter-list was introduced (FUNCTION_WITHOUT_ARGS).  The text of an
existing diagnostic was changed (DT_SPLIT_BINDING_PATCH).  The text of
LONG_LINE diagnostics was enhanced to provide the actual line length.

Linux commit dfa05c28ca7ffc0a ("checkpatch: remove email address
comment from email address comparisons") was removed because
differences in the scripts resulted in false signed-off-by check
diagnostics when a full name included characters not in Basic Latin,
due to changes in how the author name was extracted.  Earlier upstream
changes not integrated into Zephyr may be required.

Linux commit b95692f8b3000166 ("checkpatch: prefer fallthrough; over
fallthrough comments") was removed because Zephyr doesn't support the
upstream pseudo keyword.

Linux commit bdc48fa11e46f867 ("checkpatch/coding-style: deprecate
80-column warning") was edited to not actually change the 80-column
maximum line limit as this change has not been mooted for Zephyr.

Linux commit ced69da1db0b57bb ("checkpatch: fix CONST_STRUCT when
const_structs.checkpatch is missing") was edited to the CONST_STRUCT
file as that's not supported in Zephyr.

Signed-off-by: Peter A. Bigot <pab@pabigot.com>
2020-08-21 13:57:02 -04:00
Henrik Brix Andersen
96a39f00c2 west: runners: jlink: add support for -nogui 1 command line parameter
Add support for the J-Link Commander "-NoGui 1" command line parameter
in the West J-Link runner.

This command line parameter suppresses GUI dialogs (except for license
dialogs) in J-Link Commander starting from v6.80.

Signed-off-by: Henrik Brix Andersen <hebad@vestas.com>
2020-08-20 13:04:37 -05:00
Martí Bolívar
2ed2d3bfa9 scripts: runners: add --no-erase support
If a runner supports the --erase option, make it so that the user can
add --no-erase to the command line to explicitly disable it.

Add a diagnostic at info() level whenever this option is requested.

The intent of this commit is to make it more obvious when a mass erase
was requested, especially in situations (like MCUboot with
CONFIG_MCUBOOT_TRY_MASS_ERASE=y) where this option may be turned on by
default.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-08-19 16:38:26 -05:00
Torsten Rasmussen
edde894d04 cmake: Zephyr CMake package clean-up and minor fix
Fixes: #27375

This is a cleanup of the Zephyr CMake package export.
The code has been simplified so that the export now happens through a
CMake script. This avoids several generated CMake build files compared
to previous export mode, and thus removes the need for a CMake pristine
script.

A benefit of this cleanup is that it also fixes #27375.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2020-08-19 12:52:33 -04:00
Maciej Perkowski
c67a0cd098 sanitycheck: Improve tests counting
This patch should improve couting and reporting of the number
of passed/skipped tests.

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2020-08-19 12:49:42 -04:00
Martí Bolívar
112e1d32b2 scripts: west_commands: remove unused helper
The cached_runner_config() helper is no longer used, so remove it.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-08-19 09:51:13 -04:00
Martí Bolívar
7492997ba7 scripts: west sign: clean up and address TODOs
Avoid use of the cached_runner_config() helper, which we have a TODO
item to get rid of. Adjust the output and do some other minor
improvements.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-08-19 09:51:13 -04:00
Martí Bolívar
9ca1de18eb scripts: west_commands: add .config parsing helper
Rolling yet another parser turns out to be the best way to let west
extensions respond to Kconfig values. See source code comments in the
patch for details.

The sanitylib library has some similar functionality but it isn't
exactly the same, and it gets strings wrong. For example, that parser
can't handle this option:

CONFIG_FOO="he said \"no\" to me"

This one can, and it has a couple of other features we'll find useful
for west extensions eventually besides.

(Not to mention that sanitylib also rolled its own CMake cache parser,
which also exists in west_commands.)

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-08-19 09:51:13 -04:00
Anas Nashif
74f4891a8f requirements: add anytree
Needed now by both ram and rom report targets.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-08-14 18:49:26 -04:00
Anas Nashif
239d175eff scripts: size_report: fix path prepending
Fixes potential issue on windows.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-08-14 18:49:26 -04:00
Anas Nashif
ac215a570c size_report: use anytree
Use anytree module to display tree and to allow easy exporting into
json.
Add option to export results into a json file.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-08-14 18:49:26 -04:00
Øyvind Rønningstad
6cc7ea51e4 nrfjprog.py: Allow passing a glob to --snr
You can now do --snr 6*1* to achieve the same as --snr 683010000

Signed-off-by: Øyvind Rønningstad <oyvind.ronningstad@nordicsemi.no>
2020-08-14 13:32:36 +02:00
Øyvind Rønningstad
3b595ebd3e nrfjprog.py: Small refactor of get_board_snr()
Move some functionality out into separate functions for clarity.

Signed-off-by: Øyvind Rønningstad <oyvind.ronningstad@nordicsemi.no>
2020-08-14 13:32:36 +02:00
Wayne Ren
938642cf7f scripts: add the support of DW_AT_count for array type
In DWARF 4, e.g. ARC Metaware toolchain, DW_AT_count is
used not DW_AT_upper_bound. We should consider this corner
case.

Signed-off-by: Wayne Ren <wei.ren@synopsys.com>
2020-08-14 13:30:41 +02:00
Anas Nashif
d548a890b8 ci: do not run sanitycheck on MAINTAINERS.yml changes
Skip sanitycheck if we are only changing MAINTAINERS.yml

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-08-13 17:03:47 +02:00
Kumar Gala
cfc54f38be sanitycheck: set selected_plaforms for --test-only
When running with --test-only we get:

Traceback (most recent call last):
  File "./scripts/sanitycheck", line 1168, in <module>
    main()
  File "./scripts/sanitycheck", line 1160, in main
    options.only_failed)
  File "scripts/sanity_chk/sanitylib.py", line 2543, in save_reports
    self.xunit_report(filename + ".xml", full_report=False, append=only_failed)
  File "scripts/sanity_chk/sanitylib.py", line 3220, in xunit_report
    return fails, passes, errors, skips
UnboundLocalError: local variable 'fails' referenced before assignment

This is due to the fact that selected_platforms was not set.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-08-13 07:45:17 -05:00
Christian Taedcke
b2be804421 sanitycheck: Fix xml testsuite attribute skipped
In the junit output the attribute containing the number of skipped tests
must be named "skipped" instead of "skip".
See e.g. https://github.com/junit-team/junit5/blob/main/platform-tests/
src/test/resources/jenkins-junit.xsd#L95

Signed-off-by: Christian Taedcke <christian.taedcke@lemonbeat.com>
2020-08-13 11:51:19 +02:00
Wenbo Yang
999290278e hardening: Deal with empty lines in csv file
The extra empty line broke the "ninja hardenconfig" on my machine with
Python 3.7.5, it complains:

"
... ...
File "/home/zephyrproject/zephyr/scripts/kconfig/hardenconfig.py",
line 46, in compare_with_hardened_conf

name = row[0]
IndexError: list index out of range
FAILED: CMakeFiles/hardenconfig
"

The csv.reader reads this empty line and gets an empty list which will
not be successfully "de-referenced". Adding extra check to skip the
empty lines.

Signed-off-by: Wenbo Yang <wenbo.yangcn@gmail.com>
2020-08-12 16:43:18 -04:00
Peter Bigot
78f36fa534 python3: add check_compliance package requirements
Running check_compliance on a PR before submitting it can avert
embarrassing mistakes.  Ensure the packages needed to do so are
installed along with all the others.

Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
2020-08-11 14:49:48 -04:00
Marcin Niestroj
2652dc71a3 sanitycheck: fix --coverage-tool gcovr
Currently '--coverage-tool gcovr' results in using lcov. Fix that to use
gcovr, as requested by user.

Fixes: f6462a3a8c ("sanitycheck: get rid of global VERBOSE")
Signed-off-by: Marcin Niestroj <m.niestroj@grinn-global.com>
2020-08-11 08:36:39 -04:00
Maciej Perkowski
e3ff4cfcd6 sanitycheck: tests: Fill results for skipped tests
Sanitycheck discards test instances if a set of given coditions
is not fulfilled. This leads to empty test results for these
instances. This can introduce ambiguity with results that are
missing due to some bugs in the framework. This commit fills
the results for skipped tests with 'skipped' states and provides
the reason for filtering them out in the msg field. The commit
also fixes the way sanitycheck counts and reports tests/test cases

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2020-08-06 11:49:39 +02:00
Torsten Rasmussen
25d57ba5af modules: adding support for ROOTs definitions in zephyr/module.yml
Fixes: #25215

This commit introduces the possibility of defining ROOTs in a Zephyr
module and have it automatically appended to list of other ROOTs.
Supported with this commit:
- BOARD_ROOT
- SOC_ROOT
- DTS_ROOT
- ARCH_ROOT

In order to support this in Zephyr module files, the detection of west
has been moved to dedicated west.cmake file and included immediately
after python.cmake.

Also the inclusion of zephyr_modules.cmake has moved before first use
of BOARD_ROOT.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2020-08-05 08:06:07 -04:00
Torsten Rasmussen
5f7cc8ded9 cmake: kconfig: support for multiple SOC_ROOT
This commit introduces support for multiple SOC_ROOT.

This means that additional SOC_ROOTs specified using -DSOC_ROOT as
argument to CMake will be forming a list together with ${ZEPHYR_BASE}.

This allows for greater flexibility, as developers can now specify
multiple out-of-tree SoCs and not worry about the SoC used for the
board they compile for.

Also it avoid code, such as:
if(BOARD STREQUAL my_board_using_out_of_tree_soc)
  set(SOC_ROOT some/out/of/tree/soc/path)
endif()
in application CMakeLists.txt.

Finally, allowing multiple SOC_ROOTs prepares for specifying SOC_ROOTs
in Zephyr modules.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2020-08-05 08:06:07 -04:00
Øyvind Rønningstad
f72aef15d2 sanitycheck: Make --hardware-map and --west-flash together
When passing a using --hardware-map, sanitycheck would ignore extra
args passed via --west-flash, such as --west-flash="--erase".

This is because the command with the extra args was overwritten by the
command with the runner info from the hardware map.

This patch merges those code paths so they are aware of each other and
of --west-runner.

Signed-off-by: Øyvind Rønningstad <oyvind.ronningstad@nordicsemi.no>
2020-08-03 20:40:50 -04:00
Maciej Perkowski
e7f4074799 sanitycheck: Modify expected_subcases
Before sanitycehck was removing all `test_` chars in test cases'
names. This test has to be modify to work with the improved behavior

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2020-08-03 19:46:15 -04:00
Maciej Perkowski
034d4f2156 sanitycheck: Fix test cases names
This commit limits the removal of `test_` from test case name
to only the first occurance. There are test cases with `test_`
also in the middle of their names and removing it couses mismatches
between extracted testcases and the names obtained when passing

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2020-08-03 19:46:15 -04:00
Andrei Emeltchenko
fb44188ebb west: sign: Move imgtool specific docs to appropriate place
Move documentation specific to imgtool to appropriate place.

Signed-off-by: Andrei Emeltchenko <andrei.emeltchenko@intel.com>
2020-08-03 13:49:17 +02:00
Andrei Emeltchenko
51182ab881 west: sign: Add rimage signing tool documentation
Add documentation specific to rimage signing tool.

Fixes #26596

Signed-off-by: Andrei Emeltchenko <andrei.emeltchenko@intel.com>
2020-08-03 13:49:17 +02:00
Andrew Boie
8ce260d8df kernel: introduce supervisor-only stacks
These stacks are appropriate for threads that run purely in
supervisor mode, and also as stacks for interrupt and exception
handling.

Two new arch defines are introduced:

- ARCH_KERNEL_STACK_GUARD_SIZE
- ARCH_KERNEL_STACK_OBJ_ALIGN

New public declaration macros:

- K_KERNEL_STACK_RESERVED
- K_KERNEL_STACK_EXTERN
- K_KERNEL_STACK_DEFINE
- K_KERNEL_STACK_ARRAY_DEFINE
- K_KERNEL_STACK_MEMBER
- K_KERNEL_STACK_SIZEOF

If user mode is not enabled, K_KERNEL_STACK_* and K_THREAD_STACK_*
are equivalent.

Separately generated privilege elevation stacks are now declared
like kernel stacks, removing the need for K_PRIVILEGE_STACK_ALIGN.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-07-30 21:11:14 -04:00
Kumar Gala
33056e48c5 ci: make test_file.txt a build artifact for debug
Rather that echo'ng the test_file.txt to the console, lets just upload
it as a buildkite artifact.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-07-30 11:25:25 -04:00
Maciej Perkowski
732ab5311d doc: Fix path to requirements
Fix path to requirements

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2020-07-30 16:40:29 +02:00
Anas Nashif
555fc6dbd5 sanitycheck: fix only_tags usage
We have been doing an AND comparison instead of an OR. AND does exclude
way to many testcases where multiple tags are being used.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-07-30 10:30:23 -04:00
Anas Nashif
314b8af797 ci: enable sanitycheck for integration
Enable sample for integration.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-07-30 08:00:03 -04:00
Anas Nashif
1636c316d9 sanitycheck: introduce integration testing
Use --integration testing for CI to limit execution and builds on
platforms that actually provide most of the coverage instead of blindly
building/running on all available platforms.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-07-30 08:00:03 -04:00
Anas Nashif
1bf1d19a42 sanitycheck: realign schema
Just a cleanup and alignment of the yaml file.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-07-30 08:00:03 -04:00
Simon Guinot
2dad0b4500 scripts/requirements: add lpc_checksum script
On some NXP LPC MCUs (such as LPC11U6x), a valid firmware image must
include a checksum in the 7th entry of the ARM Cortex-M0 exception
vector table. This checksum is verified by the bootloader.

The lpc_checksum.py python script allows to insert this checksum into
the firmware images (BIN and HEX formats).

Project page: https://github.com/basilfx/lpc_checksum

Signed-off-by: Simon Guinot <simon.guinot@seagate.com>
2020-07-29 20:12:24 +02:00
Jukka Rissanen
bfa08cd04d userspace: Make network interface a kernel object
Make net_if a kernel object with type K_OBJ_NET_IF so that we
can restrict access to it.

Signed-off-by: Jukka Rissanen <jukka.rissanen@linux.intel.com>
2020-07-29 08:13:22 -04:00
Anas Nashif
f8c024a22f sanitycheck: remove verbose log message
This was added for debugging and was left in the script by mistake.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-07-27 19:03:10 +02:00
Anas Nashif
1c2f127e3f sanitycheck: do not duplicate code in tests
Fixed tests to use library code instead of duplicating logic in the test
itself.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-07-27 11:58:32 +02:00
Anas Nashif
9e1be4c05e sanitycheck: do not expect results from build_only instances
instances that do not run will have no results (beside the fact they
built successfully), so log those as such.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-07-27 11:58:32 +02:00
Martí Bolívar
469f53c210 scripts: sanitycheck: import edtlib
This is a cargo-culted attempt to make an error observed in poorly
understood CI circumstances go away when loading edt.pickle, by making
edtlib visible in sys.modules before loading the pickle file.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-07-24 17:58:03 -04:00
Wentong Wu
517633c101 scripts: sanitylib.py: add exception check for psutil
Add exception check for psutil when running CI with QEMU.

Signed-off-by: Wentong Wu <wentong.wu@intel.com>
2020-07-24 14:26:33 -04:00
Anas Nashif
8d8875b752 scripts: add maintainer script
Adding the get_maintainer.py script

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
2020-07-24 17:07:48 +02:00
Kumar Gala
feeda05323 ci: Add a debug out of what tests we are going to build
Add a cat of the test_file.txt so we can see exactly what tests are
expected to be built and run.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-07-24 12:10:15 +02:00
Martí Bolívar
9861e5d22a sanitylib: log test exception tracebacks
This aids in debugging.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-07-23 13:36:22 -05:00
Martí Bolívar
9c92baa73f devicetree: use edt.pickle more
Consolidate creation of edtlib.EDT objects from a build directory's
devicetree into one place by loading it from build/zephyr/edt.pickle
everywhere. A previous commit creates edt.pickle from gen_defines.py.

In addition to probably speeding things up slightly by not reparsing
the devicetree, the main benefit of this approach is creating a single
point of truth for the bindings directories and warnings
configuration, meaning we don't have to worry about them getting out
of sync while being passed around between devicetree creation and
usage time.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-07-23 09:19:45 -05:00
Jan Van Winkel
716eea0b65 scripts: Increased minimal pyelftools version
Increased minimal pyelftools version to 0.26 else
scripts/footprint/size_report will fail to import LocationExpr from
elftools.dwarf.locationlists

Signed-off-by: Jan Van Winkel <jan.van_winkel@dxplore.eu>
2020-07-23 07:49:35 -04:00
Martí Bolívar
006319f043 scripts: kconfigfunctions: add a comment
I can never remember what the contents of the kconfigfunctions dict is
and have to look it up every time. I suspect I'm not alone, and not
everyone who uses these will know how to look up the answer. Add a
comment explaining the value.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-07-23 11:00:39 +02:00
Martí Bolívar
24677f93bc kconfig: add dt_path_enabled, dt_alias_enabled
There is already a way to check if a node is enabled based on its node
label, but we don't have an equivalent way to do that for a path or an
alias. Add them. These rely on the same underlying edtlib machinery to
get their jobs done, but are being treated differently within Kconfig
in order to match distinctions between paths and aliases made in the
devicetree.h API.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-07-23 11:00:39 +02:00
Luiz Augusto von Dentz
75f47a56b5 Bluetooth: Add config option to disable security checks
This adds CONFIG_BT_CONN_DISABLE_SECURITY which can be used to disable
security checks for incoming requests enabling to test accessing GATT
attributes and L2CAP channels that would otherwise require
encryption/authentication in order to be accessed.

It depends on BT_TESTING to indicate to the users that this is a
testing feature which shall not be used in production.

Signed-off-by: Luiz Augusto von Dentz <luiz.von.dentz@intel.com>
2020-07-23 11:00:04 +02:00
Maureen Helm
8c7cef6da0 runners: Implement --erase option in pyocd runner
Implements the common runner option to mass erase before flashing in the
pyocd runner.

Signed-off-by: Maureen Helm <maureen.helm@nxp.com>
2020-07-23 10:30:47 +02:00
Anas Nashif
e47866c530 sanitycheck: fix --device-testing without serial-pty
serial-pty is an extension that is not supported in the hardware map, so
we need to work around the fact that it is not part of the map when
dealing with normal serial.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-07-22 16:57:14 -05:00
Andrei Emeltchenko
d8b845be4e sanitycheck: Add serial_pty script command line option
Add option to provide to sanitycheck argument for creating
pseudoterminal. Used with hardware without serial console connected.
A pseudoterminal is used to make a sanitycheck believe that it
interacts with a terminal although it actually interacts with the
script.

E.g "sanitycheck --device-testing --device-serial-pty <script>"

Signed-off-by: Andrei Emeltchenko <andrei.emeltchenko@intel.com>
2020-07-21 20:19:33 -04:00
Anas Nashif
e8e367ada4 sanitycheck: support only_tags for boards
Support running/building only specific tags and ignoring everything
else.

Adding this to a platform yaml file will enable tests with one of of
tags listed to be executed on the platform.

This is useful for special platform configurations used for testing
specific features for example.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-07-20 17:44:16 -04:00
Ruslan Mstoi
6f26ca0b60 scripts: tests: Add tests for subfolder_list.py
This commit implements tests of subfolder_list.py

Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
2020-07-20 17:31:53 -04:00
Ruslan Mstoi
ab20641054 scripts: subfolder_list.py: refactor the code
Refactor subfolder_list.py to enable unit testing. This means split the
implementation into separate logical functions. Additionally, modify the
code to improve PEP 8 compatibility and pylint result.

Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
2020-07-20 17:31:53 -04:00
Carles Cufi
02dea9279b scripts: modules: Fix initialization of variable
The projects variable needs to be initialized with the list of modules
provided via cmd-line arguments.

This is a regression introduced by
ef3c5e5516.

Fixes #26948.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2020-07-20 22:39:09 +02:00
Anas Nashif
5092541e2b sanitycheck: disable erroring on warnings
--warnings-as-errors was basically doing nothing, it is the default.
Replace this with an option to disable erroring on warning:

 -W, --disable-warnings-as-errors

Fixes #26910

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-07-17 13:03:36 -05:00
Anas Nashif
06052927be sanitycheck: capture fails correctly on qemu crashes
We still have cases where qemu crashes are not being correctly captured
in the on-screen summary, however they are captured in the final
generated report.

Fixes #26679

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-07-16 15:44:17 -04:00
Daniel Leung
ba5f627815 scripts: size_report: rework to use pyelftools...
...instead of GNU binutils to extract DWARF information.
This is now a bit more portable across OS and toolchain.
One bouns is that this nows with on qemu_x86_64.

A few differences are:
() z_mrsh_* which are aliases to handler_no_syscalls() are now
   dropped as they are mapped to the same address, so that they
   are not counted multiple times.
() Static functions and variables with same names should now be
   attributed to the correct source files instead of being
   accumulated into the same symbol of one file (e.g. multiple
   thread_entry() in kernel tests).
() The totals for ROM and RAM are calculated from the
   corresponding sections. Previous script includes the debug
   sections as total ROM size which is not entirely correct.

Fixes #22996

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2020-07-16 12:14:22 +02:00
Spoorthy Priya Yerabolu
0e599be5fc sanitycheck: Updating README file
Updated readme with coverage commands and
organization of tests

Signed-off-by: Spoorthy Priya Yerabolu <spoorthy.priya.yerabolu@intel.com>
2020-07-09 16:03:57 -05:00
Anas Nashif
dc43c298a2 sanitycheck: do not retry build errors with --retry-failed
restore how --only-failed works by allow rebuilds in case of build
errors, however, do not rebuild when --retry-failed is used, which is a
CI usecase and nothing would change in the second iteration.

Fixes #26685

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-07-09 16:46:24 -04:00
Stephanos Ioannidis
3322489d22 config: Rename TEXT_SECTION_OFFSET to ROM_START_OFFSET
The `TEXT_SECTION_OFFSET` symbol is used to specify the offset between
the beginning of the ROM area and the address of the first ROM section.

This commit renames `TEXT_SECTION_OFFSET` to `ROM_START_OFFSET` because
the first ROM section is not always the `.text` section.

Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
2020-07-09 14:02:38 -04:00
Christian Taedcke
3dbe9f2960 sanitycheck: native: Add option to enable UBSAN
Add option for native platform to enable undefined behaviour sanitizer.

Signed-off-by: Christian Taedcke <christian.taedcke@lemonbeat.com>
2020-07-09 07:49:57 -04:00
Aastha Grover
cb0f65e410 requirements-build-test.txt: Updating mock module version
Updating mock module version to >=4.0.1 as sanitycheck testsuite
might fail in some systems due to old version.

Signed-off-by: Aastha Grover <aastha.grover@intel.com>
2020-07-09 06:26:28 -04:00
Anas Nashif
b8c7de6d9a actions: update daily version for testing
Update versions.json with the daily commit that device testing will be
performed on.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-07-09 11:29:56 +02:00
Torsten Rasmussen
ef3c5e5516 cmake: west: invoke west using same python as rest of build system
When running CMake, then Python3 will be used.
This is detected through FindPython3, with a preference for using the
python or python3 in path, if any of those matches the required Python
minimal version in Zephyr.

It is also possible for users to specify a different Python, as example
by using:
`cmake -DPYTHON_PREFER=/usr/bin/python3.x`

However, when running `west` as native command, then west will be
invoked on linux based on the python defined in:
`west` launcher, which could be: `#!/usr/bin/python3.y`

Thus there could be mismatch in Pythons used for `west` and the python
used for other scripts.

This is even worse on windows, where a user might experience:
```
>.\opt\bin\Scripts\west.exe --version
Traceback (most recent call last):
  File "C:\Python37\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  ...
  File "C:\Python37\lib\socket.py", line 49, in <module>
    import _socket
ImportError: Module use of python38.dll conflicts with this version of
Python.
```

when testing out a newer Python, but the python in path is still a 3.7.

By importing `west` into zephyr_module.py and by using, as example
`python -c "from west.util import west_topdir; print(topdir())"`
we ensure the same python is used in all python scripts.

Also it allows the user to control the python to use for west.

It also ensures that the west version being tested, is also the version
being used, where old code would test the version imported by python,
but using the west in path (which could be a different version)

If the west version installed in the current Python, and west invocation
is using a different Python interpreter, then an additional help text
is printed, to easier assist users with debugging.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2020-07-08 23:23:04 -04:00
Kumar Gala
4e2988deeb edtlib: Fold _binding_compat into _init_compat2binding
We only have one use of _binding_compat and it doesn't need self, so
just fold it into _init_compat2binding.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-07-08 22:03:24 -04:00
Kumar Gala
6bf761fc0a dts: Remove support for deprecated DTS binding syntax
We deprecated a number of aspects of the DTS binding syntax in Zephyr
2.1.  Remove the support for the deprecated syntax.  Remove from docs
about the deprecated syntax as well.

Removed reference in release-notes-2.1.rst to legacy_binding_syntax
since that anchor doesn't exist anymore.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-07-08 22:03:24 -04:00
Martí Bolívar
d8698cbdc6 sanitylib: pacify pylint
According to our compliance checks, we have 'useless-suppression'
and 'unused-import' on our hands:

W0611: Unused CLoader imported from yaml as Loader
W0611: Unused Loader imported from yaml
I0021: Useless suppression of 'unsubscriptable-object'

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-07-08 20:19:02 -04:00
Martí Bolívar
148542c080 build: remove dead stores to environment
Now that kconfigfunctions.py is loading the EDT object from a pickle
file, I can't find any other reads of the environment variables it was
previously using to do that.

(The CMake variables with the same names are read in different places;
I'm specifically talking about environment variables here.)

Remove the dead stores.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-07-08 14:02:44 -05:00
Martí Bolívar
269f350487 kconfig: load edt from edt.pickle
This saves time and avoids the need to pass additional state around in
the environment to recreate the edt exactly.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-07-08 14:02:44 -05:00
Martí Bolívar
533f451e7e dts: marshal the EDT object for later use
We need to save and restore the devicetree data to generate optimized
dependency information later on in the build, in particular during the
final application link.

Make this happen by pickling the EDT object in BUILD_DIR/edt.pickle.

The existence of this file is an implementation detail, so do not add
it to the documentation.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-07-08 14:02:44 -05:00
Martí Bolívar
e76b7205ef dts: edtlib: allow pickling/unpickling EDT objects
We have a use case for saving the EDT object to be able to open it up
again later. It would be convenient to be able to do this with the
pickle module from stdlib.

The only thing stopping us from doing that appears to be the open
reference to sys.stderr that's held the edt object even after
EDT.__init__ exits. However, there doesn't seem to be a need to keep
holding on to this object, and in fact it would be a little bit nicer
to drop the reference in case something else (even in the same Python
process that created it originally) wants the EDT object around, but
might want the warn file closed if its refcount zeroes out.

Just drop the reference at the end of __init__ and make EDT._warn()
throw an exception if it's attempted to be used after the constructor
exits.

Make pickle-ability an API guarantee so we can treat any regressions
as bugs going forward.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-07-08 14:02:44 -05:00
Martí Bolívar
e05c94e334 dts: edtlib: fix reg / unit addr warn location
I can't see a good reason to be doing this in the Node class's
unit_addr accessor. Move it up to the edtlib initialization so it only
happens once.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-07-08 14:02:44 -05:00
Anas Nashif
ae61b7edd6 sanitycheck: Use the C LibYAML parser if available
Use the C LibYAML parser if available, rather than the Python parser. It
is much faster.

This is a clean and rebased version of PR #20210 by
Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>

Co-authored-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-07-08 13:34:45 -05:00
Anas Nashif
869ca055c5 sanitycheck: handle qemu crashes
Deal with qemu crashes and do not register pass status as the default.
In case we do not have a status, report unknown.

This now captures different states from qemu that were not captured
before.

Fixes #26679

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-07-07 16:04:32 -07:00
Erwan Gouriou
2339fa00f3 scripts: sanitylib.py: Add support for STLINK-V3 probe
Recent ST boards embed the new ST-Link probe V3.
It is advertised as "STLINK-V3", update sanitylmib to take it
into account.
In handle function, it is proposed to treat it separately as current
"STM32 STLink" as processing might differ in next future (hla_serial
deprecation).

Signed-off-by: Erwan Gouriou <erwan.gouriou@linaro.org>
2020-07-07 16:41:09 -04:00
Øyvind Rønningstad
4813f469f0 sanitycheck: Improve handling of duplicate devices
Some devices have multiple serial ports, which show as duplicates in the
hardware map. This doesn't work well when using --generate-hardware-map
to regenerate an existing map. E.g. nrf5340pdk_nrf5340_cpuapp by default
prints to the 3rd of 3 devices. If it shows up on /dev/ttyACM0,
/dev/ttyACM1, and /dev/ttyACM2, printout will come on /dev/ttyACM2.
nrf9160dk_nrf9160 also provides 3 devices, but prints to the 1st.

This patch sorts the devices by the serial port and matches multiple
duplicate entries one-to-one to retain the ordering. This way, the
correct device can be given the platform name and the others can be kept
as "unknown" so that --device-testing understands correctly
1) that there is only one device (not 3)
2) where the serial output will come.

Signed-off-by: Øyvind Rønningstad <oyvind.ronningstad@nordicsemi.no>
2020-07-06 10:27:22 -04:00
Aastha Grover
8213a1509f sanitycheck: Testcases for report functions of TestSuite Class
test_reporting_testsuite.py: Adding testcases for reporting
functions of Class testSuite.
test_testsuite_class.py: Testcase for add_instances function
conftest.py: Added fixture for instances_fixture

Signed-off-by: Aastha Grover <aastha.grover@intel.com>
2020-07-03 12:20:35 -04:00
Aastha Grover
dcbd915759 sanitylib: Fix discard_report to check self.discards dict
Modifying discard_report function to fix the check for
self.discards dictionary in try block. self.discards
is None is dead code which didnot check if the dictionary was
empty. Changed it to if not self.discards.

Signed-off-by: Aastha Grover <aastha.grover@intel.com>
2020-07-03 12:20:35 -04:00
Anas Nashif
3cb1b437d2 ci: do not run sanitycheck on linter config changes
Do not run sanitycheck on linter configuration changes.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-07-03 07:58:31 -04:00
Anas Nashif
f04461e8d0 sanitycheck: fix --failed-only handling
- Report build errors as errors, not test failures
- Do not try and build/run tests with build failures
- Fix issue with empty reports when running --only-failed
- Report build errors in the detailed and target reports

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-07-03 07:25:49 -04:00
Michael Hope
bb8277962d scripts: bossac: automatically pull the flash offset from the config
BOSSAC uses a bootloader, so pull the flash address from DeviceTree or
the config and use if the version of BOSSAC supports offsets.

Signed-off-by: Michael Hope <mlhx@google.com>
2020-07-02 08:23:03 -04:00
Michael Hope
4ac195e0af scripts: bossac: add support for bossac 1.9.1
BOSSA 1.7 has built-in bootloader offset handling while 1.9.1
requires the user to supply the offset.  Add support for both by
sniffing the help output and warn the user if the flags needs adding.

Related to #22062

Signed-off-by: Michael Hope <mlhx@google.com>
2020-07-02 08:23:03 -04:00
Torsten Rasmussen
13642fe013 cmake: west: west flash dependencies in sync with CMake flash target
This commit creates a list of a phony targets for each runner, that is:
`west_flash_depends`, `west_debug_depends`, and so on.
Those targets has identical dependencies as CMake runner target.
flash, debug, debugserver, attach targets.

As example `ninja flash` correctly ensures dependencies are taken into
consideration before calling `west flash`.

Unfortunately, calling `west flash` directly would not re-run the flash
dependencies, cause `west flash` would only build the default CMake
target.

Now, `west flash` calls the phony `west_flash_depends` target, ensuring
all deps are up-to-date before flashing (unless --skip-rebuild is given)

The same is true for the other mentioned runners.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2020-07-01 07:58:10 -04:00
Anas Nashif
b5973930b1 sanitycheck: set selected_plaforms when loading tests from file
Missed the case we use in CI where we pre-generate the testcase list and
load it, this was resulting in empty test reports in CI.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-06-30 18:34:01 -04:00
Spoorthy Priya Yerabolu
473ed3412f scripts: tests: sanitycheck: Add testcases for TestCase class
test_testinstance.py: Add testcases to scan file and path for
sub testcases
test_data/testcases: Also added the ztest test files
test_testsuite_class.py: changed get_all_tests() to match
count of sub testcases in ztest files

Signed-off-by: Spoorthy Priya Yerabolu <spoorthy.priya.yerabolu@intel.com>
2020-06-29 13:15:00 -04:00
Spoorthy Priya Yerabolu
ad4d4fc7d1 scripts: tests: sanitycheck: changes to sanitylib to sort the warnings
While testing the warnings in scan_file() format is not the same
everytime we run tests
Added sorted with reverse flag so that warnings are same for
every run

Signed-off-by: Spoorthy Priya Yerabolu <spoorthy.priya.yerabolu@intel.com>
2020-06-29 13:15:00 -04:00
Anas Nashif
33c21e1f7d ci: do not skip BT simulator tests
BT simulator tests run standalone and independent of sanitycheck and
might be triggered by tests that are not managed by sanitycheck, so
leave it alone.

Fixes #26508

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-06-29 12:16:37 -04:00
Anas Nashif
5f200ee8bf ci: do not run sanitycheck on action changes
changes to actions should not trigger sanitycheck.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-06-29 10:27:57 -04:00
Martí Bolívar
6e4c2b9be9 scripts: clean up west build/flash/debug help
Just changes to the west help output; no functional changes expected.

Make option descriptions lowercase to match the argparse module's
conventions. When multiple sentences are required, move them to parser
prolog/epilog or argument group description sections.

Clarify some points that have confused multiple people.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-06-29 08:47:04 -04:00
Kumar Gala
8ca5691a46 sanitycheck: sanitylib: Fix bare --west-flash option
If we pass --west-flash with no additional argument that was getting
treated the same as if it wasn't passed at all.

Change the check to be based on 'None' to handle the bare --west-flash
case properly.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-06-28 23:05:36 -04:00
Anas Nashif
a53c813ac5 sanitycheck: reorg junit output for nicer reports
classname in the junit report was not being used correctly since the
beginning. We had the targetname in the classname and put all results in
1 single testsuite.

The new output add 1 single testsuite for each platform and simplifies
the classname so that tests of the same classname can be grouped in
output, producing readable reports.

The current output can be seen here:

https://testing.zephyrproject.org/daily_tests/zephyr-v2.2.0-2533-gdbbf834605/report/index.html

This should much better once we have this change in.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-06-25 22:36:17 -05:00
Wentong Wu
6fae53c044 sanitycheck: handle timeouts correctly with qemu
In some cases we were not handling return code correctly and evaluating
timeouts as a pass. Report failure correctly.

Fixes #26065

Signed-off-by: Wentong Wu <wentong.wu@intel.com>
2020-06-25 11:05:29 -04:00
Martí Bolívar
bd491fbb97 runners: promote --erase to common option
Move the --erase option to core.py from nrfjprog.py and jlink.py,
where it is currently supported.

Using the RunnerCaps option enforcement mechanism introduced earlier,
enforce that it will only be given to runners that support it.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-06-25 12:14:25 +02:00
Martí Bolívar
f8e8e9229d runners: enforce RunnerCaps via create() indirection
Require all implementations to provide a do_create(), a new
ZephyrBinaryRunner abstract class method, and make create() itself
concrete.

This allows us to enforce common conventions related to individual
runner capabilities as each runner provides to the core via
RunnerCaps.

For now, just enforce that:

- common options related to capabilities are always added, so runners
  can't reuse them for different ends

- common options provided for runners which don't support them emit
  sensible error messages that should be easy to diagnose and support

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-06-25 12:14:25 +02:00
Martí Bolívar
7b93bd54d4 scripts: runners: clean up super() calls
We can just call super() instead of super(MyClassName, self). The
original extra verbosity is likely due to old habits of mine from
Python 2 which are no longer necessary, but got copy/pasted around.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-06-25 12:14:25 +02:00
Andrew Boie
45979dafb4 linker: add iterable section macros
For iterable areas defined with Z_STRUCT_SECTION_ITERABLE(),
the corresponding output section in the linker script is just
boilerplate. Add macros to make these definitions simpler.

Unfortunately, we have a fair number of iterable sections not
defined with Z_STRUCT_SECTION_ITERABLE(), this patch does not
address this.

The output sections are all named <struct name>_area, update
sanitylib.py with this.

sys_sem with no userspace, and k_lifo/k_fifo are special cases
where different data types that are all equivalent need to be
put in the same iterable area. Add
Z_STRUCT_SECTION_ITERABLE_ALTERNATE() for this special case.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-06-24 17:09:22 -04:00
Daniel Leung
2959ed35c1 west: runners/misc-flasher: fix passing extra args to script
The argparse module, by default, complains about non-defined
arguments. This prevents passing arguments prefixed with '-'
or '--' to the target script (e.g. calling another Python
script using argparse). This changes the misc-flasher script
so that any arguments not recognized by west will be passed
to the target script.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2020-06-24 08:51:53 -04:00
Grzegorz Szymaszek
0a3e3958df west: runners: stm32flash: convert start_addr to string (concatenation)
The start_addr attribute of Stm32flashBinaryRunner is an integer. It
must be converted to a string before being concatenated with a colon and
the (already converted to a string) size to erase or write.

Signed-off-by: Grzegorz Szymaszek <gszymaszek@short.pl>
2020-06-19 15:18:08 +02:00
Jordan Yates
9c39a93014 west: runners: jlink: Disable DAP after flashing
Disables the Debug-Access-Port of the microcontroller after flashing.
If not disabled, the DAP consumes ~1.6mA until the debugger disables it
or a hard power cycle is applied.

The DAP is typically automatically disabled after flashing, but if other
instances of JLink software are running (not connected), it will not be.

The added command resets the value of the CTRL/STAT register of the DAP.
This clears the CSYSPWRUPREQ and CDBGPRWUPREQ bits, leaving the debug
hardware free to power off the appropriate hardware. In no way does it
hinder the ability to later connect to the device for debugging.

This resolves the jlink portion of #26139

Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
2020-06-17 19:26:54 +02:00
Carles Cufi
88f25df9ac Bluetooth: smp: Warn on enabling several Bluetooth options
Add warning about enabling the options below so that users are
aware that this is a security risk.

- CONFIG_BT_DEBUG_SMP
- CONFIG_BT_DEBUG_KEYS
- CONFIG_BT_OOB_DATA_FIXED
- CONFIG_BT_USE_DEBUG_KEYS
- CONFIG_BT_STORE_DEBUG_KEYS

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2020-06-17 17:14:33 +03:00
Aastha Grover
cf72fe8fe9 tests: sanitycheck: Testcases for load_from_file and apply_filters
test_testsuite_class.py: Added testcases for load_From_file and
apply_filters functions of Class Testsuite.
conftest.py: Modified class_testsuite fixture to create the
outdir as temporary directory which gets deleted after execution
of testcases.
test_data/sanitycheck.csv: load_from_file function uses this
customized file to load the failed or last run testcases.
test_data/sanitycheck_keyerror.csv: file used by
test_load_from_file function to raise the appropriate error
if there is a keyerror.
scripts/requirements-build-test.txt: added mock & csv python libraries

Signed-off-by: Aastha Grover <aastha.grover@intel.com>
2020-06-15 22:27:18 -04:00
Aastha Grover
27435ce810 scripts: tests: sanitycheck: Add more testcases for Class TestSuite
test_testsuite_class.py: Added testcases for add_configurations(),
get_all_testcases(), get_toolchain() and get_platforms() functions
test_data/board_config: board root directory for all board
configurations. This is also to add the platform configurations
in a list which is used for further testing.
conftest.py: Added fixtures for all_testcases_dict (returns a
dictionary of testcases) and platforms_list (returns a list of
all platforms using add_configurations function).

Signed-off-by: Aastha Grover <aastha.grover@intel.com>
2020-06-15 22:27:18 -04:00
Ruslan Mstoi
e406c9c448 scripts: gen_gcov_files.py: add argument help text
Convert the description comment at the top of the file to a
documentation string. That string then maybe viewed with the --help
argument. Also, rework a bit the documentation string: remove usage,
since that is provided by the help message of the argparse module.

Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
2020-06-13 16:33:22 -04:00
Ruslan Mstoi
69b07e3156 scripts: gen_gcov_files.py: PEP 8 style fixes
Fix PEP 8 coding style issues reported by pycodestyle.

Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
2020-06-13 16:33:22 -04:00
Anas Nashif
adb6a89be5 sanitycheck: capture non-ztest results
We were missing test results from non-ztest tests in the target reports.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-06-12 11:50:26 -04:00
Anas Nashif
288ae28c13 ci: add compliance checking via GH actions
Move check_compliance script to main tree and adapt/use with GH actions.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-06-12 07:53:09 -04:00
Flavio Ceolin
b429b12609 hardening: Add MCUMGR_CMD_FS_MGMT recommendation
MCMUMGR file system management is discouraged in production, just
adding it to the recommendation list.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2020-06-12 11:15:24 +02:00
Kumar Gala
9ca541a504 linker: Remove deprecated Kconfig options related to linker scripts
Remove Kconfig, linker script, and related bits associated with
CUSTOM_RODATA_LD, CUSTOM_RWDATA_LD, CUSTOM_SECTIONS_LD,
SOC_NOINIT_LD, SOC_RODATA_LD, and SOC_RWDATA_LD options that have been
deprecated since Zephyr 2.2.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-06-12 11:14:28 +02:00
Ruslan Mstoi
a8e0655833 scripts: file2hex.py: PEP 8 style fix
Fix pycodestyle PEP 8 issue:

E127 continuation line over-indented for visual indent

Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
2020-06-12 11:09:58 +02:00
Ruslan Mstoi
1bae76770b scripts: file2hex.py: add argument help text
Convert the description comment at the top of the file to a
documentation string. That string then maybe viewed with the --help
argument. Also, rework a bit the documentation string.

Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
2020-06-12 11:09:58 +02:00
Torsten Rasmussen
3917ee51a9 scripts: zephyr modules: introducing Zephyr module dependency handling
This commit introduces the possibility of adding dependencies between
Zephyr modules to ensure that a Zephyr module on which other modules
depends is processed first.

The dependency chain is ordered using a topological sort.

This allows to add dependencies to a zephyr/module.yml as:

build:
  cmake: .
  depends:
    - fatfs

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2020-06-12 11:07:58 +02:00
Wayne Ren
7cbe5c3e18 scripts: add unit test for mdb runner
add unit test for mdb runner

Signed-off-by: Wayne Ren <wei.ren@synopsys.com>
2020-06-11 10:02:11 -04:00
Wayne Ren
1506db5098 scripts: add the runner script for metaware debugger
* add the runner script for metaware debugger(mdb).
* mdb is required for SMP case
* mdb also can provides a GUI interface

Signed-off-by: Wayne Ren <wei.ren@synopsys.com>
2020-06-11 10:02:11 -04:00
Peter Bigot
3c36b42d03 sanitycheck: add notes key to schema
Prior to addition of schema validation running sanitycheck
--generate-hw-map would preserve the value of any field in an existing
map that it didn't update.  These fields now cause a parsing error.
Add notes as an optional key where information relevant to the board
can be added to provide useful context (such as which serial port
corresponds to the console, or why a non-standard runner is required).

Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
2020-06-10 12:03:17 +02:00
Peter Bigot
3cdd5b4be4 sanitycheck: add missing probe_id key to schema
probe_id is used as an optional key that overrides id as the source
of board_id when configured to use a runner that is not the default.

Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
2020-06-10 12:03:17 +02:00
Kumar Gala
491bbad8e8 ci: Update for buildkite environment
* Tweak west_setup:
  - log `west update` to a file to reduce noise in log
  - use `west forall` + `git reset` to make sure files are checked out
    (this is to handle a possible module cache)
* Output when we start sanity_check.  Add a banner for when we run
  sanity_check so it's a bit easier to find in console logs

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-06-09 15:28:48 -04:00
Ruslan Mstoi
7d8adddd2e scripts: process_gperf: typo fix
Fix double "e" in "repreesentation"

Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
2020-06-09 10:40:58 +02:00
Ruslan Mstoi
84822e3086 scripts: gen_syscalls: typo fix
Fix "defing" as "defining"

Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
2020-06-09 10:39:49 +02:00
Spoorthy Priya Yerabolu
d434dfc22c sanitylib: Adding small change to enable create_overlay() testing
Returning the content from create_overlay() in TestInstance class

Signed-off-by: Spoorthy Priya Yerabolu <spoorthy.priya.yerabolu@intel.com>
2020-06-08 16:38:48 -04:00
Spoorthy Priya Yerabolu
9ac30b595c scripts: tests: sanitycheck: Add testcases for TestInstance Class
test_testinstance.py: Add testcases for check_build_or_run()
create_overlay() and calculate_sizes()
conftest.py: Module for common pytest fixtures
test_data/board_config: Adding test data for board configurations

Signed-off-by: Spoorthy Priya Yerabolu <spoorthy.priya.yerabolu@intel.com>
2020-06-08 16:38:48 -04:00
Kumar Gala
93da8dc478 Revert "checkpatch: update checkpatch to warn about C99 type usage"
Now that we are standardizing C99 integer types we can revert the commit
that warned about C99 type usage.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-06-08 08:23:57 -05:00
Kumar Gala
a1b77fd589 zephyr: replace zephyr integer types with C99 types
git grep -l 'u\(8\|16\|32\|64\)_t' | \
		xargs sed -i "s/u\(8\|16\|32\|64\)_t/uint\1_t/g"
	git grep -l 's\(8\|16\|32\|64\)_t' | \
		xargs sed -i "s/s\(8\|16\|32\|64\)_t/int\1_t/g"

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-06-08 08:23:57 -05:00
Andrew Boie
be919d3bf7 userspace: improve dynamic object allocation
We now have a low-level function z_dynamic_object_create()
which is not a system call and is used for installing
kernel objects that are not supported by k_object_alloc().

Checking for valid object type enumeration values moved
completely to the implementation function.

A few debug messages and comments were improved.

Futexes and sys_mutexes are now properly excluded from
dynamic generation.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-06-03 22:33:32 +02:00
Andrew Boie
299ec8f1b5 userspace: net sockets are kernel objects
Any data structure declaration tagged with __net_socket will end up
in the kernel object table with type K_OBJ_NET_SOCKET. These all
correspond to objects which are associated with socket file
descriptors and can handle the socket vtable API.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-06-03 22:33:32 +02:00
Andrew Boie
5960119f16 scripts: parse_syscalls: generalize struct tags
Now we can build up lists of data structures matching a list
of particular tags, with __subsystem being just one case.

Relax searches to also look inside C files, since struct
prototypes may be declared there as well.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-06-03 22:33:32 +02:00
Andrew Boie
455e178b3b scripts: gen_kobject_list: generalize obj alloc
Instead of handling this ad hoc, generalize which kobject
types can be allocated.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-06-03 22:33:32 +02:00
Andrew Boie
378024c510 userspace: add z_is_in_user_syscall()
Certain types of system call validation may need to be pushed
deeper in the implementation and not performed in the verification
function. If such checks are only pertinent when the caller was
from user mode, we need an API to detect this situation.

This is implemented by having thread->syscall_frame be non-NULL
only while a user system call is in progress. The template for the
system call marshalling functions is changed to clear this value
on exit.

A test is added to prove that this works.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-06-03 22:33:32 +02:00
Martí Bolívar
e4a761cffe devicetree: add migration guide documentation and tests
Add test cases that verify various bits and pieces of the legacy
devicetree macros match the new APIs.

Writing these test cases without giving rise to deprecated macro
warnings which might break people's CI if they build with -Werror
requires turning off the __WARN() generation in
devicetree_legacy_unfixed.h. The entire file is deprecated at this
point and must be explicitly enabled with an opt-in Kconfig option, so
there isn't any harm in doing this.

Nevertheless, take a minimally invasive approach to avoiding __WARN()
generation in gen_legacy_defines.py, to avoid the possibility of
breakage. This code is basically frozen anyway, so hacks like this
won't cause maintainability problems since it isn't being actively
maintained.

Use the new tests as fodder for a migration guide from the old API in
the documentation.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-05-28 22:12:38 +02:00
Martí Bolívar
9d36b4f6e4 doc: some devicetree fixes and updates
Some updates to the reference page for the "core" APIs, and associated
follow-ups in the guides:

- centralize documentation of chosen zephyr nodes in a non-legacy
  file, provide a reference to them from the intro page in the guide
- review doxygen docstrings and correct errors for generic APIs
- add introductory text to each section in the API reference
- add missing hardware-specific pages

Documentation for layers built on top of these is mostly left to future
commits, but I do have a smattering of fixes in the guides that I
noticed while I was doing this.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-05-26 18:12:00 +02:00
Wentong Wu
6c9d4a5c57 scripts: use debug function instead of debug_die to dump objs
In step 4 of find_kobjects, use func debug instead of debug_die
to dump debug info to avoid dump wrong info.

Signed-off-by: Wentong Wu <wentong.wu@intel.com>
2020-05-25 18:07:20 +02:00
Wentong Wu
0bf5113c6b Revert "scripts: add DW_AT_abstract_origin check"
This reverts commit 2593a919ee.

This commit wasn't the root cause of the found bug in dwarf analyze.

Signed-off-by: Wentong Wu <wentong.wu@intel.com>
2020-05-25 18:07:20 +02:00
Maciej Perkowski
b2fa99caf8 sanitycheck: Add filtered test cases to the JUnit full report
If a test instance from .yaml file was skipped due to being on
a filtered list then there are no entries about it in the
sanitycheck_report.xml for a given platform. This commit fills
instance.results with skipped test cases also for skipped instances so
those test cases are visible when the full report is generated. Tested
for ./tests/kernel/mem_protect/userspace on nrf52dk_nrf52832 where
userspace.gap_filling is filtered out.

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2020-05-25 15:52:17 +02:00
Anas Nashif
3b86f13fab sanitycheck: add fixture support to hardware map
It is now possible to add a list of fixture a platform supports which is
matched to testcases requesting fixtures to be avaiable to be able to
run.

For example:

- available: true
  connected: true
  id: 0240000026334e450015400f5e0e000b4eb1000097969900
  platform: frdm_k64f
  product: DAPLink CMSIS-DAP
  runner: pyocd
  serial: /dev/ttyACM9
  fixtures:
    - gpio_loopback

Fixes #24943

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-05-24 20:25:51 +02:00
Anas Nashif
cefb9a170f sanitycheck: support fixtures
Add fixtures to hardware map schema.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-05-24 20:25:51 +02:00
Anas Nashif
ce8c12eb5a sanitycheck: cleanup fixture processing
Cleanup fixture processing and allow ztest testcases to support
harness_config with fixture definition.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-05-24 20:25:51 +02:00
Anas Nashif
77946fa835 sanitycheck: remove wrong message about --qemu-testing
--qemu-testing does not exist.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-05-21 22:47:36 -04:00
Ruslan Mstoi
56b65e43de scripts: gen_offset_header: add argument help text
Help text is set to documentation string of the script.

Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
2020-05-21 14:45:24 +02:00
Ruslan Mstoi
4695eb4e1d scripts: net: Fix incorrect error messages
If environment variables ZEPHYR_BASE or NET_TOOLS_BASE are unset, an
incorrect message is printed saying that "it is set, but it is not a
directory":

$ZEPHYR_BASE is unset
$ZEPHYR_BASE is set, but it is not a directory
$NET_TOOLS_BASE is unset, no net-tools found
$NET_TOOLS_BASE set, but it is not a directory

This patch fixes that issue.

Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
2020-05-21 14:45:07 +02:00
Martí Bolívar
414ed86269 edtlib: fix default type for interrupts property
The name of the interrupts property is typo-ed in the python script.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-05-20 10:41:07 +02:00
Aastha Grover
6f0a0926e3 scripts: tests: sanitycheck: Add testcases for schema validation.
test_sanitycheck.py: Add testcases for correct & incorrect
testcase/platform yaml schemas.

Signed-off-by: Aastha Grover <aastha.grover@intel.com>
2020-05-20 10:35:35 +02:00
Aastha Grover
5948ab6cf1 sanitycheck: add testcases for add_testcases function of testsuite class
test_testsuite_class.py: Add testcases for add_testcases function
of testsuite class in sanitycheck.
test_data/testcases/tests & /samples : Testcase root directory
to add all the testcases & to test add_testcases function.
conftest.py: Module for common pytest fixtures, also used for
passing data from one testcase to another.
Note: conftest.py has a class_testsuite fixture where board_root is
defined as the directory which will be added in a separate PR.

Signed-off-by: Aastha Grover <aastha.grover@intel.com>
2020-05-20 10:35:35 +02:00
Wentong Wu
2593a919ee scripts: add DW_AT_abstract_origin check
According to below rule which's from DWARF5 sepc, if the
attribute can't be founded in given DIE, check more entry
associated by DW_AT_abstract_origin.

For the purposes of determining whether a debugging information
entry has a particular attribute (such as DW_AT_name), if
debugging information entry A has a DW_AT_specification or
DW_AT_abstract_origin attribute pointing to another debugging
information entry B, any attributes of B are considered to be
part of A.

Signed-off-by: Wentong Wu <wentong.wu@intel.com>
2020-05-20 10:33:14 +02:00
Aastha Grover
a0ae53410e sanitylib: Remove hardcoded sample.yaml & testcase.yaml in TestSuite class
This fix creates class variables SAMPLE_FILENAME & TESTCASE_FILENAME
in Class TestSuite and remove the hardcoded sample.yaml & testcase.yaml
from add_testcases function. This makes testing for sanitylib script
easier so that shippable do not detect the test_data for sanitylib
as actual zephyr testcases.

Signed-off-by: Aastha Grover <aastha.grover@intel.com>
2020-05-14 22:48:09 +02:00
Wentong Wu
0d619aefd8 sanitycheck: Add cpu time for QEMUHandler
Add cpu time for QEMUHandler because the guest virtual time
in QEMU icount mode isn't host time and it's maintained by
counting guest instructions, we use QEMU process CPU time to
mostly simulate the time of guest OS.

Signed-off-by: Wentong Wu <wentong.wu@intel.com>
2020-05-14 13:52:07 +02:00
Wentong Wu
7ec57b4666 scripts: sanitylib.py: enhancement for QEMUHandler
Sometimes QEMU can't handle SIGTERM signal correctly and just
run as normal, in that case kill -9 QEMU process directly and
leave sanitycheck judge the testing result by console output.

For failures caused by other reasons, terminate all releated
processes to avoid CI waits 2 hours and reports timeout.

Signed-off-by: Wentong Wu <wentong.wu@intel.com>
2020-05-14 13:52:07 +02:00
Kumar Gala
bd97378870 devicetree: Add support for fixed-partitions
Add DT_NODE_BY_FIXED_PARTITION_LABEL that given a "label" in any
fixed-partitions map will return the node_id for that partition node.

Add DT_NODE_HAS_FIXED_PARTITION_LABEL that will test if a given
fixed-partitions "label" is valid.

Add DT_FIXED_PARTITION_ID that will return an unique ordinal value for
the partition give a node_id to the partition.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-05-13 21:22:53 +02:00
Kumar Gala
4a5a90aef8 gen_defines: Rework implementation of write_child_functions
Rework how write_child_functions to match how we do the code for
DT_FOREACH_OKAY_INST.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-05-13 21:22:53 +02:00
Kumar Gala
3a68566d40 edtlib: add a flag for old fixed-partitions behavior
allow the old generator to act as it did.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-05-13 21:22:53 +02:00
Kumar Gala
058842b353 edtlib: handle match of fixed-partition on any bus
If we have a fixed-partition on a flash device that is for example on
a spi controller we will not get a binding match currently.  This is
because we expect a match between both the compatible and the fact that
fixed-partition node is a decendant of the spi bus.

To address this we treat fixed-partitions as if they are on no bus.
This has the effect of causing a binding match as well as ensuring that
when we process the fixed-partition node we will do anything special to
it because of the bus it happens to be under (for example SPI CS_GPIO
processing).

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-05-13 21:22:53 +02:00
Kumar Gala
06edcb11db edtlib: Initialize node.compats earlier
Setup node.compats right after we create the Node.  This allows access
to the compats information in _bus_node.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-05-13 21:22:53 +02:00
Peter Bigot
49e8c7080f scripts/coccinelle: add script to preserve const qualifier on config_info
Drivers cast the device config_info pointer to a driver-specific
structure.  The referenced object is const-qualified; make sure the
cast doesn't inadvertently remove that qualifier.

Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
2020-05-13 18:21:52 +02:00
Anas Nashif
44f7ba0781 sanitycheck: detect wrongly named tests and fail
Tests should always start with test_. If not, they will not be in the
report.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-05-13 18:16:07 +02:00
Anas Nashif
4258d8d34b sanitycheck: give summary about built vs executed
It is important to see that some tests are only building and not
executing...

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-05-11 17:00:26 +02:00
Anas Nashif
91fd68dbbf sanitycheck: also look for cpp files when scanning
look for *.c* instead of just *.c files when scanning or testcases.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-05-11 17:00:26 +02:00
Anas Nashif
bb28035a82 sanitycheck: support exporting instances
Enable exporting tests per platform.

sanitycheck --export-tests out.txt  -p qemu_x86 -T tests/

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-05-11 17:00:26 +02:00
Anas Nashif
1a5defa924 sanitycheck: support --force-platform
This option allows running tests on platforms that would be excluded
based on the test definition.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-05-11 17:00:26 +02:00
Anas Nashif
60ae507783 sanitycheck: move export-tests handling and deduplicate code
Reuse the same code used for other options.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-05-11 17:00:26 +02:00
Martí Bolívar
0b5c58fcab scripts: runners: handle uninstalled dependencies
Catch ImportError whenever a non-standard module import fails from any
runners that do one. Complain at runtime about it if the user actually
needs the runner.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-05-09 13:05:48 +02:00
Christopher Friedt
2c0eecaa5e posix arch: build on aarch64 / allow host-specific cmake includes
This change enables specific compiler and linker options to be used in
the case that an arch/posix/os.arch.cmake file exists.

Note: os and arch in the above case are evaluations of
CMAKE_HOST_SYSTEM_NAME and CMAKE_HOST_SYSTEM_PROCESSOR.

Otherwise, the existing "generic" compiler and linker flags in
arch/posix/CMakeLists.txt are used.

Additional flags and checks are provided in
arch/posix/Linux.aarch64.cmake.

Added scripts/user_wordsize.py to detect if userspace is 64-bit or
32-bit, which should be consistent with the value of CONFIG_64BIT
for Aarch64 on Linux.

Fixes #24842

Signed-off-by: Christopher Friedt <chrisfriedt@gmail.com>
2020-05-09 12:17:24 +02:00
Kumar Gala
6a2cb94f6f sanitycheck: suppress warnings from EDT
We don't need sanitycheck's use of EDT to report warnings, we'll get
them from the build system.  So the warnings are just duplication and
noise, thus lets always suppress them.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-05-08 21:54:02 -04:00
Martí Bolívar
7e0eed9235 devicetree: allow access to all nodes
Usually, we want to operate only on "available" device
nodes ("available" means "status is okay and a matching binding is
found"), but that's not true in all cases.

Sometimes we want to operate on special nodes without matching
bindings, such as those describing memory.

To handle the distinction, change various additional devicetree APIs
making it clear that they operate only on available device nodes,
adjusting gen_defines and devicetree.h implementation details
accordingly:

- emit macros for all existing nodes in gen_defines.py, regardless
  of status or matching binding
- rename DT_NUM_INST to DT_NUM_INST_STATUS_OKAY
- rename DT_NODE_HAS_COMPAT to DT_NODE_HAS_COMPAT_STATUS_OKAY
- rename DT_INST_FOREACH to DT_INST_FOREACH_STATUS_OKAY
- rename DT_ANY_INST_ON_BUS to DT_ANY_INST_ON_BUS_STATUS_OKAY
- rewrite DT_HAS_NODE_STATUS_OKAY in terms of a new DT_NODE_HAS_STATUS
- resurrect DT_HAS_NODE in the form of DT_NODE_EXISTS
- remove DT_COMPAT_ON_BUS as a public API
- use the new default_prop_types edtlib parameter

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-05-08 19:37:18 -05:00
Martí Bolívar
e96ca54fd8 gen_defines: play tricks to benefit DT_INST users
Even though it is about to be done for sound technical reasons, a
subsequent patch adding access to all device nodes at the last minute
in the 2.3 release is going to be playing a bit of a fast one on
the Zephyr community, especially users of DT_INST APIs.

In particular, instance numbers are currently allocated only to
enabled nodes, but that will not be true soon: *every* node of a
compatible will be allocated an instance number, even disabled ones.

This is especially unfortunate for drivers and applications that
expect singletons of their compatibles, and use DT_INST(0, ...) to
mean "the one enabled instance of my compatible".

To avoid gratuitous breakage, let's prepare for that by sorting each
edt.compat2nodes sub-list so that enabled instances always come before
disabled ones.

This doesn't break any API guarantees, because there basically *are*
no ordering guarantees, in part precisely to give us the flexibility
to do things like this. And it does help patterns that use instances 0
through N-1, including the important singleton case.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-05-08 19:37:18 -05:00
Tomasz Bursztyka
1eedd5eb64 scripts: Fix gen_kobject_list.py to fit with device refactoring
device_api attribute is not at offset 4 but 8 now as name and
config_info has been directly imported into struct device.

Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
2020-05-08 23:07:44 +02:00
Tomasz Bursztyka
7544263050 scripts: Fix documunt referenc for coccicheck
s/applications/guides

Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
2020-05-08 17:44:44 +02:00
Henrik Brix Andersen
3c2984d4a9 scripts: west: add CANopen flash runner
Add west flash runner for program download via CANopen.

Signed-off-by: Henrik Brix Andersen <hebad@vestas.com>
2020-05-08 10:53:42 +02:00
Dominik Ermel
ba8b74d801 devicetree: Add DT_FOREACH_CHILD macro
The macro iterates through the list of child nodes and invokes provided
macro for each node.

Signed-off-by: Dominik Ermel <dominik.ermel@nordicsemi.no>
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-05-07 21:42:58 -05:00
Kumar Gala
0a7d4e2135 devicetree: Change DT_FOREACH_IMPL_ to not insert semicolon
Remove semicolon between instance invocations of DT_FOREACH_IMPL_ and
thus DT_INST_FOREACH.  This provides more flexibility to the user.  This
requires we fixup in tree users to add semicolon where needed.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-05-07 20:03:56 -05:00
Anas Nashif
61c6e2b172 sanitycheck: raise exception on bad exclusion in ztest
Abort if we find tests that are being skipped incorrectly. Tests should
be skipped using ztest_test_skip().

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-05-07 14:24:02 -04:00
Anas Nashif
7a69125fbf sanitycheck: raise exceptions on bad test names
Raise exception on bad test names such as 'test' or 'sample'.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-05-07 14:23:45 -04:00
Peter Bigot
f77e03bb44 scripts/coccinelle: rename and extend ms to timeout conversion script
Originally this only dealt with constant timeouts.  Add the ability to
recognize integer parameters and convert them as well.

Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
2020-05-07 19:46:03 +02:00
Martí Bolívar
eac56e4fb6 edtlib: add EDT.compat2nodes, EDT.compat2okay
These look up tables generalize the compat2enabled map in a way we
will need to make the API more flexible in Zephyr.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-05-07 08:12:49 -05:00
Martí Bolívar
bd0ecc83fb edtlib: add Node.status accessor
Let's get the actual node status, instead of relying on enabled.
Leave enabled in place for gen_legacy_defines.py's sake.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-05-07 08:12:49 -05:00
Kumar Gala
bc48f1ce45 edtlib: allow default property types for nodes without bindings
If a devicetree node doesn't have a matching binding we will at least
populate a common standard set of properties for that node.  The list of
standard properties is:
	compatible
	status
	reg
	reg-names
	label
	interrupt
	interrupts-extended
	interrupt-names
	interrupt-controller
This allows us to handle cases like memory nodes that don't have any
compatible property, we can still generate the reg values.
We limit this to known properties as for any other property we can not
fully determine the property type without a binding and thus we can't
ensure the generation for that property is correct or may not change.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-05-07 08:12:49 -05:00
Ruslan Mstoi
f699ab4ee6 sanitycheck: fix enable_coverage argument name typo
Argument name is --enable-coverage, whereas attribute name is
enable_coverage

Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
2020-05-06 12:03:15 -05:00
Anas Nashif
7ee9a0328b ci: changes to python requirements should not trigger sanitycheck
Those have no impact on code and should not trigger full sanitycheck
run.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-05-06 11:07:22 -05:00
Anas Nashif
2601c2e54b requirements: add psutil, needed by sanitycheck
Needed for tracking cpu time in sanitycheck.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-05-06 11:07:22 -05:00
Stephanos Ioannidis
95c6da0c82 scripts: checkpatch: Revert dt-binding vendor prefix check regex change
This commit reverts the device tree binding vendor prefix check regular
expression change that was introduced by the following commit:
5b10fac97e

The changed regular expression fails to detect the correctly specified
vendor prefixes in the `vendor-prefixes.txt`.

Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
2020-05-06 12:40:35 +02:00
Anas Nashif
aff616de20 sanitycheck: fix initial types in classes and minor cleanup
While testing sanitycheck classes, found some wrongly initialisized
variables which caused issues, fixed those..

All Test* classes should now be ignored by pytest and should not be
treated as pytest code.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-05-05 16:32:25 -04:00
Henrik Brix Andersen
359fd044d3 scripts: dts: gen_legacy_defines: handle bus nodes without regs
Correctly handle parent bus nodes without regs properties.

Signed-off-by: Henrik Brix Andersen <hebad@vestas.com>
2020-05-05 10:17:23 -05:00