In the 'normal' case of a property whose definition is taken from a
binding YAML file, a fair number (three at present) of the attributes
available on Property objects are directly taken from the
corresponding PropertySpec object.
Refactor the internals of how a Property gets initialized so that it
has a direct reference to its PropertySpec, and make those attributes
properties which just delegate to the PropertySpec (which in turn just
delegate to the binding). Additionally, expose the PropertySpec
directly.
This will make it easier to extend the Property class with additional
attributes that normally come from the PropertySpec without having to
touch all the locations where Property.__init__ is called.
In the case of the 'default' properties, we handle this by dummying
out some PropertySpec objects. These dummy PropertySpecs in turn
require a dummy Binding.
This change has the advantage that it improves the degree to which
these defaults are checked, e.g. it makes sure that 'status' is one of
the permitted values.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Now we create plaform reports for each enabled platform, this is time
consuming, especially if you are not using those reports for anything.
Make this an option and enable only if the generated reports are going
to be used.
Individual reports can also be generated from the JSON file if needed.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Runner implementations are only allowed to unconditionally import
modules in the python standard library. They are not allowed to import
anything that comes from pip or other third party sources unless they
catch the ImportError and gracefully do nothing.
Fix the imports in the mdb runner accordingly, sorting the imports
into sections to make it clearer what's stdlib, what's runners, and
what's third party while we're here.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Add a simpel wrapper script called 'sanitycheck' to be removed after
Zephyr 2.5.
This script has a warning and a 5 sec delay to get the attention of the
caller.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Move the "module" classes used by sanitycheck into a new location under
pylib, where we can have other python libraries reside instead of having
them all over the place.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Move the CSV files with release related data out of the python directory
into scripts/releases, which is more appropriate.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Move yaml schemas into one single location. We have quite a few schema
used across the tree, one common place will make them easier to find and
reference.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Rework the runner to improve various issues.
Every board.cmake file for an nRF SoC target is repeating boilerplate
needed for the nrfjprog runner's --nrf-family argument. The
information we need to decide the --nrf-family is already available in
Kconfig, so just get it from there instead. Keep the --nrf-family
argument around for compatibility, though.
This cuts boilerplate burden for board maintainers.
We also need to revisit how this runner handles recovery to fix it
in nRF53 and keep things consistent everywhere else.
To cleanly handle additional readback protection features in nRF53,
add a --recover option that does an 'nrfjprog --recover' before
flashing. Keep the behavior consistent across SoCs by supporting it on
those too. Because this is expected to be a bit tricky for users to
understand, check if a --recover is needed if the 'nrfjprog --program'
fails because of protection, and tell the user how to fix it.
Finally, instead of performing a separate 'nrfjprog --eraseall', just
give --chiperase to 'nrfjprog --program' process's arguments instead
of --sectorerase. This is cleaner, resulting in fewer subprocesses and
avoiding an extra chip reset.
Having a separate 'west flash --recover' option doubles the number of
test cases if we want to keep exhaustively enumerating them. That
doesn't feel worthwhile, so update the test cases by picking a
representative subset of the possibilities. Each test now has enough
state that it's worth wrapping it up in a named tuple for readability.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
script was failing during a run with -f, the status key was not in the
dictionary and the check would cause an exception. Fix that using
.get().
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
The current bossac runner have multiple entries for flash offset
parameter. Remove offset parameter from command line and rework
all infraestructure to get offset from device tree. Add proper
verification routines to validate configurations on board and
device tree entries to fix SDK compatibility checks.
Fixes#29312.
Signed-off-by: Gerson Fernando Budke <nandojve@gmail.com>
Add is_extended_samba_protocol method to verify if board is using SAM-BA
extended protocol (Arduino/Adafruit UF2). It allows enforce serial line
speed to 1200 baud automatically. This avoid add entries on board.cmake
file.
Signed-off-by: Gerson Fernando Budke <nandojve@gmail.com>
Add a dt_chosen_enabled(chosen) filter function to sanitycheck. The
function returns true if the devicetree /chosen node contains 'chosen'
and the referenced node is enabled.
Signed-off-by: Henrik Brix Andersen <henrik@brixandersen.dk>
mdb binary starts several subproceses and one of them is cld process.
In runners/mdb.py we record process id of cld on each mdb launch
to terminate simulator correctly later. However we can finish test
and terminate mdb before the cld process was found (so cld won't
be terminated correctly by sanitycheck infrastructure). It may happen
if we launch mdb on fast host machine.
That leads to several issues. First of all we get ugly error in
sanitycheck output:
------------------------>8--------------------------------
FileNotFoundError: [Errno 2] No such file or directory: '/xxxx/mdb.pid'
------------------------>8--------------------------------
Secondly (and it's more important) we terminate simulator incorrectly.
We terminate mdb leaving cld process alive, running and consuming one
cpu core permanently (until we kill it manually)
So, let's increase granularity of lookups and don't wait extra 0.5
seconds before the first lookup.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Add a new option that will enable treating RAM/SRAM overflows as errors
instead of skipping them, which is the default.
Fixes#27583
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
We were setting all devices as connected, now do that based on serial
availability.
Rename internal variables and make them shorter:
connected_hardware -> duts
ConnectedDevice = DUT
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
use a loop instead of all() to make the test easier to debug.
remove the default platform test, as this is now being filtered
differently in the code.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
The available key is a runtime variable, it does not need to be in the
hardware map. Make it optional to keep the file format compatible for
now.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Try and keep terminilogy consistent, a test application with multiple
test cases is now a 'test suite'.
Also fix accounting when retrying failed.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
If we provide arch on command line, do not check for all platforms of
architectures not specified.
Fixes#30099
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Show in debug mode what files are being written. In some cases this
takes a while and it is good to see what is going on.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
With python threading we have multiple issues with performance and
leakage. Use the python Process and implement locking using
multiprocessing library. This change improves performance and fixes
various issues with logging, concurrency and reliability of the output.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Running sanitycheck for multiple platforms but only request coverage
on a single platform fails, example:
sanitycheck --enable-coverage --coverage-platform nrf52840dk_nrf52840
-p nrf52840dk_nrf52840 -p nrf52dk_nrf52832 -T <sample>
This happens because `testcase_extra.conf` will only be created if the
current platform is included in the list of coverage platforms.
The error in the example above that would be seen is:
File not found:
<...>/sanitycheck/testcase_extra.conf
This commit now uses the existence of `testcase_extra.conf` which is
created in the method `create_overlay()` before appending
the conf file to the list of overlay files.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Get list of connected hardware and filter tests to only be run on
connected hardware in cases where no platform was specified when runn
the tests. If the platform does not exist in the hardware map, just
skip it.
Fixes#29166
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
We made a similar change for the nightly builds, but the PR builds
invoke run_ci.sh, so we need the same change here.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Generating JSON report format without changing
the existing report formats.
JSON report is divided into three sections
environment tests are run
test cases basic info and discarded tests
after applying filters
Added the complete debug log to the report
Fixes: #26443
Signed-off-by: Spoorthy Priya Yerabolu <spoorthy.priya.yerabolu@intel.com>
Move actual test cases from the run-sample-tests.sh script to
the network samples directory that are supported by Docker based
testing. Each network sample directory that supports Docker testing,
will contain docker-test.sh script that is sourced by the runner
script. The docker-test.sh script will run the test as needed and
then return return value to the runner script.
Signed-off-by: Jukka Rissanen <jukka.rissanen@linux.intel.com>
I keep seeing people miss the --context option in the "west flash
--help" output.
This option is very important for understanding the runner-specific
options and state, and missing it means people get confused about what
west flash, debug, etc. can do and are doing.
Try to avoid this problem by adding a big fat banner about the
omission of runner-specific options in the main help output, and
provide more hints about how to use --context.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
When running sanitycheck without any platform filters (-p option), we
take the default platforms and run on those only. When the -G options is
used for integration testing, the platforms listed as integration
platforms are not being added on top of default platforms.
This change adds integration platforms to the list an treats
integration_platforms as a positive filter, so that
integration_platforms are always being considered.
Fixes#29829
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Add a way to run dumb_http_server_mt sample application against
HTTP get script (using curl) running in Docker.
Signed-off-by: Jukka Rissanen <jukka.rissanen@linux.intel.com>
Allow gPTP sample to be run via run-sample-tests.sh script.
This makes it possible to do simple smoke testing and verify that
gPTP stack is not broken.
Signed-off-by: Jukka Rissanen <jukka.rissanen@linux.intel.com>
