This commit refactors kernel and arch headers to establish a boundary
between private and public interface headers.
The refactoring strategy used in this commit is detailed in the issue
This commit introduces the following major changes:
1. Establish a clear boundary between private and public headers by
removing "kernel/include" and "arch/*/include" from the global
include paths. Ideally, only kernel/ and arch/*/ source files should
reference the headers in these directories. If these headers must be
used by a component, these include paths shall be manually added to
the CMakeLists.txt file of the component. This is intended to
discourage applications from including private kernel and arch
headers either knowingly and unknowingly.
- kernel/include/ (PRIVATE)
This directory contains the private headers that provide private
kernel definitions which should not be visible outside the kernel
and arch source code. All public kernel definitions must be added
to an appropriate header located under include/.
- arch/*/include/ (PRIVATE)
This directory contains the private headers that provide private
architecture-specific definitions which should not be visible
outside the arch and kernel source code. All public architecture-
specific definitions must be added to an appropriate header located
under include/arch/*/.
- include/ AND include/sys/ (PUBLIC)
This directory contains the public headers that provide public
kernel definitions which can be referenced by both kernel and
application code.
- include/arch/*/ (PUBLIC)
This directory contains the public headers that provide public
architecture-specific definitions which can be referenced by both
kernel and application code.
2. Split arch_interface.h into "kernel-to-arch interface" and "public
arch interface" divisions.
- kernel/include/kernel_arch_interface.h
* provides private "kernel-to-arch interface" definition.
* includes arch/*/include/kernel_arch_func.h to ensure that the
interface function implementations are always available.
* includes sys/arch_interface.h so that public arch interface
definitions are automatically included when including this file.
- arch/*/include/kernel_arch_func.h
* provides architecture-specific "kernel-to-arch interface"
implementation.
* only the functions that will be used in kernel and arch source
files are defined here.
- include/sys/arch_interface.h
* provides "public arch interface" definition.
* includes include/arch/arch_inlines.h to ensure that the
architecture-specific public inline interface function
implementations are always available.
- include/arch/arch_inlines.h
* includes architecture-specific arch_inlines.h in
include/arch/*/arch_inline.h.
- include/arch/*/arch_inline.h
* provides architecture-specific "public arch interface" inline
function implementation.
* supersedes include/sys/arch_inline.h.
3. Refactor kernel and the existing architecture implementations.
- Remove circular dependency of kernel and arch headers. The
following general rules should be observed:
* Never include any private headers from public headers
* Never include kernel_internal.h in kernel_arch_data.h
* Always include kernel_arch_data.h from kernel_arch_func.h
* Never include kernel.h from kernel_struct.h either directly or
indirectly. Only add the kernel structures that must be referenced
from public arch headers in this file.
- Relocate syscall_handler.h to include/ so it can be used in the
public code. This is necessary because many user-mode public codes
reference the functions defined in this header.
- Relocate kernel_arch_thread.h to include/arch/*/thread.h. This is
necessary to provide architecture-specific thread definition for
'struct k_thread' in kernel.h.
- Remove any private header dependencies from public headers using
the following methods:
* If dependency is not required, simply omit
* If dependency is required,
- Relocate a portion of the required dependencies from the
private header to an appropriate public header OR
- Relocate the required private header to make it public.
This commit supersedes #20047, addresses #19666, and fixes#3056.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
(pinctrl-<index> is documented in
Documentation/devicetree/bindings/pinctrl/pinctrl-bindings.txt in
Linux.)
Add a new Node.pin_states property, derived from any pinctrl-<index>
properties on the node. Node.pin_states holds a list of PinState
objects, where each PinState represents a single pinctrl-<index>
property.
For example, Node.pin_states will have two elements for the 'device'
node below:
device {
pinctrl-0 = <&state_0>;
pinctrl-1 = <&state_1 &state_2>;
pinctrl-names = "default", "sleep";
};
pincontroller {
state_0: state_0 {
...
};
state_1: state_1 {
...
};
state_2: state_2 {
...
};
};
Each PinState holds the list of configurations nodes in
PinState.conf_nodes. For the node above, node.pin_states[1].conf_nodes
will contain the pincontroller/state_1 and pincontroller/state_2 nodes,
for example.
The new functionality isn't used by gen_defines.py yet, so this change
is a no-op in itself, except it adds some error checking for
pinctrl-<index> properties.
If needed, support for #pinctrl-cells and 'pinmux' (not the same thing
as the 'pinmux' properties in Zephyr I think) could be added separately
later. Not sure what belongs in edtlib.py there yet.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Let the user specify these arguments as many times as they want:
--cmd-pre-init
--cmd-pre-load
--cmd-post-verify
This makes it a bit easier to handle scripts that need to do a few
things in a row depending on conditions.
Handle --cmd-pre-init and the port arguments properly in the debug
related targets.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
I'd like to support nRF boards in a more generic way, so make it
optional. We can clean up the STM32 files later if there's a lot of
overlap in their .cfg files.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
None of the splitting logic is needed if 64-bit return
values or parameters fit inside a register.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
We need to pass system call args using a register-width
data type and not hard-code this to u32_t.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
This commit fixes an issue in windows where zephyr_modules.txt contains
a \ as path separator.
This causes issues later when using the path generated by
zephyr_module.txt are used as variables in CMake on windows.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
If we don't have a DTS (like nrf52_bsim) we shouldn't try and create an
EDT, but we still need to call expr_parser.parse to filter testcases.
So move the os.path.exists(dts_path) around the creation of the EDT.
Fixes: #20371
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
If we have something like dma-names we should ended up generating
something like:
DT_ST_STM32_I2S_40015000_RX_DMAS_CONTROLLER_1
This isn't quite what we expect. We should instead get:
DT_ST_STM32_I2S_40015000_RX_DMAS_CONTROLLER
DT_ST_STM32_I2S_40015000_DMAS_CONTROLLER_1
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
nrf52_bsim does not generate any DTS data to be processed by
sanitycheck, skip filtering if we have no dts data.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Device tree nodes have a dependency on their parents, as well as other
nodes. To ensure driver instances are initialized in the proper we
need to identify these dependencies and construct a total order on
nodes that ensures no node is initialized before something it depends
on.
Add a Graph class that calculates a partial order on nodes, taking
into account the potential for cycles in the dependency graph.
When generating devicetree value headers calculate the dependency
graph and document the order and dependencies in the derived files.
In the future these dependencies may be expressed in binding data.
Signed-off-by: Peter A. Bigot <pab@pabigot.com>
environment is passed to execution thread and should be available
regardless of SAN being enabled or not.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
sanitycheck now requires yaml.FullLoader to support generating
hardware maps. This feature was introduced in PyYAML 5.1.
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
Add function that will return 'y' or 'n' if a node pointed to by a
chosen property exists and is enabled.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
All users of dt kconfig symbols for tests and samples are using dt
functions instead so we can remove the support for parsing the
generated file.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Add the following functions to allow filtering based on device tree
dt_compat_enabled(compat) - Returns true if a device tree node
compatible matches 'compat' and the node is enabled.
dt_alias_exists(alias) - Returns true if a device tree node exists with
'alias' and the node is enabled.
dt_compat_enabled_with_alias - Returns true if a device tree node
compatible matches 'compat' and the node has 'alias' and the node is
enabled.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Just calls through to concurrent.futures.Executor.shutdown() in the base
class, which has the same signature. Removing it means the base class
version will get used directly.
Fixes this pylint warning:
W0235: Useless super delegation in method 'shutdown'
(useless-super-delegation)
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Fixes this pylint warning:
R0201: Method could be a function (no-self-use)
Could also make it a class method.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
We try to demonstrate some concepts for user mode:
- Multiple logical applications, each with their own memory
domain
- Creation of a sys_mem_pool and assignment to a memory
partition
- Use of APIs like k_queue_alloc_append() which require
thread resource pools to be configured
- Management of permissions for kernel objects and drivers
- Show how application-specific system calls are defined
- Show IPC between ISR and application (using k_msgq) and
application-to-application IPC (using k_queue)
Fixes: #14683
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Multi-line comments were stuck as-is between /* and */ in the generated
header, which looks ugly and confusing e.g. for multi-line
binding/property descriptions.
Use this format for multi-line comments in the header instead:
/*
* First line
* Second line
*
* Line after blank line
*/
Also clean up the output a bit by turning some things that were separate
comments into a single multi-line comment. Add some air and reduce line
lengths too.
Before:
/* Generated by gen_defines.py */
/* DTS input file: hifive1.dts.pre.tmp */
/* Directories with bindings: $ZEPHYR_BASE/dts/bindings */
/* Devicetree node: /cpus/cpu@0/interrupt-controller */
/* Binding (compatible = riscv,cpu-intc): $ZEPHYR_BASE/... */
/* Binding description: This binding describes the RISC-V ...
Some extra lines
for testing */
#define DT_INST_0_RISCV_CPU_INTC 1
After:
/*
* Generated by gen_defines.py
*
* DTS input file:
* hifive1.dts.pre.tmp
*
* Directories with bindings:
* $ZEPHYR_BASE/dts/bindings
*/
/*
* Devicetree node:
* /cpus/cpu@0/interrupt-controller
*
* Binding (compatible = riscv,cpu-intc):
* $ZEPHYR_BASE/dts/bindings/interrupt-controller/...
*
* Description:
* This binding describes the RISC-V CPU Interrupt Controller
*
* Some extra lines
* for testing
*/
#define DT_INST_0_RISCV_CPU_INTC 1
Also tweak Node.description and Property.description in edtlib to be
strip()ed instead of rstrip()ed. There's probably no reason to
preserving leading whitespace in them either.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Use the LibYAML-based yaml.CLoader if available instead of yaml.Loader,
which is written in Python and slow. See
https://pyyaml.org/wiki/PyYAMLDocumentation.
This speeds up gen_defines.py from 0.2s to 0.07s on my system, for
-DBOARD=hifive1. It should also make scripts/kconfig/kconfig.py faster,
because it indirectly uses edtlib via
scripts/kconfig/kconfigfunctions.py.
yaml.CLoader seems to be available out of the box when installing with
pip on Ubuntu at least.
Helps with https://github.com/zephyrproject-rtos/zephyr/issues/20104.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Simplify use of property values that have multiple elements by
providing all of them in an initializer list, in addition to each one
as an enumerated value.
For example if a driver requires a sequence of operations with
instance-specific delays between stages the durations can be specified
with:
delays = <30000 20 45000>;
and the driver can use:
static u32_t delays[] = DT_INST_0_COMPAT_DELAYS;
rather than enumerating the instances. This is particularly useful
when the number of entries in the array varies per instance, in which
case such an initialization is not easily expressed in code.
Signed-off-by: Peter A. Bigot <pab@pabigot.com>
Replace:
dt_chosen_reg_addr
dt_chosen_reg_size
dt_node_reg_addr
dt_node_reg_size
with:
dt_chosen_reg_addr_int
dt_chosen_reg_size_int
dt_chosen_reg_addr_hex
dt_chosen_reg_size_hex
dt_node_reg_addr_int
dt_node_reg_size_int
dt_node_reg_addr_hex
dt_node_reg_size_hex
So that we get the proper formatted string for the type of symbol.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
save_reports should be one of the last tasks executed because it
closes the log file. Withouth it, other functions that use debug
functions like info and error will try to write into a close file.
This fixes the following problem:
sanitycheck -x=USE_CCACHE=0 -p native_posix -T samples/hello_world/ -b
-N --log-file sanity.log
JOBS: 8
Building initial testcase list...
1 test configurations selected, 0 configurations discarded due to filters.
Adding tasks to the queue...
total complete: 1/ 1 100% skipped: 0, failed: 0
1 of 1 tests passed (100.00%), 0 failed, 0 skipped with 0 warnings in
2.91 seconds
Traceback (most recent call last):
File "./zephyr/scripts/sanitycheck", line 3866, in <module>
main()
File "./zephyr/scripts/sanitycheck", line 3854, in main
suite.summary(options.disable_unrecognized_section_test)
File "./zephyr/scripts/sanitycheck", line 2306, in summary
self.duration))
File "./zephyr/scripts/sanitycheck", line 432, in info
log_file.write(what + "\n")
ValueError: I/O operation on closed file.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
Using exact versions for our PyPI dependencies (i.e. requirements.txt
lines that look like "dependency==X.Y.Z") is rude since we're asking
users to install these dependencies outside of any virtual
environment. This causes conflicts with other packages which may
depend on the same things as us.
Use inclusive ordered comparison (>=) release specifiers instead:
https://www.python.org/dev/peps/pep-0440/#inclusive-ordered-comparison
If this causes problems for individual packages, we can refine them
using compatible release specifiers (~=):
https://www.python.org/dev/peps/pep-0440/#compatible-release
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Script to be used when creating a release. For regular releases this can
be called this way:
$ list_issues.py -f issues.md -s 2019-09-01
Will list all closed issues since september 1st, 2019 and will create a
markdown file with all issues that can be added as is to the release
notes.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
The lvgl library is configured in some tests to use
the libC malloc to allocate heap. In native_posix this
ends up in the underlying OS heap, which cannot be safely
freed in general when the program finnishes, and
ends up being reported by valgrind as leaked on exit. See
https://github.com/zephyrproject-rtos/zephyr/pull/18471
for a longer discussion.
Instead of trying to fix the leak, let's just accept it
but suppress the errors so it does not confuse other
developers
Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
Failed unit tests were setting wrong fail string (error instead of
failed) which made unit tests always pass.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Set threads spawned by BinaryHandler dameon, so when the main thread
exits for some unusual reason (e.g SIGINT) the child threads are
automatically killed.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
os.mkdir() is not suitable to create arbitrary directory path (can
create only a subdir of an existing dir, will error out if already
exists), os.makedirs() should be always used in such cases.
Signed-off-by: Paul Sokolovsky <paul.sokolovsky@linaro.org>
Previously, if two bindings had the same 'compatible:'/'parent-bus:'
values, the binding that happened to be loaded last would get used.
Turn it into an error instead. This avoids tricking people into thinking
that bindings get loaded in a defined order.
Maybe overriding bindings could be allowed later, if we need it.
Fixes: #19536
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Simplifies the code a bit.
Looks like the description wasn't rstrip()ed when it came from a
'child-binding:' either. This also indirectly fixes that.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Make sure we cleanup the manifest files at the right spot of the CI
process. When not building a PR, behavior is different.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
A complete overhaul of the sanitycheck script and how we build and run
tests. This new version of sanitycheck uses python for job distribution
and drop use of Make.
In addition to the move to python threading library, the following has
been changed:
- All handlers now run in parallel, meaning that any simulator will run
in parallel and when testing on multiple devices (using
--device-testing) the tests are run in parallel.
- Lexicial filtering (using the filter keyword in yaml files) is now
evaluated at runtime and is no long being pre-processed. This will allow
us to immediately start executing tests and skip the wait time that was
needed for filtering.
- Device testing now supports multiple devices connected at the same
time and is managed using a hardware map that needs to be generated and
maintained for every test environment. (using --generate-hardware-map
option).
- Reports are not long stored in the Zephyr tree and instead stored in
the output directory where all build artifacts are generated.
- Each tested target now has a junit report in the output directory.
- Recording option for performance data and other metrics is now
available. This will allow us to record the output from the console and
store the data for later processing. For example benchmark data can be
captured and uploaded to a tracking server.
- Test configurations (or instances) are no longer being sorted, this
will help with balancing the load when we run sanitycheck on multiple
hosts (as we do in CI).
And many other cleanups and improvements...
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
k_thread_create and K_THREAD_DEFINE both take a delay as the final
parameter. Most uses of K_THREAD_DEFINE pass either `K_NO_WAIT` or
`K_FOREVER`. Ensure that all uses of K_THREAD_DEFINE follow that
practice, and that the runtime k_thread_create calls do so as well.
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
k_sleep uses the same underlying thread infrastructure as the other
functions that take timeouts, so the delay should be specified as a
timeout rather than milliseconds.
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
Sort the functions within the regular expression so they can be
checked more easily.
Remove k_thread_deadline_set as it takes an argument in cycles. (The
one in-tree call to this function was not affected by this error.)
Add missed k_mbox_data_block_get.
Fix an overly ambitious multi-match disjunct that covered some
non-existent functions.
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
Add support for the report and patch modes so this can be invoked by
coccicheck.
Use PCRE options to make the kernel timeout API identifier rule more
readable. Extend the pattern to new API.
Use rule extends and depends clauses, and pattern disjunction, to
avoid replicating metavariable content.
Hint that using --include-headers may be helpful (some patterns can be
found in static inline functions).
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
