extract_property() is meant to generate #defines for a single property,
like 'foo = <1 2 3>'. Currently, it also generates node-level #defines
related to parent buses.
That makes the intent of the code hard to understand, and also means
that identical node-level #defines get redundantly added multiple times
(once per property).
Generate the node-level bus #defines before processing properties, in
generate_node_defines(). Use a new generate_bus_defines() helper.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Due to the way the code was structured,
flash.extract_partition(node_path) was called multiple times for the
same 'node_path'.
That must've been a mistake to begin with (but was hard to spot before
the code was cleaned up). Move the generation of per-node #defines out
of the property loop.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Do not load the GENERATED_DTS_BOARD_CONF if in doc mode, since it will
not defined as it doesn't apply. No need to defined it to a dummy value.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
generate_node_defines() immediately returns if the node's 'compatible'
is not in the binding.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Derive 'node_compat' and 'prop_val' (the dictionary for the property
from the YAML binding) inside extract_property().
That gives it just two parameters and makes it clearer that it's just
generating #define's for a single device tree property. 'prop_val' was
only used to look up prop_val['type'].
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Moves it closer to where it's used, and will allow other simplifications
in generate_node_defines().
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
After lots of reverse-engineering, I understand why deepcopy() is used:
1) Various functions defined in scripts/dts/extract/* modify the list
of names in-place
2) A plain list.copy() won't work when the *-names property happens to
have a single name in it, because devicetree.py gives a string
instead of a list in that case
Using deepcopy() to solve (2) is very confusing (especially with no
comments), because no deep copying is actually needed.
Get rid of deepcopy(), add a helper function for fetching the names, and
some comments.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
- Remove dead "if 'props' in nodes" check
- Rename the 'nodes' parameter to 'node'. It represents a single node.
- Use a defaultdict(int) for 'last_used_id' (makes unset keys give 0)
- Use a global for 'last_used_id'
- Move some related code so it appears together
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Update Kconfiglib (and menuconfig, just to sync) to upstream revision
99a7af769352b, to add the commit below, for a doc issue reported by
Sebastian Bøe:
Document that kconfig_filenames keeps absolute paths as-is
Came up in https://github.com/ulfalizer/Kconfiglib/issues/67.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
The commit header as added by "git commit -s" is "Signed-off-by:",
and not any other casing. While it perhaps case-insensitive, use
canonical casing to not confuse (attentive) users.
Signed-off-by: Paul Sokolovsky <paul.sokolovsky@linaro.org>
Turn
if a:
if b:
...
into
if a and b:
...
Simplify some error messages with .format() as well, and get rid of some
redundant str()s. '{0} {1} {1}'.format('foo', 'bar') gives
'foo bar bar'.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
- Reduce the indentation and remove the awkward line breaks. This is
possible now that there's no recursion.
- Be consistent with quotes
- Move some initialization closer to where it's used
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
The recursion was used to implement 'properties:' within 'properties:'
in binding files, which seems to be a dead leftover (and undocumented).
Removing it gets rid of code and makes things more transparent.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Use more descriptive naming, get rid of unused 'v' variable, and an
unnecessary None check (None won't appear in get_binding_compats()).
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
...to generate_node_defines().
More explicit. "extract node include info" can be read in many ways:
- Extract a node's "include info"
- Extract node and include info
- etc.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
That way it consistently generates all #define's.
Add some related clarifying comments to main() too.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Update the pyocd version to pick up a fix for the command 'pyocd erase',
which was broken in 0.15.0 and fixed in 0.16.0. Although we don't have
any in-tree uses of this command in zephyr, there will soon be in-tree
uses in mcuboot when the deprecated 'pyocd-flashtool' and 'pyocd-tool'
commands are replaced with the new unified pyocd commands.
Signed-off-by: Maureen Helm <maureen.helm@nxp.com>
We have been overriding OVERLAY_CONFIG coming from tests. Concat all
files and create one variable that we pass to kconfig as list.
Fixes#13320
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Found by "disordered --shuffle-dirents=yes".
Sorting os.walk() in scripts/subfolder_list.py removes the randomness in
the following files:
build/zephyr/misc/generated/syscalls_subdirs.txt
build/zephyr/CMakeFiles/syscall_list_h_target.dir/ or build.ninja
Sorting os.walk() in scripts/parse_syscalls.py removes the randomness
in:
build/zephyr/misc/generated/syscalls.json
build/zephyr/include/generated/syscall_dispatch.c
build/zephyr/include/generated/syscall_list.h
Note my (limited so far) testing did *not* observe any randomness in any
object file that this would address; the main purpose here is to remove
a very large amount of noise in diffoscope.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Fix some usability issues with this command.
- add help making it clear that either imgtool must be installed, or
the path to imgtool.py must be provided using --tool-path.
- in case people don't read that, print a helpful message if imgtool
is not installed and --tool-path is not provided.
- in case the build directory is not properly set up for an MCUboot
chain-loaded image, make the BuildConfiguration inspection more
robust, printing out errors using check_force() when values are
missing.
- mark the --tool option required to print usage and avoid a
RuntimeError if it is not provided.
- make sure we search for the default build directory before checking
for its existence, in case it was not provided
Signed-off-by: Marti Bolivar <marti@foundries.io>
A trivial bug that caused the following error when this situation
actually happened:
Too many thread objects (21)
Traceback (most recent call last):
File "/zephyr/scripts/gen_kobject_list.py", line 307, in <module>
main()
File "/zephyr/scripts/gen_kobject_list.py", line 281, in main
-(-thread_counter // 8))
TypeError: write() takes exactly one argument (2 given)
With this commit, the detected error is printed correctly, e.g:
Too many thread objects (21)
Increase CONFIG_MAX_THREAD_BYTES to 3
Signed-off-by: Tomasz Gorochowik <tgorochowik@antmicro.com>
If kernel page table isolation is enabled, we generate a second
set of page tables. These tables, except for the shared page, have
all non-user pages marked as non-present.
The MMU generation script has been refactored:
- Debugging output has been make significantly simpler and less
verbose
- Useless globals removed or adjusted
- MMU region list is validated as it is read
- Some tuples unpacked into individual variables to make the
code easier to read.
- Useless command line option for output binary endian-ness
remobved
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
It's special-cased for merging bindings and not a general dictionary
merging function.
Also simplify the documentation a bit.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
The Bindings class was only used to implement '!include foo.yaml'
(easier to see after some things were moved out of it). Use plain
functions instead, which might be a bit more transparent and gives
simpler code.
Also remove the recursive '!include' detection for now, which is broken
in that the same .yaml being included twice will always trigger it, even
for non-circular cases.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
As written, this function could take e.g. '!include bar', find 'foo-bar'
(due to the endswith()), and include it. This is undocumented and
doesn't seem to be used, so I'm guessing that it isn't intentional.
Removing that bug/feature also makes the code much simpler.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
load_bindings() is trivial, and it allows some stuff to simplified, like
the check for no bindings being found, and the assignments to the
extract.globals.{bindings,bus_bindings,bindings_compat}.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
This is (probably) a temporary change just to be able to move some stuff
out of the Bindings class.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Only write out to the .conf file if the define starts with "DT_". The
the conf file should only be used by sanitycheck, west, and
kconfigfunctions at this time. In the future we should remove it, so
lets limit what's it exposing at this time.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
DT_FLASH_AREA_<X>_DEV should be the label of the flash controller. For
flashes that are "jedec,spi-nor" the flash itself is considered the
controller. For "soc-nv-flash" the parent of the flash is assumed to be
the controller.
Rework the logic in how we determined DT_FLASH_AREA_<X>_DEV to handle
both cases.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Removes some logic from merge_included_bindings() and makes it easier to
follow.
Also remove an outdated comment ('id' no longer exists) and some
comments that might not be helpful.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
The yaml_list dictionary is three regular variables in disguise, as keys
are never added. Use regular variables instead, which is less confusing.
Hopefully the naming makes sense.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
The consistency checks at the beginning of the function were not done
for "leaf" .yaml files.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
...to merge_included_bindings().
This gives a better hint to what it does (merges properties from
!include'd files).
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Remove the '!include {foo: bar}' support from Bindings._include(). Only
'!include foo.yaml' and '!include [foo.yaml, bar.yaml]' are documented,
and yaml_traverse_inherited() doesn't implement the dict case either.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Make kconfigfunctions.py agnostic to where GENERATED_DTS_BOARD_CONF is
located.
We don't want to encode the path of GENERATED_DTS_BOARD_CONF into more
places than necessary. Also, re-using GENERATED_DTS_BOARD_CONF makes
it easier to change where generated_dts_board.conf is located.
Signed-off-by: Sebastian Bøe <sebastian.boe@nordicsemi.no>
kconfigfunctions.py will silently set all values to '0' when the
environment variable 'PROJECT_BINARY_DIR' is not set.
But PROJECT_BINARY_DIR not being set indicates an internal error, so
we should instead error out with an appropriate error message.
Silently failing is dangerous and increases debug time. It is better
to fail fast.
Signed-off-by: Sebastian Bøe <sebastian.boe@nordicsemi.no>
Documentation for kconfigfunctions, mentions they look up elements
in "the DTS generated "conf" style database". This indication
could be cryptic for new zephyr users. Adding the exact name and
path of the file for disambiguation.
Signed-off-by: Erwan Gouriou <erwan.gouriou@linaro.org>
When we change documentation or board images, do not build all tests for
this board. Only add more tests when actual code has changed.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
When we change documentation or board images, do not build all tests for
this board. Only add more tests when actual code has changed.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Mark any define that doesn't start with DT_, LED, SW, or have PWM_LED in
the name as deprecated via __DEPRECATED_MACRO.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Some more work will be done here later (yaml_list could be replaced by a
few plain variables, for example), but this is a start:
- Extract the compat string from the 'compatible:' line directly in the
regex, instead of separately
- Flatten the code, turning
if a:
...
if b:
...
if c:
...
into
if not a:
continue
...
if not b:
continue
...
if not c:
continue
...
- Add some comments
- Use clearer naming
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Previously, all_compats() returned a dictionary mapping nodes to their
'compatible' values. This dictionary was then (only) used to generate a
set() of all 'compatible' strings.
Save a step and get rid of some code by having all_compats() return a
set() with all 'compatible' strings directly.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
- Rename to all_compats(). The "compats" part is consistent with
elsewhere.
- Don't require an empty dictionary to be passed in. Build the
result within the function instead.
- Use more specific names (k, v -> child_name, child_node)
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
It creates the global 'phandles' dictionary. get_*() makes it sound like
it returns a value.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
It creates the global 'reduced' dictionary. get_*() makes it sound like
it returns a value.
Fix a comment that might make people think the global dictionaries are
Python lists too.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Check if the node is missing from 'defs' first, and insert it if not.
That way, later code can assume it exists.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Use DT_FLASH_WRITE_BLOCK_SIZE prefixed defined instead of
FLASH_WRITE_BLOCK_SIZE as the non-DT version is deprecated.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Its possible that a reg property is address only. If that's the case we
should produce any alias information for the size. We should check
nr_size_cells to decide to handle the alias case or not.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
If we have an alias pointing to another alias we would get None in the
resulting conf file. However that alias might exist, so try looking it
up again in the aliases db.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
We generate the following FLASH prefixed defines:
FLASH_AREA_MCUBOOT_LABEL
FLASH_AREA_MCUBOOT_OFFSET
FLASH_AREA_MCUBOOT_OFFSET_0
FLASH_AREA_MCUBOOT_READ_ONLY
FLASH_AREA_MCUBOOT_SIZE
FLASH_AREA_MCUBOOT_SIZE_0
Now we generate DT_ prefixed versions:
DT_FLASH_AREA_MCUBOOT_LABEL
DT_FLASH_AREA_MCUBOOT_OFFSET
DT_FLASH_AREA_MCUBOOT_OFFSET_0
DT_FLASH_AREA_MCUBOOT_READ_ONLY
DT_FLASH_AREA_MCUBOOT_SIZE
DT_FLASH_AREA_MCUBOOT_SIZE_0
And will deprecate the non DT_ prefixed versions in the future. We keep
them around as aliases for the time being.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
We generate the following FLASH prefixed defines:
FLASH_LABEL
FLASH_ERASE_BLOCK_SIZE
FLASH_WRITE_BLOCK_SIZE
Now we generate DT_ prefixed versions:
DT_FLASH_LABEL
DT_FLASH_ERASE_BLOCK_SIZE
DT_FLASH_WRITE_BLOCK_SIZE
And will deprecate the non DT_ prefixed versions in the future. We keep
them around as aliases for the time being.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
This commit allows for Zephyr modules to be natively integrated into
the build system with CMakeLists.txt and Kconfig files.
The sourcing of module files are done in following order:
- If <module>/zephyr/module.yml exists, use cmake and kconfig settings
for sourcing of additional file
- Else if <module>/zephyr/CMakeLists.txt exists, source this file into
CMake build tree and add <module>/zephyr/Kconfig as osource
If none of the above files are present, the project is considered to
not be a Zephyr module
Signed-off-by: Torsten Rasmussen <torsten.rasmussen@nordicsemi.no>
We've removed all references to base_label in dts bindings and are
removing the functionality. We should depend on DT_ prefixed defines
going forward.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Clarify the docs for dt_str_val that if the name isn't found we return
and empty string. Also cleanup the code slightly as we don't need to
escape the double quote.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Replace generating CONFIG_ symbols with DT_ symbols for chosen
properties like 'zephyr,console' or 'zephyr,bt-mon-uart'. We now use a
kconfigfunctions (dt_str_val) to extract the info from dts into Kconfig.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Since we know do DTS before Kconfig we should try and remove dts from
creating Kconfig namespaced symbols and leave that to Kconfig. So
rename CONFIG_CCM_<FOO> to DT_CCM_<FOO>.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Device testing has various problems:
- the main thread doesn't halt the serial monitoring thread when an
error occurs while flashing, meaning flash errors can look like
timeouts
- serial_line is unbound if the ser.readline() call raises an
exception, so the following "if serial_line" would raise NameError
in that case
- the serial monitoring thread is not a daemon, so exceptions or
errors in the handle() thread that cause thread exit will leave the
process sitting around until the serial monitoring thread has a
chance to exit
Fix these issues:
- Rewrite the serial monitoring thread as a select loop over the
serial port file descriptor and a pipe shared by the main thread and
serial monitoring thread. If flashing fails, the main thread uses
the pipe to signal to the serial monitoring thread that it should
exit.
- Make sure serial_line is always bound when read.
- Mark the serial thread daemonic.
Signed-off-by: Marti Bolivar <marti@foundries.io>
If the user has requested verbose output, then use check_call() when
running the flash target so that the build system output goes to the
controlling terminal's stdout instead of being swallowed.
This makes it easier to debug flashing failures.
Signed-off-by: Marti Bolivar <marti@foundries.io>
This was never a long-term solution, more of a gross hack
to get test cases working until we could figure out a good
end-to-end solution for memory domains that generated
appropriate linker sections. Now that we have this with
the app shared memory feature, and have converted all tests
to remove it, delete this feature.
To date all userspace APIs have been tagged as 'experimental'
which sidesteps deprecation policies.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Zephyr may be linked against third-party libraries which
were not part of the build. These may contain globals
which need to end up in a memory partition.
We can now specify the names of these libraries, as well
as a destination partition for their globals.
Some excessively long variables were renamed.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
* K_APP_DMEM_SECTION/K_MEM_BMEM_SECTION macros now exist
to specifically define the name of the sections for data
and bss respectively.
* All boards now use the gen_app_partitions.py script, the
padding hacks for non-power-of-two arches didn't work right
in all cases. Linker scripts have been updated.
* The defined k_mem_partition is now completely initialized
at build time. The region data structures now only exist
to zero BSS.
Based on some work submitted by Adithya Baglody
<adithya.baglody@intel.com>
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Parse CMakeCache.txt and filter based on variables defined in CMake.
The CMakeCache.txt parsing is borrowed from west.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
As in "#define string". More explicit.
Rename the parameters to name/value instead of k/v too.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
That the 'aliases' key isn't a regular property makes things a bit
confusing. Write the aliases separately, which should be a bit clearer.
That also makes it clear that the order doesn't matter.
The 'aliases' key always exists in defs[node], so use [] instead of
.get(). Also write "'aliases'" instead of "prop" when the value is
known.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Probably easier to understand.
Also rename 'fd' to 'f'. "File descriptor" might make it sound like an
integer.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
