Commit Graph

953 Commits

Author SHA1 Message Date
Marti Bolivar
90c0cfdc81 scripts: zephyr_flash_debug: add debug support to pyocd
Signed-off-by: Marti Bolivar <marti.bolivar@linaro.org>
2017-10-28 13:31:22 -04:00
Marti Bolivar
9cf7d106f1 scripts: zephyr_flash_debug: refactor pyocd runner
Prep work for adding debug and debugserver support. No significant
functional differences.

Signed-off-by: Marti Bolivar <marti.bolivar@linaro.org>
2017-10-28 13:31:22 -04:00
Marti Bolivar
77c2ef54f5 scripts: zephyr_flash_debug: add debug support to openocd
Signed-off-by: Marti Bolivar <marti.bolivar@linaro.org>
2017-10-28 13:31:22 -04:00
Marti Bolivar
4a665366d9 scripts: zephyr_flash_debug: refactor openocd runner
Preparatory work for supporting debugging and flashing. No significant
behavioral changes.

Signed-off-by: Marti Bolivar <marti.bolivar@linaro.org>
2017-10-28 13:31:22 -04:00
Marti Bolivar
9224b54105 scripts: zephyr_flash_debug: fix ARC OPENOCD_EXTRA_INIT
Though it isn't used by any in-tree Makefile.boards, looking at the
RIOT OS build system, this is meant to be split along lexical
boundaries defined by the shell, not just whitespace.

Signed-off-by: Marti Bolivar <marti.bolivar@linaro.org>
2017-10-28 13:31:22 -04:00
Marti Bolivar
9e619b0d26 scripts: zephyr_flash_debug: make ARC runner portable
Use run_client_and_server() to work across supported platforms.

Signed-off-by: Marti Bolivar <marti.bolivar@linaro.org>
2017-10-28 13:31:22 -04:00
Marti Bolivar
3e01cef06f scripts: zephyr_flash_debug: teach runners a client/server abstraction
Several debugging scripts run setsid before executing a server
process, then run GDB with SIGINT ignored.

Relying on setsid is not portable. Add a popen_ignore_int() helper
that provides a portable alternative, and provide a generic
run_server_and_client() in ZephyrBinaryRunner which uses it to
abstract the pattern.

Subsequent patches will use this to implement the 'debug' command.

Signed-off-by: Marti Bolivar <marti.bolivar@linaro.org>
2017-10-28 13:31:22 -04:00
Marti Bolivar
45440be89e scripts: zephyr_flash_debug: debug like arc_debugger.sh
Add debug and debugserver support.

Signed-off-by: Marti Bolivar <marti.bolivar@linaro.org>
2017-10-28 13:31:22 -04:00
Marti Bolivar
8ceb7ca385 scripts: zephyr_flash_debug: add missing nrfjprog line
The --pinreset line from the shell script is missing.

Signed-off-by: Marti Bolivar <marti.bolivar@linaro.org>
2017-10-28 13:31:22 -04:00
Marti Bolivar
fbe2fcda88 scripts: zephyr_flash_debug: generalize to support debugging
Replace the 'flash' method with a 'run' method, which takes a command
to run (flash, debug, or debugserver).

Rename the classes involved appropriately, and generalize the factory
interfaces as needed.

Add documentation and theory of ops.

Signed-off-by: Marti Bolivar <marti.bolivar@linaro.org>
2017-10-28 13:31:22 -04:00
Anas Nashif
789d51c4aa ci: do not treat message parts as xml
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2017-10-28 09:00:50 -04:00
Andrew Boie
8c20db7d44 sanitycheck: tone down verbose output
We only had a few hundred tests run when sanitycheck was first written,
and printing out the reasoning why tests were skipped seemed reasonable
at the time. Now that we are running tens of thousands of tests, this
is too much information.

The dump of what tests were skipped and why now requires two instances
of --verbose on the command line.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2017-10-27 10:55:12 -07:00
Kumar Gala
1f0c60cd4b bossa: add support for specifying the port
Now that we have a newer version of bossa in the SDK we can use a
version where there -p option works properly.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2017-10-25 03:55:15 +02:00
Adithya Baglody
d1703691c8 x86: MMU: Generation of PAE tables
If CONFIG_X86_PAE_MODE is enabled for the build, then gen_mmu.py
would generate the boot time page tables in PAE format.
This supports 3 level paging i.e Page Directory Pointer(PDPT), Page
Directory(PD) and Page Table(PT). Each Page Table Entry(PTE) maps to
a 4KB region. Each Page Directory Entry(PDE) maps a 2MB region.
Each Page Directory Pointer Entry(PDPTE) maps to a 1GB region.

JIRA: ZEP-2511

Signed-off-by: Adithya Baglody <adithya.nagaraj.baglody@intel.com>
2017-10-23 10:13:07 -07:00
Kumar Gala
03fb9ff5f9 scripts/dts/extract_dts_includes.py: Cleanup yaml file finding
Cleanup how we find the yaml files for device tree bindings.  Move to a
recursive dir search of the dts/ dir.  This will be useful for
supporting re-organizing of the yaml files to match binding dir
structure.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2017-10-23 15:02:00 +02:00
Anas Nashif
f35002c04d ci: fix encoding of documentation warning file
The file was opened as text, to avoid issues with UTF-8 in the future,
make it all binary and encode to UTF-8 correctly.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2017-10-20 15:53:17 -04:00
Leandro Pereira
1d38d98284 scripts: Fix ESP_TOOL detection in zephyr_flash_debug.py
`espidf` was written as `espdif`, causing auto-detection to fail.

Signed-off-by: Leandro Pereira <leandro.pereira@intel.com>
2017-10-19 17:13:59 -04:00
Marti Bolivar
a22a7fe1f8 scripts: zephyr_flash_debug: fix PYOCD_DAPARG
The environment variable is PYOCD_DAPARG, not PYOCD_DAPARG_ARG.

Signed-off-by: Marti Bolivar <marti.bolivar@linaro.org>
2017-10-18 15:24:38 -04:00
Andrew Boie
877f82e847 userspace: add K_THREAD_ACCCESS_GRANT()
It's possible to declare static threads that start up as K_USER,
but these threads can't do much since they start with permissions on
no kernel objects other than their own thread object.

Rather than do some run-time synchronization to have some other thread
grant the necessary permissions, we introduce macros
to conveniently assign object permissions to these threads when they
are brought up at boot by the kernel. The tables generated here
are constant and live in ROM when possible.

Example usage:

K_THREAD_DEFINE(my_thread, STACK_SIZE, my_thread_entry,
                NULL, NULL, NULL, 0, K_USER, K_NO_WAIT);

K_THREAD_ACCESS_GRANT(my_thread, &my_sem, &my_mutex, &my_pipe);

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2017-10-18 07:37:38 -07:00
Anas Nashif
981f77f756 sanitycheck: fix extra_configs with multiple values
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2017-10-18 08:33:03 -04:00
Anas Nashif
ebc329dc30 doc: document extra_configs option for testcases
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2017-10-17 14:13:52 -04:00
Anas Nashif
fa695d2b9e sanitycheck: add extra_configs for testing with multiple values
Support new keywords in testcase.yaml that would allow us to inject
configuration options to be merged with default configuration instead of
having to provide a prj.conf for each variant of the test which is very
difficult to keep in sync.  Sanitycheck script will create an overlay
file that is merged during the build process.

This is now done using the extra_configs option which is a yaml list of
option with the values, for example:

 extra_configs:
   - CONFIG_XXXX=y
   - CONFIG_YYYY=y

With this option we can have multiple tests that for example run on
hardware with different values. This type of testing is good on HW but
it does not make sense to be built in normal sanitycheck operation
because it will be just rebuilding the same code with different values.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2017-10-17 08:55:38 -04:00
Andrew Boie
bca15da650 userspace: treat thread stacks as kernel objects
We need to track permission on stack memory regions like we do
with other kernel objects. We want stacks to live in a memory
area that is outside the scope of memory domain permission
management. We need to be able track what stacks are in use,
and what stacks may be used by user threads trying to call
k_thread_create().

Some special handling is needed because thread stacks appear as
variously-sized arrays of struct _k_thread_stack_element which is
just a char. We need the entire array to be considered an object,
but also properly handle arrays of stacks.

Validation of stacks also requires that the bounds of the stack
are not exceeded. Various approaches were considered. Storing
the size in some header region of the stack itself would not allow
the stack to live in 'noinit'. Having a stack object be a data
structure that points to the stack buffer would confound our
current APIs for declaring stacks as arrays or struct members.
In the end, the struct _k_object was extended to store this size.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2017-10-16 19:02:00 -07:00
Andrew Boie
c5354552ce gen_syscall_header: use compiler barrier
We need to enforce that if the implementation function is inlined,
and we are using a syscall declaration macro where a runtime check
is performed, that all memory access in the inlined implementation
function is done after the user context check is performed.

Fixes bad memory access issues observed due to the compiler fetching
member data from a kernel object when the calling context was in
user mode.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2017-10-16 16:16:56 -07:00
Andrew Boie
ac1db4f08a sanitycheck: add last_sanity.xml to gitignore
This file is otherwise polluting the tree after doing a
sanitycheck run.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2017-10-14 19:22:55 -04:00
Andrew Boie
47f8fd1d4d kernel: add K_INHERIT_PERMS flag
By default, threads are created only having access to their own thread
object and nothing else. This new flag to k_thread_create() gives the
thread access to all objects that the parent had at the time it was
created, with the exception of the parent thread itself.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2017-10-13 12:17:13 -07:00
Marti Bolivar
a645257988 fix "scripts: zephyr_flash_debug: flash like dfuutil.sh"
- When flashing with dfu-util while alt is not a number, the name must
  be quoted.

- Add missing commas in self.list_pattern

- Always call dfu-util with the VID/PID

Fixes: 257fa4af9 ("scripts: zephyr_flash_debug: flash like dfuutil.sh")
Signed-off-by: Marti Bolivar <marti.bolivar@linaro.org>
2017-10-13 12:50:40 -04:00
Marti Bolivar
73d560e412 fix "scripts: zephyr_flash_debug: flash like openocd.sh"
All commands need to have quotation marks stripped.

Fixes: bee6f9e73 ("scripts: zephyr_flash_debug: flash like openocd.sh")
Signed-off-by: Marti Bolivar <marti.bolivar@linaro.org>
2017-10-13 12:50:40 -04:00
Marti Bolivar
b1e6ea27a7 scripts: zephyr_flash_debug: flash like arc_debugger.sh
Only tested by comparing commands that would be run.

Signed-off-by: Marti Bolivar <marti.bolivar@linaro.org>
2017-10-12 23:53:20 -07:00
Marti Bolivar
cdffad214f scripts: zephyr_flash_debug: flash like nios2.sh
Only tested by comparing commands that would be run.

Signed-off-by: Marti Bolivar <marti.bolivar@linaro.org>
2017-10-12 23:53:20 -07:00
Marti Bolivar
bee6f9e73e scripts: zephyr_flash_debug: flash like openocd.sh
Add support for flashing targets compatible with openocd.sh.

Tested on disco_l475_iot1.

Signed-off-by: Marti Bolivar <marti.bolivar@linaro.org>
2017-10-12 23:53:20 -07:00
Marti Bolivar
61aa0e06f5 scripts: zephyr_flash_debug: flash like nrf_flash.sh
Add support for flashing targets compatible with nrf_flash.sh.

Tested on nrf52840_pca10056.

Signed-off-by: Marti Bolivar <marti.bolivar@linaro.org>
2017-10-12 23:53:20 -07:00
Marti Bolivar
1126523396 scripts: zephyr_flash_debug: flash like esp32.sh
Add support for flashing targets compatible with esp32.sh.

Only tested by comparing commands that would be run.

Signed-off-by: Marti Bolivar <marti.bolivar@linaro.org>
2017-10-12 23:53:20 -07:00
Marti Bolivar
289c0f0304 scripts: zephyr_flash_debug: flash like bossa-flash.sh
Add support for flashing targets compatible with bossa-flash.sh.

This is something of a bug-for-bug reimplementation, as the existing
flashing script makes some potentially unsafe assumptions.

Only tested by comparing commands that would be run.

Signed-off-by: Marti Bolivar <marti.bolivar@linaro.org>
2017-10-12 23:53:20 -07:00
Marti Bolivar
4bfbe251ea scripts: zephyr_flash_debug: flash like pyocd.sh
Add support for flashing targets compatible with pyocd.sh.

Tested on 96b_nitrogen, nrf52_blenano2, and frdm_k64f, with and
without PYOCD_BOARD_ID. Additionally, frdm_k64f was tested with
PYOCD_DAPARG_ARG='limit_packets=True'.

Signed-off-by: Marti Bolivar <marti.bolivar@linaro.org>
2017-10-12 23:53:20 -07:00
Marti Bolivar
257fa4af90 scripts: zephyr_flash_debug: flash like dfuutil.sh
Add support for flashing targets compatible with dfuutil.sh.

Tested on 96b_carbon.

Signed-off-by: Marti Bolivar <marti.bolivar@linaro.org>
2017-10-12 23:53:20 -07:00
Marti Bolivar
113ee65b89 build: interpose zephyr_flash_debug.py in flash target
Zephyr board flashing and debugging is done via shell scripts. It
would improve the CMake transition to remove the shell dependency.

Add zephyr_flash_debug.py to allow phasing out the shell scripts.
This takes two arguments:

- a command (eventually flash, debug, and debugserver, but just flash
  for now)

- the path to the corresponding shell script

zephyr_flash_debug.py runs the command in pure Python if it
knows how. Otherwise, it falls back on the shell script. In
this patch, it always falls back. Subsequent patches add support
for existing flash backends.

Invoke zephyr_flash_debug.py from the Makefile flash target, but only
if USE_ZEPHYR_FLASH_DEBUG_SHELL is empty. This lets users keep existing
behavior in case of issues, and can be removed later once the Python
script is more widely tested.

Signed-off-by: Marti Bolivar <marti.bolivar@linaro.org>
2017-10-12 23:53:20 -07:00
Andrew Boie
3ff41b9484 kernel: allow system call with 64-bit return val
This is subject to the constraint that such system calls must have a
return value which is "u64_t" or "s64_t".

So far all the relevant kernel calls just have zero or one arguments,
we can later add more _syscall_ret64_invokeN() APIs as needed.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2017-10-12 16:25:00 -07:00
Anas Nashif
2516d6e509 toolchains: fix xtools configurations
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2017-10-12 16:27:12 -05:00
Anas Nashif
7bd98a09d5 ci: compliance: decode output to utf8
Fixes GH-1580.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2017-10-12 08:47:14 -04:00
Andrew Boie
6093a94454 gen_kobject_list: fix on ARM targets
On ARM, a zero memory address actually falls within the expected bounds
of kernel memory.

Move the NULL check outside the bounds check, so that kernel objects
with NULL memory addresses in the DWARF info (because gc-sections
discarded them) won't confound the script's logic.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2017-10-11 15:39:08 -07:00
Anas Nashif
86c8e2330d sanitycheck: fix support for unit tests
When we moved to yaml, we missed porting the 'unit' architecture. This
is now being added again.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2017-10-11 09:21:05 -05:00
Anas Nashif
7fae29c711 sanitycheck: inifile -> yamlfile
because we do not use ini files anymore, to avoid confusion, rename this
to be yamlfile, which is the format we use for testcases now.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2017-10-11 09:21:05 -05:00
Andrew Boie
a949b50fe7 syscalls: rename __ZEPHYR_KERNEL__
__ZEPHYR_SUPERVISOR__ more accurately represents what this means: that
the code is intended for scenarios when the CPU is expected to be
running in supervisor (privileged) mode. This could be in the kernel or
in the application.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2017-10-10 22:42:06 -04:00
Maureen Helm
9c390d7655 scripts: Add support for passing daparg argument to pyocd commands
pyocd occasionally throws USB timeout exceptions when running in
VirtualBox, and recently added a command-line option to limit the USB
packet count as a workaround. Introduce an environment variable
PYOCD_DAPARG so Zephyr can pass the argument to pyocd. For example:

$ make BOARD=frdm_k64f PYOCD_DAPARG='limit_packets=True' flash

This workound comes with a performance penalty when flashing and
debugging with pyocd, so it should only be used when running pyocd in
VirtualBox.

Signed-off-by: Maureen Helm <maureen.helm@nxp.com>
2017-10-10 11:56:47 -05:00
Anas Nashif
a8d259939b checkpatch: define typedefsfile to deal with a few false positives
checkpatch expects typedefs to be suffixed with _t and has different
rules when typedefs are being used as arguments of a function. This
seems to be a known issue and defining typedefs in a file resolves this
issue.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2017-10-09 10:43:05 -04:00
Andrew Boie
6cdc8e90a1 xtools: fix some build issues
$DTC wasn't exported, causing a build failure if it wasn't
defined outside the build system.

The provided ct-ng configuration files define
CT_TARGET_VENDOR="zephyr". Fix CROSS_COMPILE definition so
that the compiler can be found.

Change-Id: I4e25c775e1f02a435704b6a874adb221c677b13a
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2017-10-09 09:10:39 -04:00
Andrew Boie
c74983e8b4 kernel: remove some kernel objects from tracking
These are removed as the APIs that use them are not suitable for
exporting to userspace.

- Kernel workqueues run in supervisor mode, so it would not be
appropriate to allow user threads to submit work to them. A future
enhancement may extend or introduce parallel API where the workqueue
threads may run in user mode (or leave as an exercise to the user).

- Kernel slabs store private bookkeeping data inside the
user-accessible slab buffers themselves. Alternate APIs are planned
here for managing slabs of kernel objects, implemented within the
runtime library and not the kernel.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2017-10-07 10:45:15 -07:00
Andrew Boie
990bf16206 kernel: abolish __syscall_inline
This used to exist because in earlier versions of the system call
interfaces, an "extern" declaration of the system call implementation
function would precede the real inline version of the implementation.
The compiler would not like this and would throw "static declaration
of ‘foo’ follows non-static declaration". So alternate macros were
needed which declare the implementation function as 'static inline'
instead of extern.

However, currently the inline version of these system call
implementations appear first, the K_SYSCALL_DECLARE() macros appear in
the header generated by gen_syscalls.py, which is always included at the
end of the header file. The compiler does not complain if a
static inline function is succeeded by an extern prototype of the
same function. This lets us simplify the generated system call
macros and just use __syscall everywhere.

The disassembly of this was checked on x86 to ensure that for
kernel-only or CONFIG_USERSPACE=n scenarios, everything is still being
inlined as expected.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2017-10-03 16:16:03 -04:00
Paul Sokolovsky
94620bdb0b scripts: gen_syscalls: Use explicit encoding when reading headers
In Python, if open() doesn't specify "encoding" parameter,
locale.getpreferredencoding(False) will be used as the default,
as explained in
https://docs.python.org/3/library/functions.html#open ,
which may differ from system to system. So, explicitly specify
"encoding" param in open() call.

Also, fix a typo in a comment.

Signed-off-by: Paul Sokolovsky <paul.sokolovsky@linaro.org>
2017-10-03 10:07:41 -07:00