The core kernel z_setup_new_thread() calls into arch_new_thread(),
which calls back into the core kernel via z_new_thread_init().
Move everything that doesn't have to be in z_new_thread_init() to
z_setup_new_thread() and convert to an inline function.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Runtime initialization failed to reset the lock field, causing
problems when the pipe object is located on a stack and passed by
reference to other code. Lacking an API for initializing a spinlock
by itself use the idiom from _K_PIPE_INITIALIZER().
To simplify maintainability the initialization order is changed
slightly to match the structure field declaration order.
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
Legacy code can switch back to the original implementation where it
needs it, but we don't want new code to be unintentionally dependent
on the behavior of the older allocator. The new one is a better
general purpose choice.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Add a shim layer implementing the legacy k_mem_pool APIs backed by a
k_heap instead of the original implementation.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
This adds a k_heap data structure, a synchronized wrapper around a
sys_heap memory allocator. As of this patch, it is an alternative
implementation to k_mem_pool() with somewhat better efficiency and
performance and more conventional (and convenient) behavior.
Note that commit involves some header motion to break dependencies.
The declaration for struct k_spinlock moves to kernel_structs.h, and a
bunch of includes were trimmed.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Almost all of the k_mem_pool API is implemented in terms of three
lower level primitives: K_MEM_POOL_DEFINE(), k_mem_pool_alloc() and
k_mem_pool_free_id(). These are themselves implemented on top of the
lower level sys_mem_pool abstraction.
Make this layering explicit by splitting the low level out into its
own files: mempool_sys.c/h.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Replace CONFIG_ENTROPY_NAME with DT_CHOSEN_ZEPHYR_ENTROPY_LABEL. We now
set zephyr,entropy in the chosen node of the device tree to the entropy
device.
This allows us to remove CONFIG_ENTROPY_NAME from dts_fixup.h. Also
remove any other stale ENTROPY related defines in dts_fixup.h files.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Those are used only in tests, so remove them from kernel Kconfig and set
them in the tests that use them directly.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Add a call to get the system tick count as an official API (and
redefine the existing millisecond API in terms of it). Sophisticated
applications need to be able to count ticks directly, and the newer
timeout API supports that. Uptime should too, for symmetry.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Add tick-based (i.e. precision resistant) inspection APIs for kernel
timeouts visible via k_timer, k_delayed work and thread timeouts
(i.e. pended/sleeping threads). These are each available in
"remaining" and "expires" variants returning time values relative to
current time and system start. All have system calls where applicable
(i.e. everywhere but k_delayed_work, which is not a userspace API)
The pre-existing millisecond "remaining_get()" predicates for timer
and delayed work remain, but are expressed in terms of the newer
calls.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Add support for "absolute" timeouts, which are expressed relative to
system uptime instead of deltas from current time. These allow for
more race-resistant code to be written by allowing application code to
do a single timeout computation, once, and then reuse the timeout
value even if the thread wakes up and needs to suspend again later.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Add a CONFIG_TIMEOUT_64BIT kconfig that, when selected, makes the
k_ticks_t used in timeout computations pervasively 64 bit. This will
allow much longer timeouts and much faster (i.e. more precise) tick
rates. It also enables the use of absolute (not delta) timeouts in an
upcoming commit.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Add a k_timeout_t type, and use it everywhere that kernel API
functions were accepting a millisecond timeout argument. Instead of
forcing milliseconds everywhere (which are often not integrally
representable as system ticks), do the conversion to ticks at the
point where the timeout is created. This avoids an extra unit
conversion in some application code, and allows us to express the
timeout in units other than milliseconds to achieve greater precision.
The existing K_MSEC() et. al. macros now return initializers for a
k_timeout_t.
The K_NO_WAIT and K_FOREVER constants have now become k_timeout_t
values, which means they cannot be operated on as integers.
Applications which have their own APIs that need to inspect these
vs. user-provided timeouts can now use a K_TIMEOUT_EQ() predicate to
test for equality.
Timer drivers, which receive an integer tick count in ther
z_clock_set_timeout() functions, now use the integer-valued
K_TICKS_FOREVER constant instead of K_FOREVER.
For the initial release, to preserve source compatibility, a
CONFIG_LEGACY_TIMEOUT_API kconfig is provided. When true, the
k_timeout_t will remain a compatible 32 bit value that will work with
any legacy Zephyr application.
Some subsystems present timeout (or timeout-like) values to their own
users as APIs that would re-use the kernel's own constants and
conventions. These will require some minor design work to adapt to
the new scheme (in most cases just using k_timeout_t directly in their
own API), and they have not been changed in this patch, instead
selecting CONFIG_LEGACY_TIMEOUT_API via kconfig. These subsystems
include: CAN Bus, the Microbit display driver, I2S, LoRa modem
drivers, the UART Async API, Video hardware drivers, the console
subsystem, and the network buffer abstraction.
k_sleep() now takes a k_timeout_t argument, with a k_msleep() variant
provided that works identically to the original API.
Most of the changes here are just type/configuration management and
documentation, but there are logic changes in mempool, where a loop
that used a timeout numerically has been reworked using a new
z_timeout_end_calc() predicate. Also in queue.c, a (when POLL was
enabled) a similar loop was needlessly used to try to retry the
k_poll() call after a spurious failure. But k_poll() does not fail
spuriously, so the loop was removed.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Replace all occurences of BUILD_ASSERT_MSG() with BUILD_ASSERT()
as a result of merging BUILD_ASSERT() and BUILD_ASSERT_MSG().
Signed-off-by: Oleg Zhurakivskyy <oleg.zhurakivskyy@intel.com>
This adds a sys init level which allows device and sys_init
to be done after SMP initialization, z_smp_init(), when all
cores are up and running.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Don't generate a Z_OOPS() if k_thread_abort() is called on a
thread that isn't running. Just return to the caller instead,
much like how k_thread_join() functions.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
This reverts commit 8739517107.
Pull Request #23437 was merged by mistake with an invalid manifest.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Replace all occurences of BUILD_ASSERT_MSG() with BUILD_ASSERT()
as a result of merging BUILD_ASSERT() and BUILD_ASSERT_MSG().
Signed-off-by: Oleg Zhurakivskyy <oleg.zhurakivskyy@intel.com>
This never needed to be put in a separate gperf table.
Privilege mode stacks can be generated by the main
gen_kobject_list.py logic, which we do here.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Private type, internal to the kernel, not directly associated
with any k_object_* APIs. Is the return value of z_object_find().
Rename to struct z_object.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Rather than stuffing various values in a uintptr_t based on
type using casts, use a union for this instead.
No functional difference, but the semantics of the data member
are now much clearer to the casual observer since it is now
formally defined by this union.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
The set of interrupt stacks is now expressed as an array. We
also define the idle threads and their associated stacks this
way. This allows for iteration in cases where we have multiple
CPUs.
There is now a centralized declaration in kernel_internal.h.
On uniprocessor systems, z_interrupt_stacks has one element
and can be used in the same way as _interrupt_stack.
The IRQ stack for CPU 0 is now set in init.c instead of in
arch code.
The extern definition of the main thread stack is now removed,
this doesn't need to be in a header.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
z_swap_unlocked() does the same construction of using a
dummy spinlock; just use that and make the code simpler.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
z_reschedule_unlocked() is a no-op if the caller is
cooperative, because the logic that maintains the ready queue
ensures that the co-op thread is always at the front unless
some special handling is done like in k_yield(), which does
not happen here.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Add iterating over the static threads for k_thread_foreach and
k_thread_foreach_unlocked iterator functions
Signed-off-by: Joakim Andersson <joakim.andersson@nordicsemi.no>
We want to be regression-testing the spurious ISR functionality.
Therefore, in z_fatal_error() we need to allow a test to continue
if an error has occured due to a spurious IRQ being triggered.
Only in test mode, wee allow the function to return without an
error. In normal mode the current thread will be aborted.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
For architectures that support detection of nested interrupts,
we need to check the validity of the exception stack frame,
before we can supply it as a pointer to the function that
evaluates whether we are in a nested interrupt context. This
commits adds the required esf pointer checks in z_fatal_error().
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
The lock in kernel/thread.c was pulling double-duty, protecting
both the thread monitor linked list and also serializing access
to k_thread_suspend/resume functions.
The monitor list now has its own dedicated lock.
The object tracing test has been updated to use k_thread_foreach().
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
This function had a to sys_rand_get() even without random source. As
Zephyr is built with linkage garbage collection and this function is
called only if either ENTROPY_HAS_DRIVER or TEST_RANDOM_GENERATOR is
enabled and these options automatically enable a random source.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
This logic should be using the sched_lock and not its own
separate lock for these two functions.
Some simplications were made; z_thread_single_resume and
z_thread_single_suspend were only used in one place, and there was
some redundant logic for whether to reschedule in the suspend case.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Don't pretend with have stack randomization without multithreading.
When multithreading is disabled the "main" thread never starts. Zephyr
will run on the stack used for the z_cstart(), which on most
architectures is the interrupt stack.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
This assertion, if built in, allows users threads to crash
the kernel in a critical section by passing a negative timeout
value, creating a DoS attack vector.
Remove this assertion, immediately below it there's a check
which just resets it to 0 anyway.
Fixes: #22999
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
A recent patch allowed an error code to be returned even though the
execution path treated it as a non-error condition. Clear the code
before returning.
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
The recent synchronization work required that the kernel guarantee
switch_handle is non-null, but it did it in a way that works for ARC
and x86_64 but would clobber the work xtensa had already done to
populate that field.
There's no point: just make this an assert, as it's always been the
arch layer's job.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
This is aligned with the documentation which states that an error shall
be returned if the work has been completed:
'-EINVAL Work item is being processed or has completed its work.'
Though in order to be able to resubmit from the handler itself it needs
to be able to distinct when the work is already completed so instead of
-EINVAL it return -EALREADY when the work is considered to be completed.
Fixes#22803
Signed-off-by: Luiz Augusto von Dentz <luiz.von.dentz@intel.com>
We need to unlock IRQs in early return points of
z_fatal_error() functions; not only at the normal
return point.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
The syscall exception frame was stored on the CPU struct during
syscall execution, but that's not right. System calls might "feel
like" exceptions, but they're actually perfectly normal kernel mode
code and can be preempted and migrated between CPUs at any time.
Put the field on the thread struct.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
The code underneath z_fatal_error() (which is usually run in an
exception context, but is not required to be) was running with
interrupts enabled, which is a little surprising.
The only bug present currently is that the CPU ID extracted for
logging is subject to a race (i.e. it's possible but very unlikely
that such a handler might migrate to another CPU after the error is
flagged and log the wrong CPU ID), but in general users with custom
error handlers are likely to be surprised when their dying threads
gets preempted by other code before they can abort.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Use of the _current_cpu pointer cannot be done safely in a preemptible
context. If a thread is preempted and migrates to another CPU, the
old CPU record will be wrong.
Add a validation assert to the expression that catches incorrect
usages, and fix up the spots where it was wrong (most important being
a few uses of _current outside of locks, and the arch_is_in_isr()
implementation).
Note that the resulting _current expression now requires locking and
is going to be somewhat slower. Longer term it's going to be better
to augment the arch API to allow SMP architectures to implement a
faster "get current thread pointer" action than this default.
Note also that this change means that "_current" is no longer
expressible as an lvalue (long ago, it was just a static variable), so
the places where it gets assigned now assign to _current_cpu->current
instead.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
The existing stack_analyze APIs had some problems:
1. Not properly namespaced
2. Accepted the stack object as a parameter, yet the stack object
does not contain the necessary information to get the associated
buffer region, the thread object is needed for this
3. Caused a crash on certain platforms that do not allow inspection
of unused stack space for the currently running thread
4. No user mode access
5. Separately passed in thread name
We deprecate these functions and add a new API
k_thread_stack_space_get() which addresses all of these issues.
A helper API log_stack_usage() also added which resembles
STACK_ANALYZE() in functionality.
Fixes: #17852
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
These arch_timing_ defines get used in certain timer
drivers and need to be in the public include space,
and not the private kernel headers.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
These got dropped by an earlier patch, but are required on SMP systems
so synchronously notify other CPUs of changed scheduler state.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
This got clobbered by commit adac4cbafa in what I think was a rebase
mistake. Without it, on SMP systems it's possible to select a new
_current thread and try to return into it before another CPU has
actually finished switching away from it.
Interestingly: the frequency with which this bug got caught once it
was reintroduced was much, much higher than it was when it was fixed
the first time due to the instruction pointer poisoning introduced in
the interrim. Incompletely saved threads now have deliberately broken
state when assertions are enabled and will panic synchronously.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Similar to the suspend refactoring earlier, this really nees to be
done in an atomic block. There were two confirmable races here,
though it's not completely clear either was being hit in practice:
1. The bit operations in z_mark_thread_as_started() aren't atomic so
it needs to be protected.
2. The intermediate state in z_ready_thread() could result in a dead
or suspended thread being added to the ready queue if another
context tried a simultaneous abort or suspend.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Kernel wait_q's and the thread pended_on backpointer are scheduler
state and need to be modified under the scheduler lock. There was one
spot in pend() where they were not.
Also unpack z_remove_thread_from_ready_q() into an unsynchronized
utility so that it can be called by this process in a single lock
block.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
This had the same race that queue did: you have to be 100% done with
state management before calling z_ready_thread(), because another CPU
can pick up the thread before the return value was set.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Under SMP, when a thread is marked aborting, this thread may still
be running on another CPU. However, if there is only one thread
available to run, this thread may be selected to run again due to
next_up() not checking for the aborting state. Moreover, when
there is no IPI to signal to others k_thread_abort() being called,
the k_thread_abort() target thread is marked dead after a new
thread is selected to run. This causes the original thread calling
k_thread_abort() to mistaken that target thread is no longer
running and returns.
Note that, with working IPI, z_sched_ipi() is called as an ISR
to mark the target thread dead. A new thread is then selected to
run, so that the target thread would not be selected due to it
being dead.
This moves the code to mark thread dead into next_up(), where
the next best thread is selected, and the current thread being
swapped out. z_sched_ipi() now becomes an empty function, and
calls to it are removed.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Only dump data when we are interested in the analysing coverage. By
default just collect the data.
CONFIG_COVERAGE_DUMP is used to control this behaviour.
This will help speed up sanitycheck and will avoid lots of noise in the
log when some tests with coverage enabled failed. Dumping data to
console is also suspected to be one of the reason why qemu hangs in CI.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
_THREAD_PRESTART means the thread was not started yet and is being
setup, for example this is the case when starting a thread with a
timeout. We do not have a 'restart' thread state.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Before C sources can be compiled any generated header that they
include must be generated. Currently, the target 'offsets_h' happens
to depend directly or indirectly on all generated headers.
This means that to compile safely, one can simply depend on
'offsets_h'. But this is coincidental and might not be true in the
future.
To be able to safely depend on a target that represents all generated
headers being ready we introduce the target
'zephyr_generated_headers'.
Any third-party build scripts can now safely depend on
'zephyr_generated_headers' and be protected from any internal changes
to the build system, like the removal of offsets_h.
Signed-off-by: Sebastian Bøe <sebastian.boe@nordicsemi.no>
Fixes an issue where calling z_thread_malloc() would
borrow the resource pool of whatever thread happened
to be interrupted at the time.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Device config structure is placed in rom section but there was
no const prefix used. Lack of prefix suggested that structure
is in ram (ram_report is also fooled). Added const prefix to
explicitly inform that it goes to rom.
Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>
Calling z_ready_thread() means the thread is now ready and can wake up
at any moment on another CPU. But we weren't finished setting the
return value! So the other side could wake up with a spurious "error"
condition if it ran too soon. Note that on systems with a working
IPI, that wakeup can happen much faster than you might think.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
On SMP, there is an inherent race when swapping: the old thread adds
itself back to the run queue before calling into the arch layer to do
the context switch. The former is properly synchronized under the
scheduler lock, and the later operates with interrupts locally
disabled. But until somewhere in the middle of arch_switch(), the old
thread (that is in the run queue!) does not have complete saved state
that can be restored.
So it's possible for another CPU to grab a thread before it is saved
and try to restore its unsaved register contents (which are garbage --
typically whatever state it had at the last interrupt).
Fix this by leveraging the "swapped_from" pointer already passed to
arch_switch() as a synchronization primitive. When the switch
implementation writes the new handle value, we know the switch is
complete. Then we can wait for that in z_swap() and at interrupt
exit.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
It's possible for a thread to abort itself simultaneously with an
external abort from another thread. In fact in our test suite this is
a common thing, as ztest will abort its own spawend threads at the end
of a test, as they tend to be exiting on their own.
When that happens, the thread marks itself DEAD and does all its
scheduler bookeeping, but it is STILL RUNNING on its own stack until
it makes its way to its final swap. The external context would see
that "dead" metadata and return from k_thread_abort(), allowing the
next test to reuse and spawn the same thread struct while the old
context was still running. Obviously that's bad.
Unfortunately, this is impossible to address completely without
modifying every SMP architecture to add a API-visible hook to every
swap that signals completion. In practice the best we can do is add a
delay. But note the optimization: almost always, the scheduler IPI
catches the running thread and kills it from interrupt context
(i.e. on a different stack). When that happens, we know that the
interrupted thread will never be resumed (because it's dead) and can
elide the delay. We only pay the cost when we actually detect a race.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
These two spots were calling z_sched_ipi() (the IPI handler run under
the ISR, which is a noop here because obviously the current thread
isn't DEAD) and not arch_sched_ipi() (which triggers an IPI on other
CPUs to inform them of scheduling state changes), presumably because
of a typo.
Apparently we don't have tests for k_wakeup() and
k_thread_priority_set() that are sensitive to latency in SMP
contexts...
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
The original intent was that the output handle be written through the
pointer in the second argument, though not all architectures used that
scheme. As it turns out, that write is becoming a synchronization
signal, so it's no longer optional.
Clarify the documentation in arch_switch() about this requirement, and
add an instruction to the x86_64 context switch to implement it as
original envisioned.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Toggling this symbol probably doesn't make sense, because the
architecture is already known when Kconfig runs.
SCHED_IPI_SUPPORTED is enabled through being selected by the ARC_CONNECT
(maybe that one shouldn't be configurable either) and X86_64 symbols.
Note that it's not possible to disable the symbol when it's being
selected, so trying to turn it off on e.g. X86_64 won't work either.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Add runtime error handling for k_msgq_cleanup. We return 0 on success
now and -EAGAIN when cleanup is not possible.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Add runtime error checking to k_pipe_cleanup and k_pipe_get and remove
asserts.
Adapted test which was expecting a fault to handle errors instead.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Remove static helper functions used only once and integrate them into
calling functions.
In k_sem_take, return at the end.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Check for errors at runtime and stop depending on ASSERTs.
This changes the API for
- k_sem_init
k_sem_init now returns -EINVAL on invalid data.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
k_mutex_unlock will now perform error checking and return on failures.
If the current thread does not own the mutex, we will now return -EPERM.
In the unlikely situation where we own a lock and the lock count is
zero, we assert. This is considered an undefined bahviour and should not
happen.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Improve positioning of tracing calls. Avoid multiple calls and missing
events because of complex logix. Trace the event where things happen
really.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Historically, these routines were placed in thread.c and would use the
scheduler via exported, synchronized functions (e.g. "remove from
ready queue"). But those steps were very fine grained, and there were
races where the thread could be seen by other contexts (in particular
under SMP) in an intermediate state. It's not completely clear to me
that any of these were fatal bugs, but it's very hard to prove they
weren't.
At best, this is fragile. Move the z_thread_single_suspend/abort()
functions into the scheduler and do the scheduler logic in a single
critical section.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
The original implementation left this function hidden in init.h which
prevented it from showing up in documentation. Move it to kernel.h,
and document it consistent with the other functions that allow caller
customization based on context.
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
Allow caller to supply 0 events in which case the function just
does the sleep. This is useful so that the caller does not need
to create artificial events.
Signed-off-by: Jukka Rissanen <jukka.rissanen@linux.intel.com>
Same deal as in commit 41713244b3 ("kconfig: Remove '# Hidden' comments
on promptless symbols"). I forgot to do a case-insensitive search.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
We have been using thread, th and t for thread variables making the code
less readable, especially when we use t for timeouts and other time
related variables. Just use thread where possible and keep things
consistent.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Implement thread foreach processing with limited locking
to allow threads processing that may take more time but allows
missing some threads processing when the thread list is modified.
Signed-off-by: Radoslaw Koppel <radoslaw.koppel@nordicsemi.no>
SPIN_VALIDATE is, as it was previously, enabled per default when having
less than 4 CPUs and either having no flash or a flash size greater than
32kB.
Small targets, which needs to have asserts enabled, can chose to have
the spinlock validation enabled or not and thereby decide whether the
overhead added is acceptable or not.
Signed-off-by: Danny Oerndrup <daor@demant.com>
Fix a gap where k_sleep(K_FOREVER) could execute a code path that
would not verify that the call was not from interrupt context.
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
64-bit systems generate some compiler warnings about
data type sizes, use uintptr_t where int/u32_t was being cast
to void *.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
We need a size_t and not a u32_t for partition sizes,
for 64-bit compatibility.
Additionally, app_memdomain.h was also casting the base
address to a u32_t instead of a uintptr_t.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Builds of docs with doxygen 1.8.16 has a number of warnings of the form:
'warning: unbalanced grouping commands'. Fix those warnings be either
balancing the group command or removing it.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Device initialization may require use of generic services such as
starting up power rails, some of which may be controlled by GPIOs on
an external controller that can't be used until full kernel services
are available. Generic services can check k_is_in_isr() and mediate
their behavior that way, but currently have no way to determine that
the kernel is not available.
Provide a function that indicates whether initialization is still in
pre-kernel stages where no kernel services are available.
Signed-off-by: Peter A. Bigot <pab@pabigot.com>
Remove leading/trailing blank lines in .c, .h, .py, .rst, .yml, and
.yaml files.
Will avoid failures with the new CI test in
https://github.com/zephyrproject-rtos/ci-tools/pull/112, though it only
checks changed files.
Move the 'target-notes' target in boards/xtensa/odroid_go/doc/index.rst
to get rid of the trailing blank line there. It was probably misplaced.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
The k_mutex is a priority-inheriting mutex, so on unlock it's possible
that a thread's priority will be lowered. Make this a reschedule
point so that reasoning about thread priorities is easier (possibly at
the cost of performance): most users are going to expect that the
priority elevation stops at exactly the moment of unlock.
Note that this also reorders the code to fix what appear to be obvious
race conditions. After the call to z_ready_thread(), that thread may
be run (e.g. by an interrupt preemption or on another SMP core), yet
the return value and mutex weren't correctly set yet. The spinlock
was also prematurely released.
Fixes#20802
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Just use printk directly instead of going over defines.
For some reason, this change lets us pass on master when running
tests/kernel/timer/timer_monotonic test. This test started failing after
rc2 was tagged, just because the changing git version string passing to
BUILD_VERSION. This is still under investigation.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
When suspending a thread, cancel any pending timeouts which might wake
it up unexpectedly. Also, make suspending the current thread
(specifically) a schedule point, as callers are clearly going to
expect that to be synchronous.
Also fix a documentation weirdness. The phrasing in the earlier docs
for k_thread_suspend() was confusing: it could be interpreted as
either document the current (essentially buggy) behavior that threads
will "wake up" due to preexisting timeouts, OR to mean that thread
timeouts will continue to be tracked so that resuming a thread that
was sleeping will continue to sleep until the timeout (something that
has never been implemented: k_sleep() is implemented on top of
suspend). Rewrite to document what we actually implement.
Fixes#20033
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
In some platforms the size of size_t can be different of 4 bytes. Use
sys_rand_get to proper fill this variable.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
STACK_CANARIES relies on random value for the canarie so
ENTROPY_GENERATOR or TEST_RANDOM_GENERATOR needs to be
selected to get sys_rand32_get included in the build.
Fixes: #20587
Signed-off-by: David Leach <david.leach@nxp.com>
When a MetaIRQ preempts a cooperative thread, that thread would be
added back to the generic run queue. When the MetaIRQ is done, the
highest priority thread will be selected to run, which may obviously
be a cooperative thread of a higher priority than the one that was
preempted.
But that's wrong, because the original thread was promised that it
would NOT be preempted until it reached a scheduling point on its own
(that's the whole point of a cooperative thread, of course).
We need to track the thread that got preempted (one per CPU) and
return to it instead of whatever else the scheduler might have found.
Fixes#20255
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Add assert when negative (except K_FOREVER) is passed as timeout.
Add negative timeout correction to 0.
Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>
Previously, passing K_FOREVER to k_sleep() would return
immediately.
Forever is a long time. Even if woken up at some point,
we still had forever to sleep, so return K_FOREVER in this
case.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Entering irq_offload() on multiple CPUs can cause
difficult to debug/reproduce crashes. Demote irq_offload()
to non-inline (it never needed to be inline anyway) and
wrap the arch call in a semaphore.
Some tests which were unnecessarily killing threads
have been fixed; these threads exit by themselves anyway
and we won't leave the semaphore dangling.
The definition of z_arch_irq_offload() moved to
arch_interface.h as it only gets called by kernel C code.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Mark the old time conversion APIs deprecated, leave compatibility
macros in place, and replace all usage with the new API.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Promote the private z_arch_* namespace, which specifies
the interface between the core kernel and the
architecture code, to a new top-level namespace named
arch_*.
This allows our documentation generation to create
online documentation for this set of interfaces,
and this set of interfaces is worth treating in a
more formal way anyway.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
need to release spinlock first before busy_wait,
or other cores cannot get the spinlock when the holder is
busy waitting.
Signed-off-by: Wayne Ren <wei.ren@synopsys.com>
This commit modifies the z_new_thread_init function, that was
previously declared as ALWAYS_INLINE to be a normal function.
z_new_thread_init function is only called by the z_arch_new_thread
function and, since this is not a performance-critical function, there
is no good justification for inlining it.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
This commit refactors kernel and arch headers to establish a boundary
between private and public interface headers.
The refactoring strategy used in this commit is detailed in the issue
This commit introduces the following major changes:
1. Establish a clear boundary between private and public headers by
removing "kernel/include" and "arch/*/include" from the global
include paths. Ideally, only kernel/ and arch/*/ source files should
reference the headers in these directories. If these headers must be
used by a component, these include paths shall be manually added to
the CMakeLists.txt file of the component. This is intended to
discourage applications from including private kernel and arch
headers either knowingly and unknowingly.
- kernel/include/ (PRIVATE)
This directory contains the private headers that provide private
kernel definitions which should not be visible outside the kernel
and arch source code. All public kernel definitions must be added
to an appropriate header located under include/.
- arch/*/include/ (PRIVATE)
This directory contains the private headers that provide private
architecture-specific definitions which should not be visible
outside the arch and kernel source code. All public architecture-
specific definitions must be added to an appropriate header located
under include/arch/*/.
- include/ AND include/sys/ (PUBLIC)
This directory contains the public headers that provide public
kernel definitions which can be referenced by both kernel and
application code.
- include/arch/*/ (PUBLIC)
This directory contains the public headers that provide public
architecture-specific definitions which can be referenced by both
kernel and application code.
2. Split arch_interface.h into "kernel-to-arch interface" and "public
arch interface" divisions.
- kernel/include/kernel_arch_interface.h
* provides private "kernel-to-arch interface" definition.
* includes arch/*/include/kernel_arch_func.h to ensure that the
interface function implementations are always available.
* includes sys/arch_interface.h so that public arch interface
definitions are automatically included when including this file.
- arch/*/include/kernel_arch_func.h
* provides architecture-specific "kernel-to-arch interface"
implementation.
* only the functions that will be used in kernel and arch source
files are defined here.
- include/sys/arch_interface.h
* provides "public arch interface" definition.
* includes include/arch/arch_inlines.h to ensure that the
architecture-specific public inline interface function
implementations are always available.
- include/arch/arch_inlines.h
* includes architecture-specific arch_inlines.h in
include/arch/*/arch_inline.h.
- include/arch/*/arch_inline.h
* provides architecture-specific "public arch interface" inline
function implementation.
* supersedes include/sys/arch_inline.h.
3. Refactor kernel and the existing architecture implementations.
- Remove circular dependency of kernel and arch headers. The
following general rules should be observed:
* Never include any private headers from public headers
* Never include kernel_internal.h in kernel_arch_data.h
* Always include kernel_arch_data.h from kernel_arch_func.h
* Never include kernel.h from kernel_struct.h either directly or
indirectly. Only add the kernel structures that must be referenced
from public arch headers in this file.
- Relocate syscall_handler.h to include/ so it can be used in the
public code. This is necessary because many user-mode public codes
reference the functions defined in this header.
- Relocate kernel_arch_thread.h to include/arch/*/thread.h. This is
necessary to provide architecture-specific thread definition for
'struct k_thread' in kernel.h.
- Remove any private header dependencies from public headers using
the following methods:
* If dependency is not required, simply omit
* If dependency is required,
- Relocate a portion of the required dependencies from the
private header to an appropriate public header OR
- Relocate the required private header to make it public.
This commit supersedes #20047, addresses #19666, and fixes#3056.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
We need to pass system call args using a register-width
data type and not hard-code this to u32_t.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Use this short header style in all Kconfig files:
# <description>
# <copyright>
# <license>
...
Also change all <description>s from
# Kconfig[.extension] - Foo-related options
to just
# Foo-related options
It's clear enough that it's about Kconfig.
The <description> cleanup was done with this command, along with some
manual cleanup (big letter at the start, etc.)
git ls-files '*Kconfig*' | \
xargs sed -i -E '1 s/#\s*Kconfig[\w.-]*\s*-\s*/# /'
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Clean up space errors and use a consistent style throughout the Kconfig
files. This makes reading the Kconfig files more distraction-free, helps
with grepping, and encourages the same style getting copied around
everywhere (meaning another pass hopefully won't be needed).
Go for the most common style:
- Indent properties with a single tab, including for choices.
Properties on choices work exactly the same syntactically as
properties on symbols, so not sure how the no-indentation thing
happened.
- Indent help texts with a tab followed by two spaces
- Put a space between 'config' and the symbol name, not a tab. This
also helps when grepping for definitions.
- Do '# A comment' instead of '#A comment'
I tweaked Kconfiglib a bit to find most of the stuff.
Some help texts were reflowed to 79 columns with 'gq' in Vim as well,
though not all, because I was afraid I'd accidentally mess up
formatting.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
In z_fatal_error() we invoke the arch-specific API that
evaluates whether we are in a nested exception. We then
use the result to log a message that the error occurred
in ISR. In non-test mode, we unconditionally panic, if
an exception has occurred in an ISR and the fatal error
handler has not returned (apart from the case of an
error in stack sentinel check).
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
Duplicate definitions elsewhere have been removed.
A couple functions which are defined by the arch interface
to be non-inline, but were implemented inline by native_posix
and intel64, have been moved to non-inline.
Some missing conditional compilation for z_arch_irq_offload()
has been fixed, as this is an optional feature.
Some massaging of native_posix headers to get everything
in the right scope.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
The intel64 switch implementation doesn't actually use a switch handle
per se, just the raw thread struct pointers which get stored into the
handle field. This works fine for normally initialized threads, but
when switching out of a dummy thread at initialization, nothing has
initialized that field and the code was dumping registers into the
bottom of memory through the resulting NULL pointer.
Fix this by skipping the load of the field value and just using an
offset instead to get the struct address, which is actually slightly
faster anyway (a SUB immediate instruction vs. the load).
Actually for extra credit we could even move the switch_handle field
to the top of the thread struct and eliminate the instruction
entirely, though if we did that it's probably worth adding some
conditional code to make the switch_handle field disappear entirely.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
include/sys/arch_inlines.h will contain all architecture APIs
that are used by public inline functions and macros,
with implementations deriving from include/arch/cpu.h.
kernel/include/arch_interface.h will contain everything
else, with implementations deriving from
arch/*/include/kernel_arch_func.h.
Instances of duplicate documentation for these APIs have been
removed; implementation details have been left in place.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Re-run with updated script to convert integer literal delay arguments
to k_thread_create and K_THREAD_DEFINE to use the standard timeout
macros.
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
This maximum is implicit in the kernel support for SMP, e.g.,
kernel/init.c and kernel/smp.c assume CONFIG_MP_NUM_CPUS <= 4.
Signed-off-by: Charles E. Youse <charles.youse@intel.com>
A loop in k_mem_pool_alloc() around z_sys_mem_pool_block_alloc() assumes
the later may return -EAGAIN with an elaborate comment about it. But
-EAGAIN is no longer returned by that function since commit 7845e1b01e
("lib/mempool: Fix spurious -ENOMEM due to agressive latency control").
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
This commit adds new k_work_poll interface. It allows to
submit given work to a workqueue automatically when one of the
watched pollable objects changes its state.
Signed-off-by: Piotr Zięcik <piotr.ziecik@nordicsemi.no>
Signed-off-by: Peter A. Bigot <pab@pabigot.com>
This commit separates k_poll() infrastructure from k_poll() API
implementation, allowing other (future) API calls to use the same
framework.
Signed-off-by: Piotr Zięcik <piotr.ziecik@nordicsemi.no>
Remove FUNC_NORETURN attribute from _StackCheckHandler to address the
following warning from gcc-9.2:
kernel/compiler_stack_protect.c:62:32: error: '__stack_chk_fail'
specifies less restrictive attribute than its target
'_StackCheckHandler': 'noreturn' [-Werror=missing-attributes]
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
We now define z_is_idle_thread_object() in ksched.h,
and the repeated definitions of a function that does
the same thing now changed to just use the common
definition.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
This takes an entry point and not a thread as argument.
Rename to z_is_idle_thread_entry() to make this clearer.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
The main and idle threads, and their associated stacks,
were being referenced in various parts of the kernel
with no central definition. Expose these in kernel_internal.h
and namespace with z_ appropriately.
The main and idle threads were being defined statically,
with another variable exposed to contain their pointer
value. This wastes a bit of memory and isn't accessible
to user threads anyway, just expose the actual thread
objects.
Redundance MAIN_STACK_SIZE and IDLE_STACK_SIZE defines
in init.c removed, just use the Kconfigs they derive
from.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
These are renamed to z_timestamp_main and z_timestamp_idle,
and now specified in kernel_internal.h.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
This is part of the core kernel -> architecture interface and
has been renamed z_arch_kernel_init().
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
z_set_thread_return_value is part of the core kernel -> arch
interface and has been renamed to z_arch_thread_return_value_set.
z_set_thread_return_value_with_data renamed to
z_thread_return_value_set_with_data for consistency.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
This is part of the core kernel -> architecture interface
and is appropriately renamed z_arch_is_in_isr().
References from test cases changed to k_is_in_isr().
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
This is part of the core kernel -> architecture interface
and should have a leading prefix z_arch_.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Global variables related to timing information have been
renamed to be prefixed with z_arch, with naming arranged
in increasing order of specificity.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
An #endif and the brace terminating a compound statement were
transposed, causing compilation errors with the above-specified
combination of configuration options.
Signed-off-by: Charles E. Youse <charles.youse@intel.com>
The callback function has been ignored in z_timeout_init() since the
timer rework in fall 2018. Passing real handlers to it in code is
distracting when they will be overridden by whatever callback is
provided in z_add_timeout().
As this function is an internal API deprecation is not necessary.
Remove the parameter and change all call sites to drop the argument.
Signed-off-by: Peter A. Bigot <pab@pabigot.com>
Corrected the define of SMP_FALLBACK to prevent llvm warning.
llvm issues a warning as the behaviour of using defined(x) inside a
macro expansion is undefined (https://reviews.llvm.org/D15866).
Signed-off-by: Jan Van Winkel <jan.van_winkel@dxplore.eu>
If an architecture declares support for IPI, we still want to use it
only when running in SMP mode.
(This also fixes a build failure on ARC, which declares
CONFIG_SCHED_IPI_SUPPORTED but doesn't actually implement
z_arch_sched_ipi() yet).
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
The timeout code has an optimization where it refuses to send a new
timeout to the driver unless it is sooner than one already scheduled.
This won't work on SMP, though, because the timeout value when
timeslicing is enabled depends on the current thread, and on SMP the
decision as to the next thread will not be made until later (when we
swap, or exit an interrupt).
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Now that we have a working IPI framework, there's no reason for the
default spin loop for the SMP idle thread. Just use the default
platform idle and send an IPI when a new thread is readied.
Long term, this can be optimized if necessary (e.g. only send the IPI
to idling CPUs, or check priorities, etc...), but for a 2-cpu system
this is a very reasonable default.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
