One of the stack sentinel policies was to check the sentinel
any time a cooperative context switch is done (i.e, _Swap is
called).
This was done by adding a hook to _check_stack_sentinel in
every arch's __swap function.
This way is cleaner as we just have the hook in one inline
function rather than implemented in several different assembly
dialects.
The check upon interrupt is now made unconditionally rather
than checking if we are calling __swap, since the check now
is only called on cooperative _Swap(). The interrupt is always
serviced first.
Issue: ZEP-2244
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
The kernel tracks time slice usage with the _time_slice_elapsed global.
Every time the timer interrupt goes off and the timer driver calls
_nano_sys_clock_tick_announce() with the elapsed time, this is added to
_time_slice_elapsed. If it exceeds the total time slice, the thread is
moved to the back of the queue for that priority level and
_time_slice_elapsed is reset to zero.
In a non-tickless kernel, this is the only time _time_slice_elapsed is
reset. If a thread uses up a partial time slice, and then cooperatively
switches to another thread, the next thread will inherit the remaining
time slice, causing it not to be able to run as long as it ought to.
There does exist code to properly reset the elapsed count, but it was
only compiled in a tickless kernel. Now it is built any time
CONFIG_TIMESLICING is enabled.
Issue: ZEP-2107
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
This places a sentinel value at the lowest 4 bytes of a stack
memory region and checks it at various intervals, including when
servicing interrupts or context switching.
This is implemented on all arches except ARC, which supports stack
bounds checking directly in hardware.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Adds event based scheduling logic to the kernel. Updates
management of timeouts, timers, idling etc. based on
time tracked at events rather than periodic ticks. Provides
interfaces for timers to announce and get next timer expiry
based on kernel scheduling decisions involving time slicing
of threads, timeouts and idling. Uses wall time units instead
of ticks in all scheduling activities.
The implementation involves changes in the following areas
1. Management of time in wall units like ms/us instead of ticks
The existing implementation already had an option to configure
number of ticks in a second. The new implementation builds on
top of that feature and provides option to set the size of the
scheduling granurality to mili seconds or micro seconds. This
allows most of the current implementation to be reused. Due to
this re-use and co-existence with tick based kernel, the names
of variables may contain the word "tick". However, in the
tickless kernel implementation, it represents the currently
configured time unit, which would be be mili seconds or
micro seconds. The APIs that take time as a parameter are not
impacted and they continue to pass time in mili seconds.
2. Timers would not be programmed in periodic mode
generating ticks. Instead they would be programmed in one
shot mode to generate events at the time the kernel scheduler
needs to gain control for its scheduling activities like
timers, timeouts, time slicing, idling etc.
3. The scheduler provides interfaces that the timer drivers
use to announce elapsed time and get the next time the scheduler
needs a timer event. It is possible that the scheduler may not
need another timer event, in which case the system would wait
for a non-timer event to wake it up if it is idling.
4. New APIs are defined to be implemented by timer drivers. Also
they need to handler timer events differently. These changes
have been done in the HPET timer driver. In future other timers
that support tickles kernel should implement these APIs as well.
These APIs are to re-program the timer, update and announce
elapsed time.
5. Philosopher and timer_api applications have been enabled to
test tickless kernel. Separate configuration files are created
which define the necessary CONFIG flags. Run these apps using
following command
make pristine && make BOARD=qemu_x86 CONF_FILE=prj_tickless.conf qemu
Jira: ZEP-339 ZEP-1946 ZEP-948
Change-Id: I7d950c31bf1ff929a9066fad42c2f0559a2e5983
Signed-off-by: Ramesh Thomas <ramesh.thomas@intel.com>
Future tickless kernel patches would be inserting some
code before call to Swap. To enable this it will create
a mcro named as the current _Swap which would call first
the tickless kernel code and then call the real __swap()
Jira: ZEP-339
Change-Id: Id778bfcee4f88982c958fcf22d7f04deb4bd572f
Signed-off-by: Ramesh Thomas <ramesh.thomas@intel.com>
This avoids asm files from having to explicitly define the _ASMLANGUAGE
symbol themselves.
Change-Id: I71f5a169f75d7443a58a0365a41c55b20dae3029
Signed-off-by: Benjamin Walsh <walsh.benj@gmail.com>
They are internal states, not user-facing.
Also prepend an underscore since they are kernel internal symbols.
Change-Id: I53740e0d04a796ba1ccc409b5809438cdb189332
Signed-off-by: Benjamin Walsh <walsh.benj@gmail.com>
Replace the existing Apache 2.0 boilerplate header with an SPDX tag
throughout the zephyr code tree. This patch was generated via a
script run over the master branch.
Also updated doc/porting/application.rst that had a dependency on
line numbers in a literal include.
Manually updated subsys/logging/sys_log.c that had a malformed
header in the original file. Also cleanup several cases that already
had a SPDX tag and we either got a duplicate or missed updating.
Jira: ZEP-1457
Change-Id: I6131a1d4ee0e58f5b938300c2d2fc77d2e69572c
Signed-off-by: David B. Kinder <david.b.kinder@intel.com>
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
These two fields in the thread structure control the preemptibility of a
thread.
sched_locked is decremented when the scheduler gets locked, which means
that the scheduler is locked for values 0xff to 0x01, since it can be
locked recursively. A thread is coop if its priority is negative, thus
if the prio field value is 0x80 to 0xff when looked at as an unsigned
value.
By putting them end-to-end, this means that a thread is non-preemptible
if the bundled value is greater than or equal to 0x0080. This is the
only thing the interrupt exit code has to check to decide to try a
reschedule or not.
Change-Id: I902d36c14859d0d7a951a6aa1bea164613821aca
Signed-off-by: Benjamin Walsh <benjamin.walsh@windriver.com>
Some thread fields were 32-bit wide, when they are not even close to
using that full range of values. They are instead changed to 8-bit fields.
- prio can fit in one byte, limiting the priorities range to -128 to 127
- recursive scheduler locking can be limited to 255; a rollover results
most probably from a logic error
- flags are split into execution flags and thread states; 8 bits is
enough for each of them currently, with at worst two states and four
flags to spare (on x86, on other archs, there are six flags to spare)
Doing this saves 8 bytes per stack. It also sets up an incoming
enhancement when checking if the current thread is preemptible on
interrupt exit.
Change-Id: Ieb5321a5b99f99173b0605dd4a193c3bc7ddabf4
Signed-off-by: Benjamin Walsh <benjamin.walsh@windriver.com>
This will allow for an enhancement when checking if the thread is
preemptible when exiting an interrupt.
Change-Id: If93ccd1916eacb5e02a4d15b259fb74f9800d6f4
Signed-off-by: Benjamin Walsh <benjamin.walsh@windriver.com>
Some kernel operations, like scheduler locking can be optmized out,
since coop threads lock the scheduler by their very nature. Also, the
interrupt exit path for all architecture does not have to do any
rescheduling, again by the nature of non-preemptible threads.
Change-Id: I270e926df3ce46e11d77270330f2f4b463971763
Signed-off-by: Benjamin Walsh <benjamin.walsh@windriver.com>
The way the ready thread cache was implemented caused it to not always
be "hot", i.e. there could be some misses, which happened when the
cached thread was taken out of the ready queue. When that happened, it
was not replaced immediately, since doing so could mean that the
replacement might not run because the flow could be interrupted and
another thread could take its place. This was the more conservative
approach that insured that moving a thread to the cache would never be
wasted.
However, this caused two problems:
1. The cache could not be refilled until another thread context-switched
in, since there was no thread in the cache to compare priorities
against.
2. Interrupt exit code would always have to call into C to find what
thread to run when the current thread was not coop and did not have the
scheduler locked. Furthermore, it was possible for this code path to
encounter a cold cache and then it had to find out what thread to run
the long way.
To fix this, filling the cache is now more aggressive, i.e. the next
thread to put in the cache is found even in the case the current cached
thread is context-switched out. This ensures the interrupt exit code is
much faster on the slow path. In addition, since finding the next thread
to run is now always "get it from the cache", which is a simple fetch
from memory (_kernel.ready_q.cache), there is no need to call the more
complex C code.
On the ARM FRDM K64F board, this improvement is seen:
Before:
1- Measure time to switch from ISR back to interrupted task
switching time is 215 tcs = 1791 nsec
2- Measure time from ISR to executing a different task (rescheduled)
switch time is 315 tcs = 2625 nsec
After:
1- Measure time to switch from ISR back to interrupted task
switching time is 130 tcs = 1083 nsec
2- Measure time from ISR to executing a different task (rescheduled)
switch time is 225 tcs = 1875 nsec
These are the most dramatic improvements, but most of the numbers
generated by the latency_measure test are improved.
Fixes ZEP-1401.
Change-Id: I2eaac147048b1ec71a93bd0a285e743a39533973
Signed-off-by: Benjamin Walsh <benjamin.walsh@windriver.com>
Also remove NO_METRIC, which is not referenced anywhere anymore.
Change-Id: Ieaedf075af070a13aa3d975fee9b6b332203bfec
Signed-off-by: Benjamin Walsh <benjamin.walsh@windriver.com>
There was a lot of duplication between architectures for the definition
of threads and the "nanokernel" guts. These have been consolidated.
Now, a common file kernel/unified/include/kernel_structs.h holds the
common definitions. Architectures provide two files to complement it:
kernel_arch_data.h and kernel_arch_func.h. The first one contains at
least the struct _thread_arch and struct _kernel_arch data structures,
as well as the struct _callee_saved and struct _caller_saved register
layouts. The second file contains anything that needs what is provided
by the common stuff in kernel_structs.h. Those two files are only meant
to be included in kernel_structs.h in very specific locations.
The thread data structure has been separated into three major parts:
common struct _thread_base and struct k_thread, and arch-specific struct
_thread_arch. The first and third ones are included in the second.
The struct s_NANO data structure has been split into two: common struct
_kernel and arch-specific struct _kernel_arch. The latter is included in
the former.
Offsets files have also changed: nano_offsets.h has been renamed
kernel_offsets.h and is still included by the arch-specific offsets.c.
Also, since the thread and kernel data structures are now made of
sub-structures, offsets have to be added to make up the full offset.
Some of these additions have been consolidated in shorter symbols,
available from kernel/unified/include/offsets_short.h, which includes an
arch-specific offsets_arch_short.h. Most of the code include
offsets_short.h now instead of offsets.h.
Change-Id: I084645cb7e6db8db69aeaaf162963fe157045d5a
Signed-off-by: Benjamin Walsh <benjamin.walsh@windriver.com>
Interrupt stubs now just push the ISR and parameter onto the stack
and jump to the common interrupt code, never to return.
Change-Id: I82543d8148b5c7dfe116c43f41791f852614bb28
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Re-enabling interrupts before running the ISR must only be done
when CONFIG_NESTED_INTERRUPTS is turned on.
Change-Id: I2c04f2ce08d41cfef5553ee8554a90d1be0e86a3
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
The x86 architecture port is fitted with support for the unified kernel,
namely:
- the interrupt exit code now calls _Swap() if the current
thread is not a coop thread and if the scheduler is not locked
- there is no 'task' fields in the _nanokernel anymore: _Swap()
now calls _get_next_ready_thread instead
- the _nanokernel.fiber field is replaced by a more sophisticated
ready_q, based on the microkernel's priority-bitmap-based one
- nano_private includes nano_internal.h from the unified directory
- the FIBER, TASK and PREEMPTIBLE flags do not exist anymore: the thread
priority drives the behaviour
- the tcs uses a dlist for queuing in both ready and wait queues instead
of a custom singly-linked list
- other new fields in the tcs include a schedule-lock count, a
back-pointer to init data (when the task is static) and a pointer to
swap data, needed when a thread pending on _Swap() must be passed more
then just one value (e.g. k_stack_pop() needs an error code and data)
- fiberRtnValueSet() is aliased to _set_thread_return_value since it
also operates on preempt threads now
- _set_thread_return_value_with_data() sets the swap_data field in
addition to a return value from _Swap()
- convenience aliases are created for shorter names:
- _current is defined as _nanokernel.current
- _ready_q is defined as _nanokernel.ready_q
- _Swap() sets the threads's return code to -EAGAIN before swapping out
to prevent timeouts to have to set it (solves hard issues in some
kernel objects).
- Floating point support.
Note that, in _Swap(), the register holding the thread to be swapped in has
been changed from %ecx to %eax in both the legacy kernel and the unified kernel
to take advantage of the fact that the return value of _get_next_ready_thread()
is stored in %eax, and this avoids moving it to %ecx.
Work by: Dmitriy Korovkin <dmitriy.korovkin@windriver.com>
Allan Stephens <allan.stephens@windriver.com>
Benjamin Walsh <benjamin.walsh@windriver.com>
Change-Id: I4ce2bd47bcdc62034c669b5e889fc0f29480c43b
Signed-off-by: Benjamin Walsh <benjamin.walsh@windriver.com>
Originally, x86 just supported APIC. Then later support
for the Mint Valley Interrupt Controller was added. This
controller is mostly similar to the APIC with some differences,
but was integrated in a somewhat hacked-up fashion.
Now we define irq_controller.h, which is a layer of abstraction
between the core arch code and the interrupt controller
implementation.
Contents of the API:
- Controllers with a fixed irq-to-vector mapping define
_IRQ_CONTROLLER_VECTOR_MAPPING(irq) to obtain a compile-time
map between the two.
- _irq_controller_program() notifies the interrupt controller
what vector will be used for a particular IRQ along with triggering
flags
- _irq_controller_isr_vector_get() reports the vector number of
the IRQ currently being serviced
- In assembly language domain, _irq_controller_eoi implements
EOI handling.
- Since triggering options can vary, some common defines for
triggering IRQ_TRIGGER_EDGE, IRQ_TRIGGER_LEVEL, IRQ_POLARITY_HIGH,
IRQ_POLARITY_LOW introduced.
Specific changes made:
- New Kconfig X86_FIXED_IRQ_MAPPING for those interrupt controllers
that have a fixed relationship between IRQ lines and IDT vectors.
- MVIC driver rewritten per the HAS instead of the tortuous methods
used to get it to behave like LOAPIC. We are no longer writing values
to reserved registers. Additional assertions added.
- Some cleanup in the loapic_timer driver to make the MVIC differences
clearer.
- Unused APIs removed, or folded into calling code when used just once.
- MVIC doesn't bother to write a -1 to the intList priority field since
it gets ignored anyway
Issue: ZEP-48
Change-Id: I071a477ea68c36e00c3d0653ce74b3583454154d
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
The toolchain headers included an abstraction for defining symbol
names in assembly context in the situation where we're using a
DOS-style assembler that automatically prepends an underscore to
symbol names.
We aren't. Zephyr is an ELF platform. None of our toolchains do
this. Nothing sets the "TOOL_PREPENDS_UNDERSCORE" macro from within
the project, and it surely isn't an industry standard. Yank it out.
Now we can write assembler labels in natural syntax, and a few other
things fall out to simplify too.
(NOTE: these headers contain assembly code and will fail checkpatch.
That is an expected false positive.)
Change-Id: Ic89e74422b52fe50b3b7306a0347d7a560259581
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Until now, EOI had always been sent out to the APIC with interrupts
unlocked. Depending on timing, there is a race where the next interrupt
on the same line could arrive before _IntExit disables interrupts
and pops context. If this happens consecutively enough times, the
interrupt stack will overflow.
Now we disable interrupts at the beginning of _IntExitWithEoi and they
remain that way until 'iret'.
Change-Id: Ibb28e0db902ff483d7a885389f231ac2d1864657
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Some issues have been noted with nested interrupts on quark SE.
In particular, the wrong vector # being sent to the IOAPIC EOI
register. Now when doing EOI, we lock interrupts so that the act
of reading the current vector being serviced, and sending EOI
to both controllers happens atomically.
Change-Id: Id9ad992740e197bb9d4638764952b04a27c4af61
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Having two parallel implementations is a maintenance issue, especially
when some strategically placed #ifdefs will suffice.
We prefer the ASM versions for SYS V, as we need complete control of
the emitted assembly for interrupt handling and context switching.
The SYS V code is far more mature. IAMCU C code has known issues with
-fomit-frame-pointer.
The only difference between the two calling conventions is that the
first three function arguments are provided in eax, edx, ecx instead
of on the stack.
Issue: ZEP-49
Change-Id: I9245e4b0ffbeb6d890a4f08bc8a3a49faa6d8e7b
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Move the files that *know* about the calling convention in use by the
compiler. The routines exposed by the files moved to the i386_sysV_abi
directory follow the C calling convention specified by the
i386_sysV_abi which is the default for GCC. The upstream GCC has been
enhanced to support the iamcu_ABI that is optimized for processors
that implement the IA MCU instruction set. This new ABI provides code,
data and stack size improvements on IA MCU based systems.
This change is the first step in adding support for the IA MCU
optimized toolchains to Zephyr OS
Change-Id: I13bffee8007fb3f82aa31389b2c241065e8e315d
Original-work-by: Dirk Brandewie <dirk.j.brandewie@intel.com>
Signed-off-by: Benjamin Walsh <benjamin.walsh@windriver.com>
As the system always operates in ring 0, neither the SS nor ESP registers
are pushed onto the stack when an exception or an interrupt occurs.
However, as the ESP field is still relevant to debugging fatal errors, a
place has been carved for it in the NANO_ESF.
Change-Id: Ibb2578c69fa6365fd6e9dbf7b51f461063dadc68
Signed-off-by: Peter Mitsis <peter.mitsis@windriver.com>
This was only needed on legacy platforms which are no longer
supported.
Change-Id: I4a3312f3698c4fc8bbf0df4610af7b69a9056f80
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
This option is not building and currently not supported, removing
it because there does not seem to be a use case for it.
Change-Id: Idb8ffedf83f43cffc68a01573c6f2d1a90fc40fb
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
We are interested in supporting some XIP x86 platforms which are
unable to fetch CPU instructions from system RAM. This requires
refactoring our dynamic IRQ/exc code which currently synthesizes
assembly language instructions to create IRQ stubs on-the-fly.
Instead, a new approach is taken. Given that the configuration at
build time specifies the number of required stubs, use this
to generate a build time a set of tiny stub functions which simply
push a 'stub id' and then call common dynamic interrupt code.
The handler function and handler argument is saved in a table keyed by
this stub id.
CONFIG_EOI_HANDLER_SUPPORTED removed, the code hasn't been conditionally
compiled for some time and in all cases we call _loapic_eoi() when
finished with an interrupt.
Some other out-of-date verbiage in comments related to supporting
non-APIC removed.
Previously, when dynamic exceptions were created a pointer would
be passed in by the caller reserving ram for the stub code. Since
this is no longer feasible, two new Kconfig options have been added.
CONFIG_NUM_DYNAMIC_EXC_STUBS and CONFIG_NUM_DYNAMIC_EXC_NO_ERR_STUBS
control how many stubs are created for exceptions that push
an error code, and no error code, respectively.
SW Interrupts are no longer triggered by "int <vector>" hard-coded
assembly instructions. Instead this is done by sending a self-directed
inter-processor interrupt from the LOAPIC, using a new API
loapic_int_vect_trigger(). In this way we get rid of dynamically
generated code in irq_test_common.h.
All interrupts call _loapic_eoi() when finished, since this is now
the right thing to do for all IRQs, including SW interrupts.
_irq_handler_set() for x86 no longer requires the old function pointer
to be supplied.
Change-Id: I78993d3d00dd153c9051c518b417cce8d3acee9e
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
In order to have a name according to the functionality of the feature.
This commit rename any text, function and variable related with the
Profiler name to Event logger.
Change-Id: I4f612cbc7c37965c35a64f06cc3ce5e3249d90e5
Signed-off-by: Yonattan Louise <yonattan.a.louise.mendoza@intel.com>
The \NOMANUAL tag is a remnant from days of yore and is no longer
needed or useful. Cleaning up the code references to this.
Change-Id: I1b8cc9c9560d1dbb711f05fa63fd23386789875c
Signed-off-by: Dan Kalowsky <daniel.kalowsky@intel.com>
Renames the Kconfig option NO_NESTED_INTERRUPTS to NESTED_INTERRUPTS
as it is typically easier to follow positive logic than it is to
follow negative logic.
Change-Id: I68f9220621545a72254ba561aa3cb488e59e402a
Signed-off-by: Peter Mitsis <peter.mitsis@windriver.com>
Change all the Intel and Wind River code license from BSD-3 to Apache 2.
Change-Id: Id8be2c1c161a06ea8a0b9f38e17660e11dbb384b
Signed-off-by: Javier B Perez Hernandez <javier.b.perez.hernandez@linux.intel.com>
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Signed-off-by: Allan Stephens <allan.stephens@windriver.com>
Signed-off-by: Benjamin Walsh <benjamin.walsh@windriver.com>
Add the interrupt profile points for x86 and ARM arquitectures. This
gives information regarding the time when interrupts occur.
Change-Id: Ic876c0e7f9e8819d53e0578416f09146f4456d3d
Signed-off-by: Yonattan Louise <yonattan.a.louise.mendoza@intel.com>
The term 'context' is vague and overloaded. Its usage for 'an execution
context' is now referred as such, in both comments and some APIs' names.
When the execution context can only be a fiber or a task (i.e. not an
ISR), it is referred to as a 'thread', again in comments and everywhere
in the code.
APIs that had their names changed:
- nano_context_id_t is now nano_thread_id_t
- context_self_get() is now sys_thread_self_get()
- context_type_get() is now sys_execution_context_type_get()
- context_custom_data_set/get() are now
sys_thread_custom_data_set/get()
The 'context' prefix namespace does not have to be reserved by the
kernel anymore.
The Context Control Structure (CCS) data structure is now the Thread
Control Structure (TCS):
- struct ccs is now struct tcs
- tCCS is now tTCS
Change-Id: I7526a76c5b01e7c86333078e2d2e77c9feef5364
Signed-off-by: Benjamin Walsh <benjamin.walsh@windriver.com>
The inline versions are renamed to remove the _inline suffix, and the
non-inline versions are removed from the code base.
Change-Id: I7314b96c42835f15df4c537ec11ab7961d4ee60f
Signed-off-by: Benjamin Walsh <benjamin.walsh@windriver.com>
Removes references to obsolete BSP terminology. Where appropriate, replaces it
with platform terminology.
Change-Id: I41fc099844d137dd0ea87cce0f675dc6f022ad40
Signed-off-by: Peter Mitsis <peter.mitsis@windriver.com>
Remove function name from comment and add @brief instead.
Also capitilize first letter.
Change-Id: Ib708b49bf02e5bc89b0066637a55874e659637e0
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Previous comment style used RETRURNS:, use @return to comply
with javadoc style.
Change-Id: Ib1dffd92da1d97d60063ec5309b08049828f6661
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
The change replaces multiple asterisks to ** at
the beginning of comments and adds a space before
the asterisks at the beginning of lines.
Change-Id: I7656bde3bf4d9a31e38941e43b580520432dabc1
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
