Commit Graph

863 Commits

Author SHA1 Message Date
Daniel Leung
eb11e6990b arch: xtensa: use asm for _xt_set_intset/_xt_set_intclear
Use assembly for _xt_set_intset() and _xt_set_intclear() instead of
calling into the Xtensa HAL, allowing these to be inlined.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2019-11-13 06:26:10 -06:00
Stephanos Ioannidis
9695763f5f arch: x86: Inline direct ISR functions.
This commit inlines the direct ISR functions that were previously
implemented in irq_manage.c, since the PR #20119 resolved the circular
dependency between arch.h and kernel_structs.h described in the issue
#3056.

Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
2019-11-08 15:50:23 +01:00
Stephanos Ioannidis
5ba0c8e0c5 arch: arm: Inline arch_isr_direct_header.
This commit inlines arch_isr_direct_header function that was previously
placed in irq_manage.c for no good reason (possibly in relation to the
FIXME for #3056).

In addition, since the PR #20119 resolved the header circular
dependency issue described in the issue #3056, this commit removes the
references to it in the code.

The reason for not inlining _arch_is_direct_pm as the #3056 FIXME
suggests is that there is little to gain from doing so and there still
exists circular dependency for the headers required by this function
(#20119 only addresses kernel_structs.h, which is required for _current
and _kernel, which, in turn, is required for handling interrupt nesting
in many architectures; in fact, Cortex-A and Cortex-R port will require
it as well).

Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
2019-11-08 12:23:05 +01:00
Andrew Boie
4f77c2ad53 kernel: rename z_arch_ to arch_
Promote the private z_arch_* namespace, which specifies
the interface between the core kernel and the
architecture code, to a new top-level namespace named
arch_*.

This allows our documentation generation to create
online documentation for this set of interfaces,
and this set of interfaces is worth treating in a
more formal way anyway.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-11-07 15:21:46 -08:00
Andrew Boie
a6de79b4af arm: remove unused header
This had two functions in it, neither were implemented
anywhere.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-11-07 15:21:46 -08:00
Andrew Boie
65e17a923f x86: fix z_x86_page_tables_get() for 64-bit
Needs a different assembly instruction.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-11-06 17:50:34 -08:00
Andrew Boie
a9e0d14c57 x86: unify stack macro definitions
Introduce a new header thread_stack.h which does the
right thing for both 32-bit and 64-bit x86.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-11-06 17:50:34 -08:00
Stephanos Ioannidis
2d7460482d headers: Refactor kernel and arch headers.
This commit refactors kernel and arch headers to establish a boundary
between private and public interface headers.

The refactoring strategy used in this commit is detailed in the issue

This commit introduces the following major changes:

1. Establish a clear boundary between private and public headers by
  removing "kernel/include" and "arch/*/include" from the global
  include paths. Ideally, only kernel/ and arch/*/ source files should
  reference the headers in these directories. If these headers must be
  used by a component, these include paths shall be manually added to
  the CMakeLists.txt file of the component. This is intended to
  discourage applications from including private kernel and arch
  headers either knowingly and unknowingly.

  - kernel/include/ (PRIVATE)
    This directory contains the private headers that provide private
   kernel definitions which should not be visible outside the kernel
   and arch source code. All public kernel definitions must be added
   to an appropriate header located under include/.

  - arch/*/include/ (PRIVATE)
    This directory contains the private headers that provide private
   architecture-specific definitions which should not be visible
   outside the arch and kernel source code. All public architecture-
   specific definitions must be added to an appropriate header located
   under include/arch/*/.

  - include/ AND include/sys/ (PUBLIC)
    This directory contains the public headers that provide public
   kernel definitions which can be referenced by both kernel and
   application code.

  - include/arch/*/ (PUBLIC)
    This directory contains the public headers that provide public
   architecture-specific definitions which can be referenced by both
   kernel and application code.

2. Split arch_interface.h into "kernel-to-arch interface" and "public
  arch interface" divisions.

  - kernel/include/kernel_arch_interface.h
    * provides private "kernel-to-arch interface" definition.
    * includes arch/*/include/kernel_arch_func.h to ensure that the
     interface function implementations are always available.
    * includes sys/arch_interface.h so that public arch interface
     definitions are automatically included when including this file.

  - arch/*/include/kernel_arch_func.h
    * provides architecture-specific "kernel-to-arch interface"
     implementation.
    * only the functions that will be used in kernel and arch source
     files are defined here.

  - include/sys/arch_interface.h
    * provides "public arch interface" definition.
    * includes include/arch/arch_inlines.h to ensure that the
     architecture-specific public inline interface function
     implementations are always available.

  - include/arch/arch_inlines.h
    * includes architecture-specific arch_inlines.h in
     include/arch/*/arch_inline.h.

  - include/arch/*/arch_inline.h
    * provides architecture-specific "public arch interface" inline
     function implementation.
    * supersedes include/sys/arch_inline.h.

3. Refactor kernel and the existing architecture implementations.

  - Remove circular dependency of kernel and arch headers. The
   following general rules should be observed:

    * Never include any private headers from public headers
    * Never include kernel_internal.h in kernel_arch_data.h
    * Always include kernel_arch_data.h from kernel_arch_func.h
    * Never include kernel.h from kernel_struct.h either directly or
     indirectly. Only add the kernel structures that must be referenced
     from public arch headers in this file.

  - Relocate syscall_handler.h to include/ so it can be used in the
   public code. This is necessary because many user-mode public codes
   reference the functions defined in this header.

  - Relocate kernel_arch_thread.h to include/arch/*/thread.h. This is
   necessary to provide architecture-specific thread definition for
   'struct k_thread' in kernel.h.

  - Remove any private header dependencies from public headers using
   the following methods:

    * If dependency is not required, simply omit
    * If dependency is required,
      - Relocate a portion of the required dependencies from the
       private header to an appropriate public header OR
      - Relocate the required private header to make it public.

This commit supersedes #20047, addresses #19666, and fixes #3056.

Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
2019-11-06 16:07:32 -08:00
Andrew Boie
800b35f598 kernel: use uintptr_t for syscall arguments
We need to pass system call args using a register-width
data type and not hard-code this to u32_t.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-11-06 09:04:16 -08:00
Peter Robinson
2e1f8090b0 linker: riscv: discard .note.GNU-stack
Similar to what was required for x86 in #12719 / PR#12752

Signed-off-by: Peter Robinson <pbrobinson@gmail.com>
2019-11-04 09:10:19 -05:00
Peter Robinson
ccd40d5502 linker: arm: discard .note.GNU-stack
Similar to what was required for x86 in #12719 / PR#12752

Signed-off-by: Peter Robinson <pbrobinson@gmail.com>
2019-11-04 09:10:19 -05:00
Kumar Gala
00c7ce7942 linker: move where we define _LINKER and _ASMLANGUAGE
Move _LINKER and _ASMLANGUAGE to target.cmake because of how we pick the
linker script that might be used.  This way regardless of how or where a
linker.ld gets included we will always set _LINKER & _ASMLANGUAGE (so
any header that needs check based on those defines they can,
specifically generated_dts_board.h)

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-11-03 12:55:16 +01:00
Daniel Leung
b7eb04b300 x86: consolidate x86_64 architecture, SoC and boards
There are two set of code supporting x86_64: x86_64 using x32 ABI,
and x86 long mode, and this consolidates both into one x86_64
architecture and SoC supporting truly 64-bit mode.

() Removes the x86_64:x32 architecture and SoC, and replaces
   them with the existing x86 long mode arch and SoC.
() Replace qemu_x86_64 with qemu_x86_long as qemu_x86_64.
() Updates samples and tests to remove reference to
   qemu_x86_long.
() Renames CONFIG_X86_LONGMODE to CONFIG_X86_64.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2019-10-25 17:57:55 -04:00
Alberto Escolar Piedras
f974cb0ae1 posix arch: Untangle headers
posix_soc_if.h is meant to be a private header between
the POSIX ARCH, SOC, and maybe boards,
it should not contain definitions meant to be used directly
by the kernel or app.

Some definitions were placed here due to a dependency moebius
loop.
Unravel that by removing all header dependencies in posix_soc_if.h,
move those definitions out to a more logical place,
and while we are here reduce the amount of users of
irq_offload.h in POSIX arch related code

Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
2019-10-25 11:23:49 -07:00
Andrew Boie
f6e82ea1bd x86: generate runtime 64-bit page tables
- Bring in CONFIG_X86_MMU and some related defines to
  common X86 Kconfig
- Don't set ARCH_HAS_USERSPACE for intel64 yet when
  X86_MMU is enabled
- Uplevel x86_mmu.c to common code
- Add logic for handling PML4 table and generating PDPTs
- move z_x86_paging_init() to common kernel_arch_func.h
- Uplevel inclusion of mmustructs.h to common x86 arch.h,
  both need it for memory domain defines

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-10-24 12:48:45 -07:00
Andrew Boie
33930352f3 x86: intel64: enable MMU region list
Same snippet as in our 32-bit linker file. Creates
an iterable section with all the MMU_BOOT_REGION() info.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-10-24 12:48:45 -07:00
Andrew Boie
4c0d044863 x86: mmustructs: use Z_STRUCT_SECTION_ITERABLE()
This does the right thing for arches with 8-byte words.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-10-24 12:48:45 -07:00
Andrew Boie
8ec765b0bf x86: intel64: set up memory sections
Some linker symbols indicating the bounds of various memory
areas needed for setting up boot MMU regions are now in
place.

MMU_BOOT_REGION macros added to align the bounds of these
sections if we're using runtime page tables.

_image_rodata_start moved to the proper place, it was not
accounting for the rodata sections specified in
linker/common_rom.ld.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-10-24 12:48:45 -07:00
Andrew Boie
cdd721db3b locore: organize data by type
Program text, rodata, and data need different MMU
permissions. Split out rodata and data from the program
text, updating the linker script appropriately.

Region size symbols added to the linker script, so these
can later be used with MMU_BOOT_REGION().

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-10-24 12:48:45 -07:00
Ioannis Glaropoulos
bb41d56ced arch: arm: do not add CODE_UNREACHABLE in Z_ARCH_EXCEPT
For ARM, Z_ARCH_EXCEPT triggers an SVC to induce a system error.
This code block may be inlined, so, if we want to return from
this error DIRECTLY to thread mode, e.g. if the system error
occurred in ISR context and we are not aborting the current
thread, we must instruct the compiler that the execution
may continue after the inlined SVC. Therefore, we must remove
the CODE_UNREACHABLE statements.

Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
2019-10-24 10:12:08 -07:00
Andrew Boie
979b17f243 kernel: activate arch interface headers
Duplicate definitions elsewhere have been removed.

A couple functions which are defined by the arch interface
to be non-inline, but were implemented inline by native_posix
and intel64, have been moved to non-inline.

Some missing conditional compilation for z_arch_irq_offload()
has been fixed, as this is an optional feature.

Some massaging of native_posix headers to get everything
in the right scope.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-10-21 10:13:38 -07:00
Wayne Ren
601b9afc9e arch: arc: implement DIRECT IRQ support
* implement DIRECT IRQ support both for normal irq and fast irq.
* add separate interrupt stack for fast irq and use CONFIG_ARC_
  _FIRQ_STACK to control it. This will bring shortest interrupt
  latency for fast irq.
* note that scheduing in DIRECT IRQ is not supported.

Signed-off-by: Wayne Ren <wei.ren@synopsys.com>
2019-10-21 09:06:17 -07:00
Ioannis Glaropoulos
684f5ec12e arch: arm: userspace: add ip and sp to clobber
We should be adding a compiler barrier for IP and SP
registers when we are doing syscall generation on
ARMv6-M architecture. The syscall generation itself
only does an SVC trigger; the execution returns to
thread mode and ARM does not guarantee that IP
register is preserved, when we finally get back to
the point where the syscall was invoked. The SP
also needs to be preserved for syscalls returning
64-bit results. In that case the r0 may hold a
pointer to the stack where the 64-bit result was
pushed, That is, the stack pointer may have been
changed due to the syscall, and C code needs to
know that.

Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
2019-10-18 14:20:32 -05:00
Ioannis Glaropoulos
c86073cf6b arch: arm: error: fix ARMv6-M assembly for Z_ARCH_EXCEPT
As we are allowed to pass any integer value as as software
fatal exception reason, we need to fix the inline assembly
for ARMv6-M, to accept large immediate offsets. We do this
by changing the way we write the exception reason to R0.

Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
2019-10-16 11:22:48 +02:00
Andrew Boie
31620b90e2 x86: refactor mmustructs.h
The struct definitions for pdpt, pd, and pt entries has been
removed:

 - Bitfield ordering in a struct is implementation dependent,
   it can be right-to-left or left-to-right
 - The two different structures for page directory entries were
   not being used consistently, or when the type of the PDE
   was unknown
 - Anonymous structs/unions are GCC extensions

Instead these are now u64_t, with bitwise operations used to
get/set fields.

A new set of inline functions for fetcing various page table
structures has been implemented, replacing the older macros.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-10-14 11:49:39 -07:00
Andrew Boie
ab4d647e6d x86: mmu: get rid of x86_page_entry_data_t typedef
This hasn't been necessary since we dropped support for 32-bit
non-PAE page tables. Replace it with u64_t and scrub any
unnecessary casts left behind.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-10-14 11:49:39 -07:00
Andrew Boie
e3ab43580c x86: move mmustructs.h
This will be used for both 32-bit and 64-bit mode.
This header gets pulled in by x86's arch/cpu.h, so put
it in include/arch/x86/.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-10-14 11:49:39 -07:00
Andrew Boie
8ffff144ea kernel: add architecture interface headers
include/sys/arch_inlines.h will contain all architecture APIs
that are used by public inline functions and macros,
with implementations deriving from include/arch/cpu.h.

kernel/include/arch_interface.h will contain everything
else, with implementations deriving from
arch/*/include/kernel_arch_func.h.

Instances of duplicate documentation for these APIs have been
removed; implementation details have been left in place.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-10-11 13:30:46 -07:00
Andrew Boie
ccfccae3bc x86: intel64: don't drop debug sections
Pull in a header specifically for suppressing orphan section
messages.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-10-10 13:41:13 -07:00
Andrew Boie
e340f8d22e x86: intel64: enable no-execute
Set the NXE bit in the EFER MSR so that the NX bit can
be set in page tables. Otherwise, the NX bit is treated
as reserved and leads to a fault if set.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-10-10 13:41:02 -07:00
Alberto Escolar Piedras
b237597ab9 arch: posix: isolate arch-soc/board IF from kernel-arch IF
The POSIX ARCH delegates some of the tasks which normally
are taken care of by the ARCH to the SOC or BOARD levels.

To avoid changes in the kernel-arch IF propagating into
the arch-soc and arch-board interfaces (which would break
off-tree posix boards) isolate them.

Also move arch inlined functions into the arch.h header,
and out from the headers which specify the  posix arch-soc
and arch-board interfaces.

Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
2019-10-09 09:14:18 -04:00
Andrew Boie
d1aca7f11b x86_64: fix arch headers
arch/cpu.h and kernel_arch_func.h are expected to define different
functions, per the architecture interface.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-10-09 09:14:18 -04:00
Andrew Boie
3ffb89ad7f arches: fix z_arch_k_cycle_get_32() definitions
These need to all be inline functions and not macros.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-10-09 09:14:18 -04:00
Andrew Boie
b1a73a17bc sys_io: pull in toolchain.h
Needed for ALWAYS_INLINE.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-10-09 09:14:18 -04:00
Charles E. Youse
17e135bc41 arch/x86: (Intel64) clear BSS before entering long mode
This is really just to facilitate CPU bootstrap code between
the BSP and the APs, moving the clear operation out of the way.

Signed-off-by: Charles E. Youse <charles.youse@intel.com>
2019-10-07 19:46:55 -04:00
Charles E. Youse
25a7cc1136 arch/x86: (Intel64) add missing linker symbols
The linker script was missing symbols that defined the boundaries
of kernel memory segments (_image_rom_end, etc.). These are added
so that core/memmap.c can properly account for those segments.

Signed-off-by: Charles E. Youse <charles.youse@intel.com>
2019-10-07 19:46:55 -04:00
Jan Van Winkel
ebb52520b2 posix: Set stack align to 8 bytes for 64-bit posix
Set STACK_ALIGN and STACK_ALIGN_SIZE to 8 bytes instead of 4 bytes in
case a 64-bit posix board is used.

Signed-off-by: Jan Van Winkel <jan.van_winkel@dxplore.eu>
2019-10-04 18:46:03 -07:00
Andrew Boie
f0ddbd7eee x86: abstract toplevel page table pointer
This patch is a preparatory step in enabling the MMU in
long mode; no steps are taken to implement long mode support.

We introduce struct x86_page_tables, which represents the
top-level data structure for page tables:

- For 32-bit, this will contain a four-entry page directory
  pointer table (PDPT)
- For 64-bit, this will (eventually) contain a page map level 4
  table (PML4)

In either case, this pointer value is what gets programmed into
CR3 to activate a set of page tables. There are extra bits in
CR3 to set for long mode, we'll get around to that later.

This abstraction will allow us to use the same APIs that work
with page tables in either mode, rather than hard-coding that
the top level data structure is a PDPT.

z_x86_mmu_validate() has been re-written to make it easier to
add another level of paging for long mode, to support 2MB
PDPT entries, and correctly validate regions which span PDPTE
entries.

Some MMU-related APIs moved out of 32-bit x86's arch.h into
mmustructs.h.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-10-04 15:53:49 -07:00
Andrew Boie
8c98a97581 arm: arch code naming cleanup
This patch re-namespaces global variables and functions
that are used only within the arch/arm/ code to be
prefixed with z_arm_.

Some instances of CamelCase have been corrected.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-10-04 10:46:23 +02:00
Kumar Gala
885b1ee7de arch/x86: Fix gcc-9.2 warning with z_arch_syscall_invoke6
Fix the following warning from gcc-9.2:

x86/ia32/syscall.h: In function 'test_kinit_preempt_thread':
x86/ia32/syscall.h:43:2: error: listing the stack pointer register
 'esp' in a clobber list is deprecated [-Werror=deprecated]
   43 |  __asm__ volatile("push %%ebp\n\t"
      |  ^~~~~~~
x86/ia32/syscall.h:43:2: note: the value of the stack pointer after
an 'asm' statement must be the same as it was before the statement

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-10-04 00:27:29 -07:00
Erwin Rol
0f5d2d323f arch/x86: add Z_ARCH_THREAD_STACK_MEMBER() macro
Implement standard Z_ARCH_THREAD_STACK_MEMBER() arch interface macro.

Signed-off-by: Erwin Rol <erwin@erwinrol.com>
2019-09-30 15:34:45 -07:00
Andrew Boie
bd6d8dc070 arc: rename k_cpu_sleep_mode
This is only used internally by the ARC arch code
and has been renamed to z_arc_cpu_sleep_mode.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-09-30 15:25:55 -04:00
Andrew Boie
07525a3d54 kernel: add arch interface for idle functions
k_cpu_idle() and k_cpu_atomic_idle() were being directly
implemented by arch code.

Rename these implementations to z_arch_cpu_idle() and
z_arch_cpu_atomic_idle(), and call them from new inline
function definitions in kernel.h.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-09-30 15:25:55 -04:00
Andrew Boie
845aa6d114 kernel: renamespace arch_nop()
This is part of the core kernel -> architecture interface
and has been renamed to z_arch_nop().

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-09-30 15:25:55 -04:00
Charles E. Youse
5b57737817 arch/x86: multiboot.h: use BIT() macros instead of explicit shifts
Because that's how we do it with Zephyr.

Signed-off-by: Charles E. Youse <charles.youse@intel.com>
2019-09-29 12:30:34 -07:00
Charles E. Youse
7637571871 arch/x86: rename CONFIG_X86_ACPI and related to CONFIG_ACPI
ACPI is predominantly x86, and only currently implemented on x86,
but it is employed on other architectures, so rename accordingly.

Signed-off-by: Charles E. Youse <charles.youse@intel.com>
2019-09-29 12:30:34 -07:00
Charles E. Youse
200056df2f arch/x86: rename CONFIG_X86_MULTIBOOT and related to CONFIG_MULTIBOOT
Simple naming change, since MULTIBOOT is clear enough by itself and
"namespacing" it to X86 is unnecessary and/or inappropriate.

Signed-off-by: Charles E. Youse <charles.youse@intel.com>
2019-09-29 12:30:34 -07:00
Charles E. Youse
2cf52476ea arch/x86: add support for non-trivial memory maps
x86 has more complex memory maps than most Zephyr targets. A mechanism
is introduced here to manage such a map, and some methods are provided
to populate it (e.g., Multiboot).

The x86_info tool is extended to display memory map data.

Signed-off-by: Charles E. Youse <charles.youse@intel.com>
2019-09-29 12:30:34 -07:00
Charles E. Youse
a414eb99cc samples/boards/x86_info: add platform information tool
This sample demonstrates basic use of the x86 multiboot and ACPI
systems, and also provides some useful information about the board
it's booted on: data handed over by the multiboot loader (which is
either QEMU or GRUB at this point), basic APIC CPU topology, and
timer driver frequency (computed empirically).

Signed-off-by: Charles E. Youse <charles.youse@intel.com>
2019-09-29 12:30:34 -07:00
Charles E. Youse
81eeff83b0 arch/x86: multiboot: migrate multiboot initialization to early C
Originally, the multiboot info struct was copied in the early assembly
language code. This code is moved to a C function in multiboot.c for
two reasons:

1. It's about to get more complicated, as we want the ability to use
   a multiboot-provided memory map if available, and
2. this will faciliate its sharing between 32- and 64-bit subarches.

Signed-off-by: Charles E. Youse <charles.youse@intel.com>
2019-09-29 12:30:34 -07:00