:sectnums:
=== Dual-Core Configuration

.Dual-Core Example Programs
[TIP]
A set of rather simple dual-core example programs can be found in `sw/example/demo_dual_core*`.

Optionally, the CPU core can be implemented as **symmetric multiprocessing (SMP) dual-core** system.
This dual-core configuration is enabled by the `DUAL_CORE_EN` <<_processor_top_entity_generics, top generic>>.
When enabled, two _core complexes_ are implemented. Each core complex consists of a CPU core and optional
instruction (`I$`) and data (`D$`) caches. Similar to the single-core <<_bus_system>>, the instruction and
data interfaces are switched into a single bus interface by a prioritizing bus switch. The bus interfaces
of both core complexes are further switched into a single system bus using a round-robin arbiter.

image::smp_system.png[align=center]

Both CPU cores are fully identical and use the same ISA, tuning and cache configurations provided by the
according <<_processor_top_entity_generics, top generics>>. However, each core can be identified by the
according "hart ID" that can be retrieved from the <<_mhartid>> CSR. CPU core 0 (the _primary_ core) has
`mhartid = 0` while core 1 (the _secondary_ core) has `mhartid = 1`.

The following table summarizes the most important aspects when using the dual-core configuration.

[cols="<2,<10"]
[grid="rows"]
|=======================
| **CPU configuration** | Both cores use the same cache, CPU and ISA configuration provided by the according top generics.
| **Debugging** | A special SMP openOCD script (`sw/openocd/openocd_neorv32.dual_core.cfg`) is required to
debug both cores at once. SMP-debugging is fully supported by the RISC-V gdb port.
| **Clock and reset** | Both cores use the same global processor clock and reset.
| **Address space** | Both cores have full access to the same physical <<_address_space>>.
| **Interrupts** | All <<_processor_interrupts>> are routed to both cores. Hence, each core has access to
all <<_neorv32_specific_fast_interrupt_requests>> (FIRQs). Additionally, the RISC-V machine-level _external
interrupt_ (via the top `mext_irq_i` port) is also send to both cores. In contrast, the RISC-V machine level
_software_ and _timer_ interrupts are core-exclusive (provided by the <<_core_local_interruptor_clint>>).
| **RTE** | The <<_neorv32_runtime_environment>> can be used for both cores. However, the RTE needs to be
explicitly initialized on each core (executing `neorv32_rte_setup()`). Note that the installed trap handlers
apply to both cores. The installed user-defined trap handlers can check the according core's ID via the
<<_mhartid>> CSR to perform core-specific trap handling.
| **Memory** | Each core has its own stack. The top of stack of core 0 is defined by the <<_linker_script>>
while the top of stack of core 1 has to be explicitly defined by core 0 (see <<_dual_core_boot>>). Both
cores share the same heap, `.data` and `.bss` sections. Hence, only core 0 setups the `.data` and `.bss`
sections at boot-up.
| **Constructors and destructors** | Constructors and destructors are executed by core 0 only
(see section <<_c_standard_library>>).
| **Cache layout** | If enabled, each CPU core has its own data and/or instruction cache.
| **Cache coherency** | Be aware that there is no cache snooping available. If any CPU1 cache is enabled
care must be taken to prevent access to outdated data - either by using cache synchronization (`fence` / `fence.i`
instructions) or by using atomic memory accesses. See <<_memory_coherence>> for more information.
| **Bootloader** | Only core 0 will boot and execute the bootloader while core 1 is held in standby.
| **Booting** | See section <<_dual_core_boot>>.
|=======================


==== Dual-Core Boot

After reset, both cores start booting. However, core 1 will - regardless of the <<_boot_configuration>> - always
enter <<_sleep_mode>> right inside the default <<_start_up_code_crt0>>. The primary core (core 0) will continue
booting, executing either the <<_bootloader>> or the pre-installed image from the internal instruction memory
(depending on the boot configuration).

To boot-up core 1, the primary core has to use a special library function provided by the NEORV32 software framework:

.CPU Core 1 Launch Function Prototype (note that this function can only be executed on core 0)
[source,c]
----
int neorv32_smp_launch(int (*entry_point)(void), uint8_t* stack_memory, size_t stack_size_bytes);
----

When executed, core 0 uses the two 32-bit `MTIMECMP` registers of the <<_core_local_interruptor_clint>> to
store the _launch configuration_ at a defined address location. This launch configuration consists of the stack
configuration (via `stack_memory` and `stack_size_bytes`) and the actual entry point for core 1. After these
registers have been populated, core 1 will trigger core 1's software interrupt (also via the CLINT) to wake it
from sleep mode. After that, core 1 will fetch the launch configuration and will start execution at the configured
entry point.

.CPU Core 1 Main Function
[source,c]
----
int core1_main(void) { // return `int`, no arguments
  return 0; // return to crt0 and go to sleep mode
}
----

.Core 1 Stack Memory
[NOTE]
The memory for the stack of core 1 (`stack_memory`) can be either statically allocated (i.e. a global
volatile memory array; placed in the `.data` or `.bss` section of core 0) or dynamically allocated
(using `malloc`; placed on the heap of core 0). In any case the memory should be aligned to a 16-byte
boundary.

After that, the primary core triggers the _machine software interrupt_ of core 1 using the
<<_core_local_interruptor_clint>>. Core 1 wakes up from sleep mode, consumes the configuration structure and
finally starts executing at the provided entry point. When `neorv32_smp_launch()` returns (with no error
code) the secondary core is online and running.
