Embench
=======

To run these benchmarks, first make sure the embench-iot submodule is checked out, then:

```bash
cd embench-iot
# Make sure testbench is up to date
make -C ../../tb_cxxrtl tb
./build_all.py --arch riscv32 --chip hazard3 --board hazard3tb --clean
# You might need a longer timeout -- some sims are tens of millions of cycles
./benchmark_speed.py --target-module run_hazard3tb --timeout 180 --sim-parallel
```

The compiler specified in `config/riscv32/chips/hazard3/chip.cfg` is `riscv32-unknown-elf-gcc` (no directory prefix -- just whichever is on PATH). On my machine this is currently GCC12, which is the first stable release to have support for the bitmanip instructions. These instructions seem to be worth about 6% performance overall -- you can disable them in the `chip.cfg` file if your local gcc12 doesn't support them.

If you want to play with the Zcb/Zcmp instructions, these currently (March 2023) require the CORE-V development toolchains, and you will have to edit the `chip.cfg` accordingly.

## Caution on stdlib/multilib

Some of the Embench benchmarks are essentially benchmarks of the soft float routines in `libm`, which respond well to the bitmanip instructions. I have a rather sprawling multilib setup which ensures I get Zb instructions in my stdlib when I link with Zb in my `-march`. However, depending on the default architecture your toolchain was configured with, you might find that it falls back to something incredibly conservative like RV32I-only if there is no exact multilib match.

To give you an idea, here the left hand side is the speed benchmark result for `-O2 -ffunction-sections` on GCC 12 with `-march=rv32im_zba_zbb_zbs` and an RV32I stdlib, and the right hand side is with the correct standard library:

```
RV32IMZbaZbbZbs with RV32I library:         With correct standard library:

Benchmark           Speed                    Benchmark           Speed
---------           -----                    ---------           -----
aha-mont64         0.8323                    aha-mont64         0.8318
crc32              0.9209                    crc32              0.9593
cubic              0.1346                    cubic              0.5378
edn                1.1087                    edn                1.1460
huffbench          1.5361                    huffbench          1.6016
matmult-int        1.2319                    matmult-int        1.2308
minver             0.2693                    minver             0.7535
nbody              0.1901                    nbody              0.8496
nettle-aes         0.8800                    nettle-aes         1.0828
nettle-sha256      0.9981                    nettle-sha256      1.3922
nsichneu           1.1674                    nsichneu           1.1674
picojpeg           0.9904                    picojpeg           1.0812
primecount         1.5352                    primecount         1.3765
qrduino            1.3538                    qrduino            1.3717
sglib-combined     1.2075                    sglib-combined     1.2749
slre               1.3740                    slre               1.4178
st                 0.2250                    st                 1.0011
statemate          2.2047                    statemate          2.1999
tarfind            2.2567                    tarfind            2.2568
ud                 0.9269                    ud                 1.0073
wikisort           0.7946                    wikisort           1.9289
---------           -----                    ---------           -----
Geometric mean     0.8488                    Geometric mean     1.1905
Geometric SD       2.1454                    Geometric SD       1.4039
Geometric range    1.4254                    Geometric range    0.8233
All benchmarks run successfully              All benchmarks run successfully
```

(Hazard3 configuration: all ISA options enabled, MUL_FAST=1, MUL_FASTER=1, MULH_FAST=1, BRANCH_PREDICTOR=1, MULDIV_UNROLL=2, REDUCED_BYPASS=0)

I haven't yet compiled a full set of benchmark results for Hazard3, partly because it means coming up with a list of specimen configurations and then doing hardware characterisation on them. This is just a note in case you try to run Embench and wonder why the performance is through the floor.
