STM32MP1-OLinuXino-LIME: flash card eMMC speed testing

[dry]

Continuing my 'speed' testing of storage options. External microSD tested here:
https://www.olimex.com/forum/index.php?topic=10025.0

I tested eMMC flash on the external flash module for the board https://www.olimex.com/Products/OLinuXino/Accessories/Flash-e32Gs16M/. So at minimum we know we have 8 data lines on this 'on-board' eMMC, vs 4 on the external microSD line.
I bought 32GB one.

I used diskpd and fio to test speed, but in the end decided to stick to fio for all tests.

I tested on top of fs first, but that gave suspicious speed bumps in some cases, so I re-tested just on top of block device level. I've ran with more 'severe' case with --sync, and 'less severe' with --end_fsync on writes. Note in scrits (attached) I also include flushing (or trying to) caches bewteen each test runs.

In brief, in sum,  I'm getting this:

Code Select Expand

Sequential read:                  ~27.5 MiB/s
Sequential write, severe sync:     ~12.8 MiB/s
Sequential write, less severe:     ~27.1 MiB/s
4 KiB random read:                 ~2.5k IOPS / ~10 MiB/s
4 KiB random write, severe sync:   ~1.0k IOPS / ~3.9 MiB/s
4 KiB random write, less severe:   ~1.7k IOPS / ~6.8 MiB/s
With Severe vs less so

Code Select Expand

| Test             |                 Severe raw |            Less severe raw | Change       |
| ---------------- | -------------------------: | -------------------------: | ------------ |
| Sequential write |             **12.8 MiB/s** |             **27.1 MiB/s** | ~2.1× faster |
| Sequential read  |             **27.4 MiB/s** |             **27.5 MiB/s** | Same         |
| Random write     | **1001 IOPS / 3.91 MiB/s** | **1730 IOPS / 6.76 MiB/s** | ~1.7× faster |
| Random read      | **2600 IOPS / 10.2 MiB/s** | **2481 IOPS / 9.70 MiB/s** | Same range   |

.. while testing with file on ext4 formated on top of that block I get suspicious/ too good read results and IOPS for random reads.

Code Select Expand

| Test             |  Filesystem severe fio |       Raw block severe fio | Better number to quote                                            |
| ---------------- | ---------------------: | -------------------------: | ----------------------------------------------------------------- |
| Seq write sync   |       ~23.7–24.1 MiB/s |             **12.8 MiB/s** | Depends on workload; raw is stricter                              |
| Seq read direct  |            ~37.5 MiB/s |             **27.4 MiB/s** | Raw is cleaner                                                    |
| Rand write sync  | ~291 IOPS / 1.14 MiB/s | **1001 IOPS / 3.91 MiB/s** | Raw for media/block capability; filesystem for real file workload |
| Rand read direct |   suspicious ~22k IOPS | **2600 IOPS / 10.2 MiB/s** | Raw result                                                        |

So better to run from this eMMC of course. I kind of hoped it could be faster as this STM32MP1x might support faster mode - see
stm forum link enable hs200/hs400 mode

Based on board design, we cannot get higher HS200 mode as I understand it - again needs that ~1.8V signaling - , but I wonder if DDR52 (at 3.3v) mode could be tried / set to see if that would work: currently based on kernel logs:

Card Type [CARD_TYPE: 0x57], so advertises modes 0x01 to 0x40 (HS400),  but also it logs
HS_TIMING: 0x01, so its in high-speed mode.

Might try it next.

[dry]

Looks like I cannot attach anything on posts, so I'll inline my fio scripts.
(Btw, thanks got bot /chatGPT for help with these reports).

Code Select Expand

#    FIO Severe test
#!/bin/sh
set -eu

# Raw block target.
# WARNING: write tests will overwrite data on this device/partition.
# Change this in one place if needed.
TARGET_DEV="/dev/mmcblk2p1"

# Output directory for logs.
LOG_DIR="/tmp"

# Common fio settings.
COMMON_OPTS="--ioengine=psync \
  --iodepth=1 \
  --numjobs=1 \
  --runtime=30 \
  --time_based \
  --direct=1 \
  --group_reporting \
  --lat_percentiles=1 \
  --percentile_list=50:90:95:99:99.9:99.99"

echo "Target device: $TARGET_DEV"
echo "Logs written to: $LOG_DIR"
echo
echo "WARNING: write tests will overwrite data on $TARGET_DEV"
echo "Make sure the target is unmounted before running write tests."
echo

sync
echo 3 | tee /proc/sys/vm/drop_caches >/dev/null

fio --name=seqwrite_raw_sync \
  --filename="$TARGET_DEV" \
  --rw=write \
  --bs=1M \
  --sync=1 \
  $COMMON_OPTS \
  --output="$LOG_DIR/seqwrite_raw_sync_test.txt"

sync
echo 3 | tee /proc/sys/vm/drop_caches >/dev/null

fio --name=seqread_raw_direct_severe \
  --filename="$TARGET_DEV" \
  --readonly=1 \
  --rw=read \
  --bs=1M \
  $COMMON_OPTS \
  --output="$LOG_DIR/seqread_raw_direct_severe_test.txt"

sync
echo 3 | tee /proc/sys/vm/drop_caches >/dev/null

fio --name=randwrite_raw_sync \
  --filename="$TARGET_DEV" \
  --rw=randwrite \
  --bs=4k \
  --sync=1 \
  $COMMON_OPTS \
  --output="$LOG_DIR/randwrite_raw_sync_test.txt"

sync
echo 3 | tee /proc/sys/vm/drop_caches >/dev/null

fio --name=randread_raw_direct_severe \
  --filename="$TARGET_DEV" \
  --readonly=1 \
  --rw=randread \
  --bs=4k \
  $COMMON_OPTS \
  --output="$LOG_DIR/randread_raw_direct_severe_test.txt"

echo
echo "Done. Logs:"
echo "  $LOG_DIR/seqwrite_raw_sync_test.txt"
echo "  $LOG_DIR/seqread_raw_direct_severe_test.txt"
echo "  $LOG_DIR/randwrite_raw_sync_test.txt"
echo "  $LOG_DIR/randread_raw_direct_severe_test.txt"



Code Select Expand

# FIO less Severe setting test
#!/bin/sh
set -eu

# Raw block target.
# WARNING: write tests will overwrite data on this device/partition.
# Change this in one place if needed.
TARGET_DEV="/dev/mmcblk2p1"

# Output directory for logs.
LOG_DIR="/tmp"

# Common fio settings.
COMMON_OPTS="--ioengine=psync \
  --iodepth=1 \
  --numjobs=1 \
  --runtime=30 \
  --time_based \
  --direct=1 \
  --group_reporting \
  --lat_percentiles=1 \
  --percentile_list=50:90:95:99:99.9:99.99"

echo "Target device: $TARGET_DEV"
echo "Logs written to: $LOG_DIR"
echo
echo "WARNING: write tests will overwrite data on $TARGET_DEV"
echo "Make sure the target is unmounted before running write tests."
echo

sync
echo 3 | tee /proc/sys/vm/drop_caches >/dev/null

fio --name=seqwrite_raw_direct \
  --filename="$TARGET_DEV" \
  --rw=write \
  --bs=1M \
  --end_fsync=1 \
  $COMMON_OPTS \
  --output="$LOG_DIR/seqwrite_raw_direct_test.txt"

sync
echo 3 | tee /proc/sys/vm/drop_caches >/dev/null

fio --name=seqread_raw_direct_less_severe \
  --filename="$TARGET_DEV" \
  --readonly=1 \
  --rw=read \
  --bs=1M \
  $COMMON_OPTS \
  --output="$LOG_DIR/seqread_raw_direct_less_severe_test.txt"

sync
echo 3 | tee /proc/sys/vm/drop_caches >/dev/null

fio --name=randwrite_raw_direct \
  --filename="$TARGET_DEV" \
  --rw=randwrite \
  --bs=4k \
  --end_fsync=1 \
  $COMMON_OPTS \
  --output="$LOG_DIR/randwrite_raw_direct_test.txt"

sync
echo 3 | tee /proc/sys/vm/drop_caches >/dev/null

fio --name=randread_raw_direct_less_severe \
  --filename="$TARGET_DEV" \
  --readonly=1 \
  --rw=randread \
  --bs=4k \
  $COMMON_OPTS \
  --output="$LOG_DIR/randread_raw_direct_less_severe_test.txt"

echo
echo "Done. Logs:"
echo "  $LOG_DIR/seqwrite_raw_direct_test.txt"
echo "  $LOG_DIR/seqread_raw_direct_less_severe_test.txt"
echo "  $LOG_DIR/randwrite_raw_direct_test.txt"
echo "  $LOG_DIR/randread_raw_direct_less_severe_test.txt"



[dry]

Ta daaam ! We can improve that performance by doing small device tree mod : add

Code Select Expand

sdmmc@58007000 {
    ....
    mmc-ddr-3_3v;  <--- Just this
};
Then, after boot:

Code Select Expand

[    0.213409] tcp_listen_portaddr_hash hash table entries: 512 (order: 0, 6144 bytes, linear)
[    4.363104] mmci-pl18x 58007000.sdmmc: mmc2: PL180 manf 53 rev1 at 0x58007000 irq 57,0 (pio)
[    4.424492] mmc1: new high speed SDXC card at address aaaa
[    4.489070] mmc2: new DDR MMC card at address 0001    < ----------- HERE
[    4.494029] mmcblk2: mmc2:0001 BJTD4R 29.1 GiB
[    4.497895] mmcblk2boot0: mmc2:0001 BJTD4R partition 1 4.00 MiB
[    4.503835] mmcblk2boot1: mmc2:0001 BJTD4R partition 2 4.00 MiB
[    4.509571] mmcblk2rpmb: mmc2:0001 BJTD4R partition 3 4.00 MiB, chardev (245:0)

sudo cat /sys/kernel/debug/mmc2/ios

Code Select Expand

actual clock:    49500000 Hz
vdd:        21 (3.3 ~ 3.4 V)
bus mode:    2 (push-pull)
chip select:    0 (don't care)
power mode:    2 (on)
bus width:    3 (8 bits)
timing spec:    8 (mmc DDR52)    <--------- NOTE HERE
signal voltage:    0 (3.30 V)
driver type:    0 (driver type B)



So re-running the fio tests, this gives a speed-up / bump:

Code Select Expand

Compared with your earlier non-DDR / MMC high-speed raw results
Test              Before: MMC high-speed SDR After: DDR52 Change
Less-severe sequential write ~27.1 MiB/s 36.6 MiB/s +35%
Less-severe sequential read ~27.5 MiB/s 34.4 MiB/s +25%
Less-severe random write ~1730 IOPS 1934 IOPS +12%
Less-severe random read         ~2481 IOPS 2726 IOPS +10%
Severe sequential write         ~12.8 MiB/s 13.3 MiB/s +4%
Severe sequential read         ~27.4 MiB/s 35.0 MiB/s +28%
Severe random write         ~1001 IOPS 1021 IOPS +2%
Severe random read         ~2600 IOPS 2862 IOPS +10%


So you can decide if you need this bump. Also, any longer run stability issues not tested, if this may affect it or not.