Olimex Support Forum

OLinuXino Android / Linux boards and System On Modules => A64 => Topic started by: mossroy on March 03, 2021, 04:09:38 PM

Title: How to debug "Starting kernel ..." of Bullseye debian-installer?
Post by: mossroy on March 03, 2021, 04:09:38 PM
Debian Bullseye will ship with a kernel 5.10, which should support Olinuxino A64 boards (at least for most features, see https://linux-sunxi.org/Linux_mainlining_effort).

I tried the debian-installer (daily images from https://d-i.debian.org/daily-images/arm64/daily/netboot/SD-card-images/ , I tested version of March 1st 2021). The installation works, but :

The kernel unfortunately does not give any feedback on what's wrong.
I tried adding "earlyprintk loglevel=7" boot parameters in u-boot before startup (in the u-boot prompt, and even by re-generating a modified boot.scr with mkimage), but did not manage to have debug information on startup.

Anybody would know how to have more information? Or has succeeded in installing Debian Bullseye on a Olinuxino A64 board?
The answer might be somewhere in the commits of https://github.com/OLIMEX/linux-olimex/ or https://github.com/OLIMEX/olimage
Maybe some DTB or DTB overlays need to be added and configured ? Should we open a bug on Debian side?

Excerpt from serial console :

Quote from: undefinedFound U-Boot script /boot.scr
2226 bytes read in 1 ms (2.1 MiB/s)
## Executing script at 4fc00000
26990448 bytes read in 1198 ms (21.5 MiB/s)
28249 bytes read in 5 ms (5.4 MiB/s)
27081515 bytes read in 1205 ms (21.4 MiB/s)
Booting Debian 5.10.0-3-arm64 from mmc 0:1...
## Flattened Device Tree blob at 4fa00000
  Booting using the fdt blob at 0x4fa00000
  Loading Ramdisk to 4862c000, end 49fffb2b ... OK
  Loading Device Tree to 0000000048622000, end 000000004862be58 ... OK

Starting kernel ...
Title: Re: How to debug "Starting kernel ..." of Bullseye debian-installer?
Post by: mossroy on April 30, 2021, 12:38:17 PM
I still have the same behavior with a recent daily Debian image (2021-04-27).

Do you know how to debug that, and what is the right way to report this issue?
Title: Re: How to debug "Starting kernel ..." of Bullseye debian-installer?
Post by: JohnS on April 30, 2021, 05:27:42 PM
Looks like a software issue and thus a good question to ask on the linux-sunxi ML.

John
Title: Re: How to debug "Starting kernel ..." of Bullseye debian-installer?
Post by: mossroy on May 01, 2021, 06:57:17 PM
Quote from: JohnS on April 30, 2021, 05:27:42 PMLooks like a software issue and thus a good question to ask on the linux-sunxi ML.
Thanks for the hint. I'll try to discuss that on their IRC.

I'm wondering if it could not come from the fact that I have the A64-OLinuXino-2Ge8G-IND version. Did anybody else test the bullseye debian-installer with another Olinuxino A64 board?

I tried to switch to another dtb (in the boot partition of the installed debian), using the sun50i-a64-olinuxino-2Ge8G.dtb file found in Olimex images, but with no luck so far
Title: Re: How to debug "Starting kernel ..." of Bullseye debian-installer?
Post by: mossroy on May 01, 2021, 11:26:48 PM
I've been advised on IRC to set "earlycon" to bootargs environment variable in u-boot (instead of earlyprintk)

It indeed gives a bit more info :
Quote from: undefinedStarting kernel ...

[    2.889410] sun50i-de2-bus 1000000.bus: Error couldn't map SRAM to device
[    3.186735] sun50i-a64-pinctrl 1c20800.pinctrl: Couldn't get bank PB regulator
[    3.193983] sun50i-a64-pinctrl 1c20800.pinctrl: request() failed for pin 40
[    3.200947] sun50i-a64-pinctrl 1c20800.pinctrl: pin-40 (1c28000.serial) status -517
[    3.208604] sun50i-a64-pinctrl 1c20800.pinctrl: could not request pin 40 (PB8) from group PB8  on device 1c20800.pinctrl
[    3.219465] dw-apb-uart 1c28000.serial: Error applying setting, reverse things back
[    3.227746] sun50i-a64-pinctrl 1c20800.pinctrl: Couldn't get bank PB regulator
[    3.234989] sun50i-a64-pinctrl 1c20800.pinctrl: request() failed for pin 40
[    3.241953] sun50i-a64-pinctrl 1c20800.pinctrl: pin-40 (1c28000.serial) status -517
[    3.249610] sun50i-a64-pinctrl 1c20800.pinctrl: could not request pin 40 (PB8) from group PB8  on device 1c20800.pinctrl
[    3.260470] dw-apb-uart 1c28000.serial: Error applying setting, reverse things back
Title: Re: How to debug "Starting kernel ..." of Bullseye debian-installer?
Post by: mossroy on May 01, 2021, 11:33:16 PM
I've tried a few other dtb/dtbo combinations using the files found in Olimex images, like :

Quote from: undefinedsetenv bootargs earlycon
setenv fdtfile allwinner/sun50i-a64-olinuxino-2Ge8G.dtb
setenv fdtoverlays /dtbs/5.10.0-6-arm64/allwinner/sun50i-a64-i2c0.dtbo

It sometimes changed the log a bit, but did not make the board start
Title: Re: How to debug "Starting kernel ..." of Bullseye debian-installer?
Post by: JohnS on May 02, 2021, 04:35:08 PM
I suppose you might try earlycon with a working kernel and report to the ML what changed.

You may be asked to bisect from git...

John
Title: Re: How to debug "Starting kernel ..." of Bullseye debian-installer?
Post by: mossroy on May 02, 2021, 06:47:35 PM
In fact, these u-boot variable values sometimes let me start debian bullseye. There must be a random aspect that makes the startup not reliable.
I had to modify the boot.scr file to remove the "quiet" kernel parameter to see what was going on (following advice from linux-sunxi IRC)

There are currently some issues, though :

But there are good news, too :
Title: Re: How to debug "Starting kernel ..." of Bullseye debian-installer?
Post by: mossroy on May 03, 2021, 11:41:53 AM
People from linux-sunxi seem to be interested to have more feedback from Olimex (it's at least what apritzel said on IRC).

About what Olimex did in their dtb/dtbo (and why), about the patches applied on the kernel (they know the github repo is public, but would need to understand the reasons behind the commits, at least on the timer jump issue) etc. So that all this might be applied in upstream kernel at some point, instead of being olimex-specific changes that need to be maintained by Olimex.

They would also be interested in more Olimex feedback and tests (not only about code).

I'd be glad to help on that, but can't do it alone.

Wouldn't it be much easier for everybody if the boards were supported by a future version of debian-installer?
Title: Re: How to debug "Starting kernel ..." of Bullseye debian-installer?
Post by: mossroy on May 03, 2021, 12:51:56 PM
I tested to install the latest 5.10.23 kernel of Olimex on the SD-card generated by debian-installer (instead of the 5.10.0-6-arm64 provided by debian, based on 5.10.28), using the dtb/dtbo from Olimex too. I had to cheat in boot.scr to do that.

The board always starts properly (even if there is always a kernel stacktrace at drivers/clocksource/arm_arch_timer.c:364 during startup), and I did not see any issue with MMC.
The board reboots normally (most of the time : it happened to not reboot, like with debian kernel)
Hardware crypto works for both aes-cbc and aes-xts, but same issue with initramfs (it's probably a problem on my side)
But of course the performance governor is used, and the sensors are missing

NB : I reverted all that, it was just for testing purpose
Title: Re: How to debug "Starting kernel ..." of Bullseye debian-installer?
Post by: mossroy on December 19, 2021, 10:34:29 AM
I found this related bug report on debian : https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=975490