Olimex Support Forum

OLinuXino Android / Linux boards and System On Modules => A20 => Topic started by: StupidBeard on July 16, 2015, 10:01:57 PM

Title: Extremely slow Ethernet on A20-OLinuXino-LIME2
Post by: StupidBeard on July 16, 2015, 10:01:57 PM
Hi,

Last week I got an A20-OLinuXino-LIME2 and have been having issues with the Ethernet. HTTP transfers over the Gigabit lan are limited to approximately 70 - 100KByte/sec, and an SSH shell is frequently laggy. The issues are not limited to HTTP or ssh, these were just the most convenient things to test. In all cases the board was powered from a 5V 2.5A wall wart via the DC jack.

I initially thought this was just my Internet connection being crap, but the transfer rate is still vastly below what it should be. In any case, to rule that out, all of the following was done over a Gigabit switch to my Linux desktop. I've also verified similar issues to my Mac and over the internet.

The following tests were done on the 4.1.2 kernel version of jessie from Igor Pecovnik's site. I have also verified the same behaviour on the official Olimex wheezy image and Debian's jessie installer as downloaded from their site directly.

To establish a baseline I have been downloading a 10MB file over HTTP from my Debian box:

Code: [Select]
Total wall clock time: 3m 11s
Downloaded: 1 files, 10M in 3m 11s (53.6 KB/s)

Copying the same file over with scp results in a comparable transfer speed.

To rule out SD card write speed, I wrote a 10MB file from /dev/zero to a file:

Code: [Select]
root@lime2:~# dd if=/dev/zero of=test count=10 bs=1M
10+0 records in
10+0 records out
10485760 bytes (10 MB) copied, 0.134305 s, 78.1 MB/s

To rule out the network, I used iperf3 to benchmark the LAN:

Code: [Select]
root@lime2:~# iperf3 -c 10.0.0.10
Connecting to host 10.0.0.10, port 5201
[  4] local 10.0.0.170 port 59548 connected to 10.0.0.10 port 5201
[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.01   sec  83.7 MBytes   693 Mbits/sec    0    144 KBytes       
[  4]   1.01-2.01   sec  88.3 MBytes   742 Mbits/sec    0    174 KBytes       
[  4]   2.01-3.00   sec  87.4 MBytes   738 Mbits/sec    0    192 KBytes       
[  4]   3.00-4.00   sec  88.1 MBytes   739 Mbits/sec    0    238 KBytes       
[  4]   4.00-5.01   sec  88.7 MBytes   740 Mbits/sec    0    284 KBytes       
[  4]   5.01-6.00   sec  87.5 MBytes   739 Mbits/sec    0    284 KBytes       
[  4]   6.00-7.01   sec  88.8 MBytes   742 Mbits/sec    0    308 KBytes       
[  4]   7.01-8.01   sec  75.7 MBytes   629 Mbits/sec    0    332 KBytes       
[  4]   8.01-9.00   sec  86.2 MBytes   730 Mbits/sec    0    392 KBytes       
[  4]   9.00-10.01  sec  88.8 MBytes   739 Mbits/sec    0    392 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.01  sec   863 MBytes   723 Mbits/sec    0             sender
[  4]   0.00-10.01  sec   863 MBytes   723 Mbits/sec                  receiver

And finally, I repeated the above tests using a USB wifi adapter. The wget test completed at 2.69MB/sec so I didn't bother with running the others over wifi.

It is pretty clear that there's some issue with the Ethernet, but I am not sure if it's a software issue or hardware. Has anyone seen this issue before or have any suggestions on how to sort it?

Thanks in advance.
Title: Re: Extremely slow Ethernet on A20-OLinuXino-LIME2
Post by: Gerrit on July 16, 2015, 11:03:33 PM
check for the right driver, gmac and not emac and if it is running at full speed

Code: [Select]
# dmesg | grep gmac
[    1.577908] [gmac]: sun6i_gmac platform driver registration completed
[   17.697307] sunxi_gmac: probed
[   17.705767] eth0: PHY ID 001cc912 at 1 IRQ 0 (sunxi_gmac-0:01) active
[   19.715542] PHY: sunxi_gmac-0:01 - Link is Up - 1000/Full


Code: [Select]
# ethtool eth0
Settings for eth0:
Supported ports: [ TP AUI BNC MII FIBRE ]
Supported link modes:   10baseT/Half 10baseT/Full
                        100baseT/Half 100baseT/Full
                        1000baseT/Half 1000baseT/Full
Supported pause frame use: No
Supports auto-negotiation: Yes
Advertised link modes:  10baseT/Half 10baseT/Full
                        100baseT/Half 100baseT/Full
                        1000baseT/Half 1000baseT/Full
Advertised pause frame use: No
Advertised auto-negotiation: Yes
Speed: 1000Mb/s
Duplex: Full
Port: MII
PHYAD: 1
Transceiver: external
Auto-negotiation: on
Current message level: 0x0000003f (63)
       drv probe link timer ifdown ifup
Link detected: yes
Title: Re: Extremely slow Ethernet on A20-OLinuXino-LIME2
Post by: StupidBeard on July 16, 2015, 11:24:14 PM
Thanks for replying. There was nothing gmac related in dmesg, but a grep on eth0 found:

Code: [Select]
root@lime2:~# dmesg|grep eth0
[    1.881348] eth0: PHY ID 001cc912 at 1 IRQ POLL (stmmac-0:01) active
[   12.097433] stmmaceth 1c50000.ethernet eth0: Link is Up - 1Gbps/Full - flow control rx/tx

That was with kernel 4.1.2; I am going to try the image with the 3.4.108 in a sec to see if that is using gmac.

How do I go about changing this to use the gmac driver instead?
Title: Re: Extremely slow Ethernet on A20-OLinuXino-LIME2
Post by: StupidBeard on July 16, 2015, 11:36:50 PM
I just tried the 3.4.108 image, which is definitely using the gmac driver and it is exhibiting the same problem. Actually it seems to be worse, currently getting 12KB/sec on the 10MB wget.

Edit:

I manually changed the link speed to 100mbit full duplex using ethtool and now I'm getting 11MB/sec on the 10MB test file. This is sounding to me like some kind of driver not being able to keep up issue.
Title: Re: Extremely slow Ethernet on A20-OLinuXino-LIME2
Post by: Gerrit on July 16, 2015, 11:48:29 PM
lets go for the obvious then, replace the cable or try an other port on your switch
Title: Re: Extremely slow Ethernet on A20-OLinuXino-LIME2
Post by: StupidBeard on July 16, 2015, 11:52:21 PM
lets go for the obvious then, replace the cable or try an other port on your switch

Those were actually the first things I tried, including another switch entirely. The only thing that has made any difference was forcing the link speed to 100mbit, which I mentioned in the edit to previous post that was probably posted at the same time you were posting this :)
Title: Re: Extremely slow Ethernet on A20-OLinuXino-LIME2
Post by: soenke on July 17, 2015, 12:11:21 PM
You already tested, that your LAN works fine. To 10.0.0.10 you get > 700Mbit.
My guess is your internet-router. Their interface is often only 100Mbit so maybe there is a lot of packet dropping if you use 1Gbit on your LIME which is then slowing down the TCP connection. Can you try your internet-speed from your 10.0.0.10-machine using its Gbit-Interface?

Try using something like wireshark to see if you get NACK-packets from your router.
Title: Re: Extremely slow Ethernet on A20-OLinuXino-LIME2
Post by: StupidBeard on July 17, 2015, 03:33:47 PM
You already tested, that your LAN works fine. To 10.0.0.10 you get > 700Mbit.
My guess is your internet-router. Their interface is often only 100Mbit so maybe there is a lot of packet dropping if you use 1Gbit on your LIME which is then slowing down the TCP connection. Can you try your internet-speed from your 10.0.0.10-machine using its Gbit-Interface?

Try using something like wireshark to see if you get NACK-packets from your router.

It's not the internet connection. I did spend a bunch of time initially going down that rabbit hole, but I ruled that out entirely by restricting testing to the LAN only, with Gigabit only.

iperf3 does show that the LIME and the LAN is capable of reasonable speed even if it's not the full gigabit, but as soon as you try and do anything useful with it (e.g. wget, scp) it gets bogged down in molasses and is essentially unusable.

Repeating the same tests with eth0 forced to 100mbit full duplex, with all other things remaining the same, results in roughly expected speeds.

I think that I have conclusively ruled out everything outside of the LIME, which just leaves a software or hardware issue.

Wireshark is a good suggestion though, I will look into that tonight.
Title: Re: Extremely slow Ethernet on A20-OLinuXino-LIME2
Post by: StupidBeard on July 17, 2015, 03:45:14 PM
I did some more digging, and came across https://linux-sunxi.org/Ethernet

Under GMAC, it states:

Quote
For reliable Gigabit networking (1000Mbit operation), several sunxi devices require an important tweak that adjusts the relative timing of the clock and data signals to the PHY, in order to compensate for differing trace lengths on the PCB (details). [snip] Recent mainline U-Boot uses CONFIG_GMAC_TX_DELAY to initialize these devices accordingly. If a necessary GMAC TX delay isn't set, then GBit Ethernet operation might be unreliable or won't work at all. 10/100 Mbit/sec negotiation is unaffected, so misconfigured devices could actually work (faster) when connected to a Fast Ethernet port instead of a GBit Ethernet port.

This is basically exactly what I'm seeing, so will look into that tonight.
Title: Re: Extremely slow Ethernet on A20-OLinuXino-LIME2
Post by: soenke on July 17, 2015, 04:40:54 PM
well, still i think it is not a communication problem between your LAN interface and the switch or a LIME hardware problem, else it would not be possible to get 700MBit to your local server. So physical communication works.
So it has to be a >layer1-problem. Maybe your linux sends jumbo packets if it is run on gigabit and your router discards/fragements them? Maybe you could also check MTU, look after fragmented packets in wireshark.

Btw, are you using VLANs? They are adding some bytes to all packets and some interfaces cant handle this.

Try to use your local server as router for your lime-box and see if that changes anything. Put your lime into a separate subnet for that to be sure the packets are not going to your router directly.
Title: Re: Extremely slow Ethernet on A20-OLinuXino-LIME2
Post by: ThibG on July 22, 2015, 01:28:06 AM
Have you resolved your issue? I'm experiencing something similar on my two LIME2. I measured 500Mbps+ between them, but then, a few hours later, one of them started to drop packets, as much as 10%! Rebooting solved the issue, but still…

Another issue I had twice but can't reproduce is my second LIME2 (now plugged to a 100Mbps port) completely stopped being reachable through the ethernet port (unplugging/replugging the cable did not change anything, but did confirm the system was not dead, as this would appear in the logs). That might be unrelated, though. In both cases, a reboot solved it. This LIME2 is plugged to both AC and a LiPo battery.

My two LIME2 are running vanilla Debian Jessie with a 4.0 kernel and 2015-04 u-boot from testing.
Title: Re: Extremely slow Ethernet on A20-OLinuXino-LIME2
Post by: StupidBeard on July 22, 2015, 01:34:47 AM
Have you resolved your issue? I'm experiencing something similar on my two LIME2. I measured 500Mbps+ between them, but then, a few hours later, one of them started to drop packets, as much as 10%! Rebooting solved the issue, but still…

Another issue I had twice but can't reproduce is my second LIME2 (now plugged to a 100Mbps port) completely stopped being reachable through the ethernet port (unplugging/replugging the cable did not change anything, but did confirm the system was not dead, as this would appear in the logs). That might be unrelated, though. In both cases, a reboot solved it. This LIME2 is plugged to both AC and a LiPo battery.

My two LIME2 are running vanilla Debian Jessie with a 4.0 kernel and 2015-04 u-boot from testing.

No, but from everything I could find it looks like it's related to the CONFIG_GMAC_TX_DELAY in uboot. It looks like to fix it will require building a custom uboot and fiddling with said setting to find one that works for the LIME2.

The links I found are on another computer right now, but if you search for CONFIG_GMAC_TX_DELAY then you should find more information on the issue.

I meant to look into sorting it out at the weekend, but I got sidetracked with other things so I probably won't get around to it until next weekend now.
Title: Re: Extremely slow Ethernet on A20-OLinuXino-LIME2
Post by: soenke on July 22, 2015, 10:10:47 AM
Another issue I had twice but can't reproduce is my second LIME2 (now plugged to a 100Mbps port) completely stopped being reachable through the ethernet port (unplugging/replugging the cable did not change anything, but did confirm the system was not dead, as this would appear in the logs). That might be unrelated, though. In both cases, a reboot solved it. This LIME2 is plugged to both AC and a LiPo battery.

My two LIME2 are running vanilla Debian Jessie with a 4.0 kernel and 2015-04 u-boot from testing.

You can try to connect via UART (the pins besides the ethernet port) and see if the local console is still works?
I had similar effects but in the end it was related to randomly failing microSD-Card-controller. Some systems stopped after some hours, others after some months with MMC-errors. The only way to pinpoint this error was a live connection to UART0 and see the kernel output. New SD-Card (SDSDSQAE-032G) --> problems solved!
Title: Re: Extremely slow Ethernet on A20-OLinuXino-LIME2
Post by: dario on November 18, 2015, 10:11:54 PM
Unfortunately I have the same problem. Has anybody found a fix?
Title: Re: Extremely slow Ethernet on A20-OLinuXino-LIME2
Post by: JohnS on November 19, 2015, 01:06:50 AM
Look at the post before yours.  Use uart output / nw SD card.

John
Title: Re: Extremely slow Ethernet on A20-OLinuXino-LIME2
Post by: soenke on November 19, 2015, 01:13:14 AM
well, which of the problems are you referring to? Network packet losses or system freeze?
Title: Re: Extremely slow Ethernet on A20-OLinuXino-LIME2
Post by: dario on November 20, 2015, 12:45:28 AM
Those who faced the issue said it is a problem with a specific variable in U-Boot, namely CONFIG_GMAC_TX_DELAY. I thought the SD card thing was not related. I have a class 4 microSD that is quite slow indeed. Why would a faster SD card fix the problem? Perhaps it is a caching related issue? I will try with a faster SD card and let you know.
Title: Re: Extremely slow Ethernet on A20-OLinuXino-LIME2
Post by: dario on November 20, 2015, 04:44:55 PM
I am referring to the packet loss in 1000 Mb/s mode.
Title: Re: Extremely slow Ethernet on A20-OLinuXino-LIME2
Post by: soenke on November 20, 2015, 04:55:14 PM
Then the timing of gmac might be the solution, please report back if that works.

You can also try if a faster sd card helps (class 4 is quite slow for running an os) but my guess is that it is not related to the packet losses. Also try another cable and adding another gbit-switch between the A20 and the router, it could also be related to the physical signals.
Title: Re: Extremely slow Ethernet on A20-OLinuXino-LIME2
Post by: dario on November 20, 2015, 07:46:43 PM
Thank you for your reply. Forcing 100 MB/s via software eliminates the problem, therefore I think the issue is unrelated. I will try with a faster microSD though (I know, it's quite slow). I have been looking on the Internet if anybody had come up with a solution to no avail. It seems nobody has yet to find a good combination of values for proper 1000 Mb/s connection. Do you own an A20-OLinuXIno-LIME2? Are you experiencing the same issue?
Title: Re: Extremely slow Ethernet on A20-OLinuXino-LIME2
Post by: soenke on November 20, 2015, 10:38:02 PM
Yes, i have some A20's but none of them is on a gigabit switch ;)
But i will make some tests to see if i can reproduce it.
Title: Re: Extremely slow Ethernet on A20-OLinuXino-LIME2
Post by: dario on November 21, 2015, 12:04:14 AM
OK, thank you very much.  :)
Title: Re: Extremely slow Ethernet on A20-OLinuXino-LIME2
Post by: Gronis93 on November 30, 2015, 07:54:33 PM
Hi everyone!

I have the same issue with my lime2 board, only that it is only on the downlink. The uplink has about 60MB/s (480Mbit/s) but the downlink is stuck at about 50kB/s which is terrible.

I bought this board only to run as a server with a gigabit link. I even upgraded my router, so that I could use the gigabit speeds. I have had this problem since I bought the board earlier this year. I have tried two different routers, 3 different cables, with and without a switch in between the router and the board, but the problem remains.

From what I have learned reading this thread, it seams like a software issue.  I'm not sure if I can help in this discussion, but like you guys, I really want to solve this problem :)

I'm using kernel 4.1.2
Title: Re: Extremely slow Ethernet on A20-OLinuXino-LIME2
Post by: JohnS on November 30, 2015, 08:19:22 PM
It does seem to be software and it appears to be open source so go for it if you want it faster :)

John
Title: Re: Extremely slow Ethernet on A20-OLinuXino-LIME2
Post by: dario on November 30, 2015, 11:46:50 PM
I have yet to get hold of a faster microSD to try out, but I'm pretty confident it won't fix the issue. Gronis93, what's the actual speed of the microSD with the operating system installed?
Title: Re: Extremely slow Ethernet on A20-OLinuXino-LIME2
Post by: Gronis93 on December 01, 2015, 01:01:26 AM
I have tested IO speeds with dd and get around 8MB/s with when writing zeroes to the home folder, and 19MB/s when reading a file into /dev/null

So there should not be a problem with the sd-card. I have ordered a new sd-card and I am going to try different distributions and kernels on that sd-card (don't want to ruin my server install for the moment). Dario, how fast is your sd-card?
Title: Re: Extremely slow Ethernet on A20-OLinuXino-LIME2
Post by: Gronis93 on December 01, 2015, 01:05:39 AM
Also, I want to add that if I start many simultaneous downloads, all of them are capped at 100~30kB/s. This is a further indication that the speed is capped by the software. Probably there is a large amount of package loss witch slows it down.
Title: Re: Extremely slow Ethernet on A20-OLinuXino-LIME2
Post by: StupidBeard on December 01, 2015, 01:34:00 AM
Hey,

Gronis sent me a PM, which I replied to, but thought I would also mention it here for anyone else who comes along later.

I gave up swearing at it and just plugged the board into a 100mbit switch. An alternative solution is to use ethtool in a pre-up script for force eth0 to always use 100mbit.

I'd still like a proper solution, but I have so many other things to do that it's unlikely I'll get back to this for quite a long while.

Stupid Beard.
Title: Re: Extremely slow Ethernet on A20-OLinuXino-LIME2
Post by: dario on December 01, 2015, 05:45:00 AM
Gronis93, I have tested the speed the same way as you did and I use my OLinuXino as a network server as you do. Actually, it is quite slow and my microSD is the same as yours. I get the same speed. It is a class 4 microSD card and it's too slow for general use. In regard to what StupidBeard wrote in the last post, it seems the only workaround so far is to connect the board to a 100Mb/s switch OR cap the speed via software with something like
Code: [Select]
mii-tool eth0 -F 100baseTx-FD This solution, however, is not feasible for my needs because I get very slow transfer speed on my LAN. I bought this board specifically for the gigabit capability.
Title: Re: Extremely slow Ethernet on A20-OLinuXino-LIME2
Post by: dario on December 04, 2015, 12:49:31 AM
I have tried the official Debian images from Olimex, the Debian image from the Debian website and Arch Linux to no avail, it's slow as hell.

EDIT: I filed a bug report to Olimex on GitHub: https://github.com/OLIMEX/OLINUXINO/issues/31
Title: Re: Extremely slow Ethernet on A20-OLinuXino-LIME2
Post by: Rainer on December 10, 2015, 05:45:19 PM
I tested 2 A20-SOM board we received last week with the original kernel (3.4.103) and a buildroot filesystem.
Test with UDP seams be ok but with TCP and scp by 1000 Mbit the performace ist not ok.
After switching to 100Mbit the performance seams to be ok.

iperf -c 192.168.99.2 -u -b 1000m
------------------------------------------------------------
Client connecting to 192.168.99.2, UDP port 5001
Sending 1470 byte datagrams
UDP buffer size:  160 KByte (default)
------------------------------------------------------------
[  3] local 192.168.99.179 port 33871 connected with 192.168.99.2 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec   592 MBytes   497 Mbits/sec
[  3] Sent 422564 datagrams
[  3] Server Report:
[  3]  0.0-10.0 sec   592 MBytes   497 Mbits/sec   0.040 ms    0/422563 (0%)

iperf -c 192.168.99.2
------------------------------------------------------------
Client connecting to 192.168.99.2, TCP port 5001
TCP window size: 21.0 KByte (default)
------------------------------------------------------------
[  3] local 192.168.99.106 port 41114 connected with 192.168.99.2 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-33.6 sec  1.63 MBytes   406 Kbits/sec

mii-diag  eth0 -F 100baseTx-FD
iperf -c 192.168.99.2
------------------------------------------------------------
Client connecting to 192.168.99.2, TCP port 5001
TCP window size: 21.0 KByte (default)
------------------------------------------------------------
[  3] local 192.168.99.106 port 41115 connected with 192.168.99.2 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec  58.4 MBytes  48.9 Mbits/sec
Title: Re: Extremely slow Ethernet on A20-OLinuXino-LIME2
Post by: soenke on December 10, 2015, 07:13:35 PM
well, that is not good...

Can you check your interface statistics for broken/dropped RX/TX packets? The same on the target machine.
Title: Re: Extremely slow Ethernet on A20-OLinuXino-LIME2
Post by: dario on December 11, 2015, 10:57:00 PM
According to the following post, the problem is shared among many other boards based on A20: http://forum.armbian.com/index.php/topic/31-problems-with-gigabit/
Title: Re: Extremely slow Ethernet on A20-OLinuXino-LIME2
Post by: infertux on February 23, 2016, 11:08:32 PM
Same problem here, I'm hovering at 15kB/s in gigabit mode whereas I easily reach 11MB/s in 100 Mbit mode.

I've added this in my /etc/network/interfaces as a temporary workaround:

Code: [Select]
auto eth0
iface eth0 inet static
  [...]
  up mii-tool -F 100baseTx-FD eth0

Kernel: 4.4.1-sunxi
Title: Re: Extremely slow Ethernet on A20-OLinuXino-LIME2
Post by: fallenguru on March 10, 2016, 10:03:14 PM
Same here.

~40 KiB/s on average over a supposed Gigabit connection.

It's not the SD card, I've tried three, one a brand new Samsung Pro. There are no errors in dmesg, SD-related or otherwise. That's on vanilla Debian stretch as of today, using 4.3.0's stmmac. The network behaves as expected when forced down to 100Mb/s. (Which is not an acceptable workaround long-term because I bought the LIME2 especially for its Gbit interface.)
Title: Re: Extremely slow Ethernet on A20-OLinuXino-LIME2
Post by: fradav on March 11, 2016, 04:08:45 PM
Same here.

~40 KiB/s on average over a supposed Gigabit connection.

It's not the SD card, I've tried three, one a brand new Samsung Pro. There are no errors in dmesg, SD-related or otherwise. That's on vanilla Debian stretch as of today, using 4.3.0's stmmac. The network behaves as expected when forced down to 100Mb/s. (Which is not an acceptable workaround long-term because I bought the LIME2 especially for its Gbit interface.)

Same here, too, joining the pack of disgruntled "native gigabit" users.
I think the potential is there, because when I use a relatively fast usb3/ethernet adapter to my main pc and plug the lime2 to it, after the first disconnect/reconnect the gigabit speed is really there. But going back to a "normal" switch and the lime2 is reverting to the old PSTN modem speed-land.

I guess something is really going on at PHY level but I don't get it (and twiddling GMAC_TX_DELAY in uboot didn't change anything)...
If someone can confirm than he gets real gigabit speed with the lime2, please post your results, and what did you use as uboot/kernel...

Regards
Title: Re: Extremely slow Ethernet on A20-OLinuXino-LIME2
Post by: JohnS on March 11, 2016, 06:03:31 PM
Don't all A20 boards have the problem?

John
Title: Re: Extremely slow Ethernet on A20-OLinuXino-LIME2
Post by: fallenguru on March 12, 2016, 12:57:23 PM
Olimex support say it "works with some switches/routers and [...] not [...] with others" -- has *anyone* had success with the LIME2 connected to a Gigabit switch at 1000 Mb/s full duplex? Meaning no packet loss and acceptable bandwidth (say >500Mb/s) in both directions? If so please post the make & model of the switch. Maybe incompatible switches would also be useful.

(You can use ping and iperf3 to measure.)

I'll start and say that the Zyxel GS1900 series does NOT work.
Title: Re: Extremely slow Ethernet on A20-OLinuXino-LIME2
Post by: olHelp on March 12, 2016, 05:00:32 PM
Just testet my Lime2 against the openWRT router it is wired to.
iperf -c on the router, and iperf -c on the lime gives ~250Mbit/sek. COuld be more, but the measly MIPS cpu in the router is at 100%.

Other way around only gives 120Mbit/sek, but again with 100%load on the router. 

Dont have any wired deviced to test, but pinging the lime at the same time from antoher computer returns all pings in the expected timeframe, no losses.

kernel is 4.4.3-1-ARCH
Title: Re: Extremely slow Ethernet on A20-OLinuXino-LIME2
Post by: fradav on March 15, 2016, 11:42:24 AM
Just testet my Lime2 against the openWRT router it is wired to.
iperf -c on the router, and iperf -c on the lime gives ~250Mbit/sek. COuld be more, but the measly MIPS cpu in the router is at 100%.

Other way around only gives 120Mbit/sek, but again with 100%load on the router. 

Dont have any wired deviced to test, but pinging the lime at the same time from antoher computer returns all pings in the expected timeframe, no losses.

kernel is 4.4.3-1-ARCH

Hi,

Self compiled kernel or not ? Do you know Which distro? uboot ?

Regards
Title: Re: Extremely slow Ethernet on A20-OLinuXino-LIME2
Post by: olHelp on March 15, 2016, 10:04:31 PM
No frills arch linux installation, running from sd card
Title: Re: Extremely slow Ethernet on A20-OLinuXino-LIME2
Post by: dancri77 on March 16, 2016, 11:31:29 PM
I have the same issue. I run the test between an Intel machine running linux, with gigabit ethernet, and Lime2.
My switch is a 24-port gigabit. Clearly no problem with my network equipment or cables as I have several PCs in my network with gigabit links working OK.

I run the Vanilla Jessie image from http://www.armbian.com/olimex-lime-2/.

DOWNLINK seems fine, getting max 701 Mbits/sec. See below.
UPLINK is terrible, avg. is 3 Mbits/s.

-------------------DOWNLINK------------------------------------
root@lime2:~# iperf3 -c 192.168.0.11
Connecting to host 192.168.0.11, port 5201
[  4] local 192.168.0.27 port 58414 connected to 192.168.0.11 port 5201
[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.00   sec  59.1 MBytes   495 Mbits/sec    0    204 KBytes
[  4]   1.00-2.02   sec  69.5 MBytes   575 Mbits/sec    0    335 KBytes
[  4]   2.02-3.00   sec  65.4 MBytes   557 Mbits/sec    0    506 KBytes
[  4]   3.00-4.02   sec  65.5 MBytes   541 Mbits/sec    0    765 KBytes
[  4]   4.02-5.00   sec  77.5 MBytes   660 Mbits/sec    0    765 KBytes
[  4]   5.00-6.00   sec  79.6 MBytes   666 Mbits/sec    0   1.02 MBytes
[  4]   6.00-7.00   sec  81.2 MBytes   683 Mbits/sec    0   1.33 MBytes
[  4]   7.00-8.01   sec  78.8 MBytes   656 Mbits/sec    0   1.33 MBytes
[  4]   8.01-9.00   sec  77.5 MBytes   653 Mbits/sec    0   1.50 MBytes
[  4]   9.00-10.01  sec  83.8 MBytes   701 Mbits/sec    0   1.50 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.01  sec   738 MBytes   619 Mbits/sec    0             sender
[  4]   0.00-10.01  sec   738 MBytes   619 Mbits/sec                  receiver

--------------------UPLINK------------------------------------------------

root@lime2:~# iperf3 -c 192.168.0.11 -R
Connecting to host 192.168.0.11, port 5201
Reverse mode, remote host 192.168.0.11 is sending
[  4] local 192.168.0.27 port 58418 connected to 192.168.0.11 port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec   650 KBytes  5.33 Mbits/sec
[  4]   1.00-2.00   sec   431 KBytes  3.53 Mbits/sec
[  4]   2.00-3.00   sec   291 KBytes  2.39 Mbits/sec
[  4]   3.00-4.00   sec   505 KBytes  4.14 Mbits/sec
[  4]   4.00-5.00   sec   535 KBytes  4.38 Mbits/sec
[  4]   5.00-6.00   sec   240 KBytes  1.97 Mbits/sec
[  4]   6.00-7.00   sec   181 KBytes  1.48 Mbits/sec
[  4]   7.00-8.00   sec  77.8 KBytes   637 Kbits/sec
[  4]   8.00-9.00   sec   573 KBytes  4.69 Mbits/sec
[  4]   9.00-10.00  sec   216 KBytes  1.77 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec  3.63 MBytes  3.05 Mbits/sec  373             sender
[  4]   0.00-10.00  sec  3.61 MBytes  3.03 Mbits/sec                  receiver
Title: Re: Extremely slow Ethernet on A20-OLinuXino-LIME2
Post by: dancri77 on March 16, 2016, 11:56:38 PM
Changing the eth0 interface speed to 100 mbit gives stable 94 Mbit/s rate in both direction.

ethtool -s eth0 speed 100 duplex full

root@lime2:~# iperf3 -c 192.168.0.9 -R
Connecting to host 192.168.0.9, port 5201
Reverse mode, remote host 192.168.0.9 is sending
[  4] local 192.168.0.27 port 52488 connected to 192.168.0.9 port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec  11.3 MBytes  94.5 Mbits/sec
[  4]   1.00-2.00   sec  11.2 MBytes  93.9 Mbits/sec
[  4]   2.00-3.00   sec  11.2 MBytes  93.9 Mbits/sec
[  4]   3.00-4.00   sec  11.2 MBytes  94.1 Mbits/sec
[  4]   4.00-5.00   sec  11.2 MBytes  94.1 Mbits/sec
[  4]   5.00-6.00   sec  11.2 MBytes  93.9 Mbits/sec
[  4]   6.00-7.00   sec  11.2 MBytes  94.0 Mbits/sec
[  4]   7.00-8.00   sec  11.2 MBytes  94.1 Mbits/sec
[  4]   8.00-9.00   sec  11.2 MBytes  94.1 Mbits/sec
[  4]   9.00-10.00  sec  11.2 MBytes  94.1 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec   112 MBytes  94.2 Mbits/sec    0             sender
[  4]   0.00-10.00  sec   112 MBytes  94.2 Mbits/sec                  receiver



root@lime2:~# iperf3 -c 192.168.0.9
Connecting to host 192.168.0.9, port 5201
[  4] local 192.168.0.27 port 52484 connected to 192.168.0.9 port 5201
[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.00   sec  11.4 MBytes  95.7 Mbits/sec    0    219 KBytes
[  4]   1.00-2.00   sec  11.2 MBytes  94.4 Mbits/sec    0    236 KBytes
[  4]   2.00-3.00   sec  11.1 MBytes  93.3 Mbits/sec    0    246 KBytes
[  4]   3.00-4.00   sec  11.5 MBytes  96.4 Mbits/sec    0    263 KBytes
[  4]   4.00-5.00   sec  11.0 MBytes  92.3 Mbits/sec    0    263 KBytes
[  4]   5.00-6.00   sec  11.3 MBytes  94.9 Mbits/sec    0    263 KBytes
[  4]   6.00-7.00   sec  11.2 MBytes  93.8 Mbits/sec    0    263 KBytes
[  4]   7.00-8.00   sec  11.4 MBytes  95.4 Mbits/sec    0    263 KBytes
[  4]   8.00-9.00   sec  11.0 MBytes  92.3 Mbits/sec    0    263 KBytes
[  4]   9.00-10.00  sec  11.7 MBytes  98.5 Mbits/sec    0    413 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec   113 MBytes  94.7 Mbits/sec    0             sender
[  4]   0.00-10.00  sec   112 MBytes  93.9 Mbits/sec                  receiver


Title: Re: Extremely slow Ethernet on A20-OLinuXino-LIME2
Post by: fallenguru on March 20, 2016, 09:22:40 PM
No change with an old dumb 3Com OfficeConnect Gigabit Switch 8. The kernel doesn't make a difference either, I've tried the legacy sunxi one and the 4.4 that's in Debian testing.
Title: Re: Extremely slow Ethernet on A20-OLinuXino-LIME2
Post by: dario on March 27, 2016, 12:29:21 AM
Some work was done. There is a patch for U-Boot here: https://github.com/OLIMEX/OLINUXINO/issues/31

I have personally tested the patch posted there. It works pretty good!
Title: Re: Extremely slow Ethernet on A20-OLinuXino-LIME2
Post by: att2 on March 27, 2016, 10:26:09 PM
I have the same issue: Internet speed of 15 kilobytes/sec.
I will try to switch to 100mbit as above and see if that fixes anything.
I use the latest Olimex A20 image via torrent as described on the Olimex homepage.
Title: Re: Extremely slow Ethernet on A20-OLinuXino-LIME2
Post by: LubOlimex on March 29, 2016, 08:51:20 AM
Hey,

We are preparing new builds and official images that would include a workaround based on the uboot patches made by Michael Haas. The new images should improve the behavior of the Gigabit interface. For the moment, try to replace u-boot with this one (there are instructions inside):

Edit 31.03.2016 - added u-boot v2015.10 (the first one used v2015.10-rc1 is still available in the archive):

https://drive.google.com/file/d/0BwplT87k9SCgSmZEcEJWOUNKbzQ/view?usp=sharing (https://drive.google.com/file/d/0BwplT87k9SCgSmZEcEJWOUNKbzQ/view?usp=sharing)

Refer to the README.txt file inside the archive for more information.

Thanks for the dedication of Michael Haas, Karsten Merker, Joe Hershberger, Hans de Goede and everybody I must have missed but got involved.

Best regards,
Lub/OLIMEX
Title: Re: Extremely slow Ethernet on A20-OLinuXino-LIME2
Post by: imrehg on March 29, 2016, 10:09:30 AM
Hi, just tried out the new u-boot, and works very well for the short amount of time I've checked, thanks!

Been using ArchLinuxARM, where the default u-boot (2016.01) is non-bootable (can see the forum posts (https://archlinuxarm.org/forum/viewtopic.php?f=33&t=9911)), so this was a doubly good replacement.
Title: Re: Extremely slow Ethernet on A20-OLinuXino-LIME2
Post by: LubOlimex on March 29, 2016, 10:48:19 AM
@imrehg

The problem with the newest u-boots is related to the LDO3/4 values. We are currently investigating it more thoroughly. There is a discussion still going on.

Unless you know what you are doing, I can't really recommend using the newest u-boots or any suggested workarounds for setting LDO3/4 with values suitable for successful boot.

Read more about that here: https://www.mail-archive.com/linux-sunxi@googlegroups.com/msg17049.html (https://www.mail-archive.com/linux-sunxi@googlegroups.com/msg17049.html)

Best regards,
Lub/OLIMEX
Title: Re: Extremely slow Ethernet on A20-OLinuXino-LIME2
Post by: mhaas on March 31, 2016, 07:42:19 AM
@imrehg
Quote
Been using ArchLinuxARM, where the default u-boot (2016.01) is non-bootable (can see the forum posts), so this was a doubly good replacement.

I've had the same problem on Debian Jessie. Does your boot simply hang somewhere shortly before login? I could workaround that by blacklisting the axp20x-i2c.ko module (or rather, mv64xxx.ko, but axp20x-i2c.ko should be the real culprit).


@lubolimex
Quote
The problem with the newest u-boots is related to the LDO3/4 values. We are currently investigating it more thoroughly. There is a discussion still going on.

I'd love to hear your input on that matter on the mailing list :) I'll be preparing a patch with Hans' suggested changes (enabling LDO3/LDO4 im u-boot) shortly.
Title: Re: Extremely slow Ethernet on A20-OLinuXino-LIME2
Post by: chradev on April 02, 2016, 11:51:08 PM
Hi to All,

I have started a new thread (https://www.olimex.com/forum/index.php?topic=5180.0 (https://www.olimex.com/forum/index.php?topic=5180.0)) about A20-OLinuXino-LIME2-eMMC with new HW rev. E.

The main differences in new HW rev D and E are:
As told me from Olimex the slow GB Ethernet problem is solved thanks to SW patch only (probably one from  Igor Pecovnik).

Unfortunately, my observations are that most probably re-routed Ethernet part in new HW rev. D and E is the main reason (see pointed above thread).

Best regarsd
Chris
Title: Re: Extremely slow Ethernet on A20-OLinuXino-LIME2
Post by: mhaas on April 03, 2016, 09:33:53 AM

Unfortunately, my observations are that most probably re-routed Ethernet part in new HW rev. D and E is the main reason (see pointed above thread).

Best regarsd
Chris
Hi Chris,

So, you disabled the ldo3/ldo4 patch to find out if GBit is still working? In that case, you disabled the wrong patch. You are looking for drivers/net/phy/realtek.c.

@lubolimex
Does the rerouted Ethernet require a different value for the GMAC clock delays?
Title: Re: Extremely slow Ethernet on A20-OLinuXino-LIME2
Post by: chradev on April 03, 2016, 11:29:53 AM
@mhaas,
I do not disable anything. I simply use Armbian 5.07 "as-is" as a patch set and note that the ldo3/ldo4 patch is present but disabled.
I find that old Lime 2 rev. C is not working with Armbian 5.07 "as-is" while new Lime 2 rev. E is working without any modifications.

Did you have Lime 2 HW rev. E for testing and is there any difference around patches?

@lubolimex
Could you share is there SW changes (and what if any) at migration to new Lime 2 HW rev. E?
What is the SW solution (if any) for old Lime 2 HW rev. C to become usable as GBit Ethernet?
Could you share how to switch from default NAND to eMMC option on Lime 2 HW rev. E as well?
Title: Re: Extremely slow Ethernet on A20-OLinuXino-LIME2
Post by: mhaas on April 03, 2016, 04:36:38 PM
@chradev,

Sorry for my confusion there :)

I do not have a rev E and can't really comnment on the effect of the re-routing changes. Please be aware that the GBit ethernet bug only happens sporadically. It depends on the auto-negotiation of the link, i.e. which device becomes master and which device becomes slave.

I tried looking up Armbian 5.07 to check the versions of affected packages, but couldn't find it. Must be my weak google-fu.
Title: Re: Extremely slow Ethernet on A20-OLinuXino-LIME2
Post by: fradav on April 03, 2016, 07:15:49 PM
Hey,

We are preparing new builds and official images that would include a workaround based on the uboot patches made by Michael Haas. The new images should improve the behavior of the Gigabit interface. For the moment, try to replace u-boot with this one (there are instructions inside):

Edit 31.03.2016 - added u-boot v2015.10 (the first one used v2015.10-rc1 is still available in the archive):

https://drive.google.com/file/d/0BwplT87k9SCgSmZEcEJWOUNKbzQ/view?usp=sharing (https://drive.google.com/file/d/0BwplT87k9SCgSmZEcEJWOUNKbzQ/view?usp=sharing)

Refer to the README.txt file inside the archive for more information.

Thanks for the dedication of Michael Haas, Karsten Merker, Joe Hershberger, Hans de Goede and everybody I must have missed but got involved.

Best regards,
Lub/OLIMEX

It should be noted than your provided uboot, which is completely functionnal and somewhat "solves" the aforementioned  problems (crash during boot and/or gigabit rx slow), isn't optimal : I got < 500Mb/s speed on rx/tx.
To get 900Mb/s speed class you really need latest uboot. Alas, latest uboot git broke something related to the usb phy and sank the uboot in a bootloop (pardon the pun).
So for a complete functional and full speed Gbit uboot, here is the recipe :
the official v2016.03 (tag) from the git.
And the cherry picked following commits :
525d187afb418a4deef8916844f5f7744da402a8
7e4e7e4618c67311fac0d4f8af6bedbbbfe68c9b
53866b6fc54142a2e7ab79d1c2b0f4313c29a2ec
affa020559bca31d6531e19cb1f009c22705a73d
563d8d93585035fa2a0b20190c1008eb625432c4

And you're set.
On a side note, congrats to olimex for designing a board with native and *real* Gbit (an by real I mean 900Mb/s with TCP), it's really astonishing to achieve this for an sbc in this budget class.

Regards,
Title: Re: Extremely slow Ethernet on A20-OLinuXino-LIME2
Post by: JohnS on April 04, 2016, 12:13:36 PM
Did you let the u-boot maintainer know?

John
Title: Re: Extremely slow Ethernet on A20-OLinuXino-LIME2
Post by: fradav on April 04, 2016, 01:59:37 PM
Did you let the u-boot maintainer know?

John

They have been aware: in fact, the commiter who caused the bug already patched in master branch (I'm running it on my lime2 right now).

Regards,

Title: Re: Extremely slow Ethernet on A20-OLinuXino-LIME2
Post by: JohnS on April 04, 2016, 02:07:27 PM
Great news.

John
Title: Re: Extremely slow Ethernet on A20-OLinuXino-LIME2
Post by: chradev on April 20, 2016, 08:25:01 PM
Hi Guys,

Hey,

We are preparing new builds and official images that would include a workaround based on the uboot patches made by Michael Haas. The new images should improve the behavior of the Gigabit interface. For the moment, try to replace u-boot with this one (there are instructions inside):

Edit 31.03.2016 - added u-boot v2015.10 (the first one used v2015.10-rc1 is still available in the archive):

https://drive.google.com/file/d/0BwplT87k9SCgSmZEcEJWOUNKbzQ/view?usp=sharing (https://drive.google.com/file/d/0BwplT87k9SCgSmZEcEJWOUNKbzQ/view?usp=sharing)

Refer to the README.txt file inside the archive for more information.

Thanks for the dedication of Michael Haas, Karsten Merker, Joe Hershberger, Hans de Goede and everybody I must have missed but got involved.

Best regards,
Lub/OLIMEX

It should be noted than your provided uboot, which is completely functionnal and somewhat "solves" the aforementioned  problems (crash during boot and/or gigabit rx slow), isn't optimal : I got < 500Mb/s speed on rx/tx.
To get 900Mb/s speed class you really need latest uboot. Alas, latest uboot git broke something related to the usb phy and sank the uboot in a bootloop (pardon the pun).
So for a complete functional and full speed Gbit uboot, here is the recipe :
the official v2016.03 (tag) from the git.
And the cherry picked following commits :
525d187afb418a4deef8916844f5f7744da402a8
7e4e7e4618c67311fac0d4f8af6bedbbbfe68c9b
53866b6fc54142a2e7ab79d1c2b0f4313c29a2ec
affa020559bca31d6531e19cb1f009c22705a73d
563d8d93585035fa2a0b20190c1008eb625432c4

And you're set.
On a side note, congrats to olimex for designing a board with native and *real* Gbit (an by real I mean 900Mb/s with TCP), it's really astonishing to achieve this for an sbc in this budget class.

Regards,

As a part of Armbian customization I use U-Boot ver. 2015.10 (because ver. 2016.03 does not boot on Lime2-eMMC boards used by me) and Olimex solution to fix slow GBit Ethernet published in their latest image.

Some time ago I have made a series of tests including Ethernet performance which summary results can be found on: http://forum.armbian.com/index.php/topic/853-armbian-customization/page-2#entry7762 (http://forum.armbian.com/index.php/topic/853-armbian-customization/page-2#entry7762). There I have measured (with iperf) speeds up to 980Mbps over GBit LAN.

These days after finishing some work around eMMC support I try to migrate to newer U-Boot and make some coarse assessment for last versions published here: http://forum.armbian.com/index.php/topic/853-armbian-customization/page-4#entry7971 (http://forum.armbian.com/index.php/topic/853-armbian-customization/page-4#entry7971).

For my surprise results from first and last measurements are too different without ability to explain why. One reason can be that I have 2 boards A20-Olimexino-Lime2-eMMC (HW ver. E) and tests can be done on the other board. The second board is not in my hand at the moment to check it but according to Olimex' statement provided solution has to work on any board even on old HW rev. C.

Other strange fact is that the fix does not work for U-Boot ver. 2016.01 at all.

Unfortunately, I did not check if all mentioned by fradav staff is applied to U-Boot (most probably not or at leas not as a whole). I use U-Boot as it is integrated into Armbian and the only patches I add are for eMMC support and slow Ethernet fix.

Can somebody confirm that?

Best regards
Chris