[Babel-users] tunnels

Wed Nov 7 19:30:21 GMT 2018

Christof Schulze <christof.schulze at gmx.net> writes:

> Hello List, Dave,
>
>
> With bbr on the destination host (running wireguard) and on the client
> device (regular linux) but not on the two hops in between (openwrt
> which is missing the kernel module in this very build) I get a
> significant improvement of latency during the test - again iperf with
> 5 streams.
> Throughput seems a little higher as well 8Mbit vs 10 but that is not
> significant to me. The latency drop from >300ms to roughly 100 for an
> otherwise unchanged equipment looks very promising.

With a single flow it should be even better, with BBR.

Still, I kind of wish we'd shipped with a codel target of 10 instead of
20. And that openwrt wasn't arbitrarily reducing the wifi quantum to
300.

>
> PING 2a06:8187:fbab:1::9000:2(2a06:8187:fbab:1::9000:2) 56 data bytes
> 64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=1 ttl=62 time=106 ms
> 64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=2 ttl=62 time=47.5 ms
> 64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=3 ttl=62 time=32.8 ms
> 64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=4 ttl=62 time=37.1 ms
> 64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=5 ttl=62 time=137 ms
> 64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=6 ttl=62 time=121 ms
> 64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=7 ttl=62 time=117 ms
> 64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=8 ttl=62 time=113 ms
> 64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=9 ttl=62 time=117 ms
> 64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=10 ttl=62 time=98.8 ms
> 64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=11 ttl=62 time=112 ms
> 64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=12 ttl=62 time=100 ms
> 64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=13 ttl=62 time=109 ms
> 64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=14 ttl=62 time=106 ms
> 64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=15 ttl=62 time=44.7 ms
> 64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=16 ttl=62 time=30.9 ms
> 64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=17 ttl=62 time=41.1 ms
> 64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=18 ttl=62 time=33.2 ms
> 64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=19 ttl=62 time=31.8 ms
> 64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=20 ttl=62 time=30.5 ms
> 64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=21 ttl=62 time=31.9 ms
> 64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=22 ttl=62 time=33.6 ms
> 64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=23 ttl=62 time=31.0 ms
>
>
>
> viele Grüße
>
> Christof
>
>
>
> On Sun, Nov 04, 2018 at 06:54:58PM +0100, Christof Schulze wrote:
>>On Sat, Oct 13, 2018 at 05:23:35PM -0700, Dave Taht wrote:
>>>On Sat, Oct 13, 2018 at 9:26 AM Justin Kilpatrick <justin at altheamesh.com> wrote:
>>
>>
>>>>> > On Sat, Oct 13, 2018 at 12:01:54AM -0700, Dave Taht wrote:
>>>>> I get that bandwidth figure a lot for wireguard. I care about  latency
>>>>> far, far more under a full bidirectional load. Having got base wifi so
>>>>> much better, and the edge connections sqm-scripts massively better, I
>>>>> am wondering if wireguard got on the stick yet?
>>
>>>>> I wrote about this problem in an early version of wireguard here:
>>>>> http://blog.cerowrt.org/post/wireguard/
>>
>>>>> As of kernel 4.4 (?) ipsec does take advantage of the fq_codel hash.
>>>>> the before latency was 100+ms in the tunnel for voip, 2ms after.
>>
>>>>I can confirm that fq_codel works with Wireguard tunnels just fine. The latency added by a tunneled hop is around 1-2ms.
>
>>>that is not a possible result. The classic simplest bufferbloat test
>>>is start a ping, then do a big 60 sec upload or download from a server
>>>on the other side of that link. Over wifi that's a minimum resulting
>>>delay of 10ms, closer to 20, nowadays. about 2 over cake on ethernet.
>
>>>but: Over the wireguard tunnel I tested 2 years or so back, 150ms induced delay.
>
>>>from tcp on an openwrt router, it's probable that the pacing_rate bug
>>>still exists which means it's tcp can't flood the link - which is a
>>>good thing in this case but... users aren't on the routers. users
>>>don't just ping or download or upload once at a time.
>> This is what happens for icmp6 for me before during and after an
>> iperf with 5 concurrent streams over a connection using one wifi
>> mesh-hop and one wireguard-vpn hop
>>
>>Bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=212 ttl=62 time=38.0 ms
>>64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=213 ttl=62 time=30.8 ms
>>64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=214 ttl=62 time=31.5 ms
>>64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=215 ttl=62 time=37.2 ms
>>64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=216 ttl=62 time=31.5 ms
>>64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=217 ttl=62 time=32.7 ms
>>64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=218 ttl=62 time=33.0 ms
>>64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=219 ttl=62 time=32.1 ms
>>64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=220 ttl=62 time=31.8 ms
>>64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=221 ttl=62 time=53.2 ms
>>64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=222 ttl=62 time=82.8 ms
>>64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=223 ttl=62 time=57.5 ms
>>64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=224 ttl=62 time=32.2 ms
>>64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=225 ttl=62 time=29.8 ms
>>64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=226 ttl=62 time=247 ms
>>64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=227 ttl=62 time=222 ms
>>64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=228 ttl=62 time=245 ms
>>64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=229 ttl=62 time=325 ms
>>64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=230 ttl=62 time=252 ms
>>64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=231 ttl=62 time=262 ms
>>64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=232 ttl=62 time=261 ms
>>64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=233 ttl=62 time=342 ms
>>64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=234 ttl=62 time=386 ms
>>64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=235 ttl=62 time=422 ms
>>64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=236 ttl=62 time=178 ms
>>64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=237 ttl=62 time=29.9 ms
>>64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=238 ttl=62 time=32.3 ms
>>64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=239 ttl=62 time=34.8 ms
>>64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=240 ttl=62 time=31.0 ms
>>64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=241 ttl=62 time=39.2 ms
>>64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=242 ttl=62 time=40.2 ms
>>64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=243 ttl=62 time=54.3 ms
>>64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=244 ttl=62 time=73.5 ms
>>64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=245 ttl=62 time=58.6 ms
>>64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=246 ttl=62 time=59.0 ms
>>64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=247 ttl=62 time=228 ms
>>64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=248 ttl=62 time=108 ms
>>64 bytes from 2a06:8187:fbab:1::9000:2: icmp_seq=249 ttl=62 time=63.6 ms
>>
>
>>>>Here's some iperf data, but not as latency focused as you would probably like.
>
>>>nope. also udp fragmenting is iperf's default mode.
>
>>>>https://forum.altheamesh.com/t/althea-performance/44/3
>>
>>>>_______________________________________________
>>>>Babel-users mailing list
>>>>Babel-users at alioth-lists.debian.net
>>>>https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/babel-users