[Babel-users] ECMP on endpoints [was: babeld slashes...]

[Toke Høiland-Jørgensen]

Daniel Gröber <dxld at darkboxed.org> writes:

> Hi Toke,
>
> On Sat, Apr 16, 2022 at 08:28:14PM +0200, Toke Høiland-Jørgensen wrote:
>> > Well I already have varying performance but right now it's sticky so
>> > per-flow varying is strictly an improvment :)
>> 
>> Are you sure? If it's everything that goes down the drain it's at least
>> obvious and straight-forward to debug...
>
> Yeah but I got tired of debugging yet another transient connectivity issue
> after like the third time anyway. Multipath tracerouters are a thing
> cf. paris traceroute etc. so I'm not really too worried about it.

Alright, fair enough.

>> > One of the the problem I'm trying to solve (using the RTT metric bit)
>> > is the underlay network path having high latency. One of my hosters
>> > had pesky problems with their path getting congested during prime time
>> > and the carrier AS responsible for the link (DTAG) refusing to do
>> > anything about it. However cost wise this is the cheapest host per
>> > unit of traffic so it's still useful to have around most of the time.
>> 
>> Given that this is IPv6-only am I right in guessing that this is all an
>> elaborate scheme to get around the fact that your ISP won't supply you
>> with v6 connectivity? :)
>
> Actually my ISP, AS8412, is pretty good with IPv6. It's the modalities
> around it and their inflexibility that is the problem. They offer (at
> reasonable prices) no static prefixes, only small /60 prefixes via
> DHCPv6-PD and no true dual-stack only v4 CGN ugh.
>
> Besides, running a multihomed network just seemed a fun and interesting
> thing to get up to :)

Heh, "for fun" is definitely a valid reason in my book!

>> For ECMP I guess you'd need some kind of threshold of when alternatives
>> are good enough to be added as alternate paths? Getting that right sounds
>> kinda fiddly; and will likely be specific to a particular (set of) paths,
>> so really hard to define good values for at the protocol level...
>
> I don't really see the problem honestly this seems an entirely local
> policy sort of concern if the routes aren't used for forwarding and
> not redistributed.
>
> I have been doing a bit of looking for literature about distributed
> multipath routing algorithms too and there certainly does seem to be
> quite some papers that would be applicable to babel. So we're not
> entirely in untrodden territory. I'll probably not have time for any
> of that though.


Sure, I'm not disputing you can probably get something that will work
for your usecase. I'm just being (maybe too?) pessimistic about whether
this would be generally useful (i.e., upstreamable) for babel/bird as a
whole. Happy to be proven wrong, of course :)

>> BTW, have you looked at source-specific routing at all? With this, you'd
>> assign each endpoint multiple addresses, each tied to a prefix specific
>> to a particular upstream; and then install source-specific default
>> routes, so the packets go to the right upstream. With this you can
>> handle multi-path (or route selection in general) entirely at the
>> application layer simply by picking different source addresses...
>
> Ah yes, I have. Ideed that could work but I would like the addresses to
> stay constant as connectivity comes/goes.

Yeah, the end-to-end solution to that would be MPTCP and/or QUIC. Which
needs support from the server side, so not really a full solution in
itself. One can dream, though!

>> Couldn't the edge routers themselves monitor this and simply retract/modify
>> their announcements if they notice they have connectivity issues?
>
> Yeah that's my thinking too. Doing this bit on the endpoint doesn't really
> make sense. The connectivity issues aren't just a boolean sort of thing
> though. It very much depends on the network path so this needs some
> automatic monitoring not just a cron job to probe some static IPs :)

Well, please allow me to plug another project I'm involved in: the
"passive ping" (pping) utility. This was originally written by Kathy
Nichols as a libpcap-based utility, but I have a PhD student who is
working on a BPF-based version specifically designed to be run in the
background processing all traffic:

https://github.com/xdp-project/bpf-examples/tree/master/pping

The utility will parse packet headers, match outgoing to incoming
packets and compute an RTT (using TCP timestamps and/or ICMP sequence
numbers). This way you can continuously monitor the latency on a link
without injecting any measurement traffic. We've yet to run any
comprehensive analysis using this, but the tool itself should work as-is
(famous last words) :)

-Toke