[Babel-users] paralysis

[Dave Taht]

Christof Schulze <christof.schulze at gmx.net> writes:

>>(why is there a 20 interface limit in kernel_netlink.c?), but
> Why indeed? I am hoping to run babeld with >150 interfaces soonish.
> Of course I can adjust the define but I'd rather have this limitation
> understood than just fiddling with it.
>
>>Then i went around looking at the other 58 branches, saw that mattieu
>>had done a bier mockup on top of the dev branch, didn't see many major
>>mods besides (where is the dtls stuff?) what I'd mentioned...
>>
>>Then I went to go fiddle with the timerwheel idea which wants a clean
>>64bit type and that meant ripping out timeval in babel and switching
>>to that... which would be a mild speedup overall...
>>
>>Total paralysis ensued. Anybody else have this problem? :/ Is there a
>>path forward? Were I to regain ambition it would be to
>>get hmac working, then try to bring the datum branch forward.
>>
>>The sole positive note is that I found a better looking netlink
>>library for go than what existed 2+ years back... which is where I
>>think the golang version of babel stalled out
>>(https://github.com/sh3rp/gabel) ... except that i can't find that lib
>>again this morning, and it's not the first hits on google.
>>
>>It's the time of year where this song keeps popping into my head.
>>
>>https://www.youtube.com/watch?v=W5_8U4j51lI
> Well I think it is time to consolidate all the different patchsets
> again. Frankly I have lost it somewhere between trees, skiplists and
> timerwheels in your many and greatly appreciated attempts to get babel
> to scale better.

Hahah. I really enjoyed learning about skiplists, though my coin-flip
would have used rdtsc rather than rand(). It's been so long since I had
any fun in userspace!

Still, I think the net sum of all that flailing about was that hashing
conventionally was still the basic win.

Anyway, after giving up, and sending this email, zen-like, I managed to
bring forward the dev "datum" branch to head, which is easier to think
about, and in. I have a couple little cleanups to make before pushing that
to babel-hacking and no idea what juliusz thinks of "datum".

I started layering the hmac thing on top of that... I'll take your patch
for the source selection also?

As for speedups... I really have to say that 97% of my ideas tend not to
pay off. I'd tried ~2 years ago to speed up the wrong things in
babel. Juliusz kept telling me about simple data structures, I was
buried in optimizing memcmp....

Anyway, going forward...

I'd like to work the datum branch towards having one true (and hashed)
picture of all possible routes and then have various other data
structures just flinging pointers around. Then... trying to do stuff in
units of work small enough (always) to make sure hello/ihu went
out. Needn't be a wheel. Somewhere in here... finish turning the atomic
branch into code.

Babel head, as it stands, is quite solid enough for 2-4k routes in my
production environment, and I don't plan to exceed a few hundred. I
haven't come up with a way of comprehensively evaluating the uthash'd
resend routines, but they feel like they are helping... and really the
only time I see that kind of churn is during power failures.

I still haven't deployed the last 6 routers (they're living in my lab).

I'm loving unicast. 

I'd like to *know* what the overhead of auth is, because I would rather
like to use it in my deployment. Some of these hash routines eat 16
intel x86 cycles *per byte* or more (and those are usually on big
things, not packets)

But it sure is fun to blow things up with that abstract goal of 64k
routes, and watch simple systems create chaos.

I haven't had this much time in userspace in about a decade.

Happy thanksgiving from the USA!

>
> Christof