[Babel-users] Babel with VPP

[Pim van Pelt]

Hoi,

On Tue, Mar 12, 2024 at 3:03 AM Embedded <lists at optimcloud.com> wrote:


    On Monday 11 March 2024 07:26:07 PM (+07:00), Dave Taht wrote:

     > He also did a nice writeup of an inexpensive 32x100Gbit switch
     > recently, running... debian.
     >
     >
    https://www.linkedin.com/pulse/debian-mellanox-100g-switch-pim-van-pelt-3pivf/
    <https://www.linkedin.com/pulse/debian-mellanox-100g-switch-pim-van-pelt-3pivf/>

    Yupp great write up, though curious why he didnt just use sonic-vpp
    as it already exist

This is actually a really good question (although off-topic) so I 
thought I'd get to this answer separately.

There are two reasons why I would not use sonic-vpp on this Mellanox 
family of switches, one is functional and the other is architechtural.
1) Sonic VPP makes the VPP dataplane available for a subset of very well 
defined YANG models and APIs. As with merchant silicon, the VPP 
dataplane on x86_64/amd64 (and possibly arm64) would be able to do the 
fast routing and switching, much like the silicon would. However, VPP 
has an incredibly rich feature set, only very few things are supported 
in Sonic VPP. In fact, several key features I use are not available in 
Sonic, nor Sonic VPP (MPLS L2VPN, GENEVE, ERSPAN).
As such, moving away from the lower level offering will give me an 
appliance with predictable and (mostly) interchangeable semantics, but 
it will lose me a tonne of functionality.

2) The Mellanox SN2700 (and other siblings in this series) have a very 
modest Atom based CPU on the Linux side. The one I wrote about has a two 
core Celeron(R) CPU 1047UE @ 1.40GHz. It's tiny, and would not perform 
very well running VPP at all. Besides the CPU, VPP does not program the 
Spectrum ASIC at all -- it is not even aware of it. So the way to make 
this work, would be to 'trap' all the traffic from the ASIC on ingress, 
and forward it over the PCIe bus to these CPUs, and have VPP send them 
back over the PCIe bus to the ASIC for egress. I noticed that the 
Spectrum chip is connected with x4 PCIe v3.0 so the throughput would be 
terrible.

But despite that, it just means 'VPP is not a good fit'. Sonic itself, 
has a build specifically for Spectrum:
https://github.com/sonic-net/SONiC/blob/sonic_image_md_update/supported_devices_platforms.md

So one might just run 'Sonic-Mellanox' on them! For me: Debian, 
ifupdown2, and Bird2 are all I need. KISS, and I get to maintain these 
switches in mostly the same way as any other hypervisor or VM.

groet,
Pim

-- 
Pim van Pelt<pim at ipng.ch>
PBVP1-RIPE -https://ipng.ch/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/babel-users/attachments/20240315/3e753825/attachment-0001.htm>