<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<div dir="ltr" class="gmail_attr">Hoi,<br>
<br>
On Tue, Mar 12, 2024 at 3:03 AM Embedded
<a class="moz-txt-link-rfc2396E" href="mailto:lists@optimcloud.com"><lists@optimcloud.com></a> wrote:<br>
</div>
<blockquote class="gmail_quote"
style="margin: 0px 0px 0px 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;"><br>
On Monday 11 March 2024 07:26:07 PM (+07:00), Dave Taht wrote:<br>
<br>
> He also did a nice writeup of an inexpensive 32x100Gbit
switch<br>
> recently, running... debian.<br>
> <br>
> <a
href="https://www.linkedin.com/pulse/debian-mellanox-100g-switch-pim-van-pelt-3pivf/"
rel="noreferrer" target="_blank">https://www.linkedin.com/pulse<wbr>/debian-mellanox-100g-switch-<wbr>pim-van-pelt-3pivf/</a><br>
<br>
Yupp great write up, though curious why he didnt just use
sonic-vpp as it already exist<br>
</blockquote>
This is actually a really good question (although off-topic) so I
thought I'd get to this answer separately.<br>
<br>
There are two reasons why I would not use sonic-vpp on this Mellanox
family of switches, one is functional and the other is
architechtural.<br>
1) Sonic VPP makes the VPP dataplane available for a subset of very
well defined YANG models and APIs. As with merchant silicon, the VPP
dataplane on x86_64/amd64 (and possibly arm64) would be able to do
the fast routing and switching, much like the silicon would.
However, VPP has an incredibly rich feature set, only very few
things are supported in Sonic VPP. In fact, several key features I
use are not available in Sonic, nor Sonic VPP (MPLS L2VPN, GENEVE,
ERSPAN). <br>
As such, moving away from the lower level offering will give me an
appliance with predictable and (mostly) interchangeable semantics,
but it will lose me a tonne of functionality.<br>
<br>
2) The Mellanox SN2700 (and other siblings in this series) have a
very modest Atom based CPU on the Linux side. The one I wrote about
has a two core Celeron(R) CPU 1047UE @ 1.40GHz. It's tiny, and would
not perform very well running VPP at all. Besides the CPU, VPP does
not program the Spectrum ASIC at all -- it is not even aware of it.
So the way to make this work, would be to 'trap' all the traffic
from the ASIC on ingress, and forward it over the PCIe bus to these
CPUs, and have VPP send them back over the PCIe bus to the ASIC for
egress. I noticed that the Spectrum chip is connected with x4 PCIe
v3.0 so the throughput would be terrible.<br>
<br>
But despite that, it just means 'VPP is not a good fit'. Sonic
itself, has a build specifically for Spectrum:<br>
<a class="moz-txt-link-freetext" href="https://github.com/sonic-net/SONiC/blob/sonic_image_md_update/supported_devices_platforms.md">https://github.com/sonic-net/SONiC/blob/sonic_image_md_update/supported_devices_platforms.md</a><br>
<br>
So one might just run 'Sonic-Mellanox' on them! For me: Debian,
ifupdown2, and Bird2 are all I need. KISS, and I get to maintain
these switches in mostly the same way as any other hypervisor or VM.
<br>
<br>
groet,<br>
Pim<br>
<br>
<pre class="moz-signature" cols="72">--
Pim van Pelt <a class="moz-txt-link-rfc2396E" href="mailto:pim@ipng.ch"><pim@ipng.ch></a>
PBVP1-RIPE - <a class="moz-txt-link-freetext" href="https://ipng.ch/">https://ipng.ch/</a></pre>
</body>
</html>