[Pkg-xen-devel] Bug#679533: Traffic forwarding issue between Xen domU/dom0 and ovs

[Hans van Kranenburg]

On 08/22/2012 06:09 PM, Hans van Kranenburg wrote:
>
> So, it happened again yesterday. Exactly at the moment when one of my
> collegues started a newly set up domU on one of our servers, a virtual
> network interface of another (!) unrelated domU on the same dom0 began
> failing. The affected system is one of a set of VRRP routers, which is
> now taken out of order, and hangs around in a broken state.
>[...]
> One of the three network interfaces inside the domU shows exactly the
> same behaviour I described before. There's no traffic possible from the
> outside/dom0 to the inside of the domU, except when there's traffic from
> the inside to the outside, suddenly old pings get ponged:

Yesterday we tried to do some more debugging.

The dom0 (say, dom0 A) where the domU with broken virtual network 
interface (say, domU X) runs has 19 domUs running on it.

Test 1: live migrate domU X to dom0 B
    --->X  :)
   A    B
Result 1: network interface domU X on dom0 B OK

Test 2: live migrate domU X back to dom0 A
   X<---   :(
   A    B
Result 2: network interface domU X on dom0 A broken

Test 3: xm shutdown domU X and xm create it again on dom0 A
   X       :(
   A    B
Result 3: network interface domU X on dom0 A broken

Conclusion: It seems there's some 'state' left on dom0 A so that when 
domU X is placed back on dom0 A, it gets broken again.

But, is it only related to domU X?

Test 4: shutdown domU X and migrate another domU to dom0 A in its place
   Y<---   :(
   A    B
Result 4: now network interface domU Y on dom0 A broken

Test 5: migrate domU X also back to dom0 A
   X<---   :)
   Y       :(
   A    B
Result 5: network interface domU X on dom0 A OK

Test 6: extended version of test 4/5: remove domU X(OK), Y(broken) and 
one extra domU Z(OK) from dom0 A. Now only 17 domUs run on dom0 A. Now, 
move/start them back in another order: the 18th domU, X is OK, the 19th, 
Z, shows up with broken networking.

It seems there's something like a 'place', (i.e. the 19th domU) on dom0 
that's broken, and whatever domU we move in it shows broken networking. 
I know everytime we start or migrate a domU to a dom0 vif devices etc. 
are newly created instead of re-used, but its still always the 19th domU 
that runs on this dom0 that invariably ends up with networking issues.

¯\(°_o)/¯

-- 
Hans van Kranenburg - System / Network Engineer
+31 (0)10 2760434 | hans.van.kranenburg at mendix.com | www.mendix.com