All projects

Debugging a Netfilter mark collision: Netbird, OVH OKS and kube-proxy

A critical connectivity incident traced to overlapping Netfilter mark bits, fixed with a DaemonSet and reported upstream.

Company
ID6TM
Period
2026
Stack
  • Netbird
  • WireGuard
  • nftables
  • iptables
  • kube-proxy
  • Kubernetes

Context

I redesigned the network and security architecture around zero-trust connectivity with Netbird (WireGuard): access between clusters and machines, ACLs for ingress and egress, and routing.

Problem

A critical connectivity incident. After upgrading Netbird (v0.30+), NodePort access stopped working on our OKS clusters, and the symptoms did not point at any single component.

Diagnosis

Low-level analysis with iptables/nftables and nft monitor trace showed that Netbird, OVH OKS and kube-proxy were using overlapping Netfilter mark bits. The collision broke the MASQUERADE step, and with it the traffic.

Fix

  • A workaround validated on a live cluster, then industrialised as a DaemonSet that patches the rules on every node.
  • A documented issue opened with Netbird upstream, so the root cause can be fixed at the source.

Why I keep this one

It is the kind of problem where nobody owns the bug: three well-behaved components interacting badly. The answer was a method (trace the packet, find the rule, prove it), not a guess.