Reliability in XDP is a state-budget problem

Replicating a frame onto disjoint paths is easy to describe and dangerous to leave underspecified. The receiver must know which copies represent the same logical frame, how long to remember that identity, how much reordering is acceptable, what happens at sequence wraparound, and when stale state may be reclaimed. Without those decisions, duplication moves loss probability while creating a new failure mode: valid traffic is discarded as a duplicate or duplicate traffic escapes into the application.

The useful abstraction is a recovery function keyed by a stable stream identity. For each stream, the data plane tracks a highest accepted sequence and a bounded bitmap of recently observed positions. A frame ahead of the window advances state. A frame inside the window is accepted only if its bit is clear. A frame behind the retained window is stale. This is not an unbounded replay cache; it is a deliberate state budget whose width encodes the maximum reordering and delay differential the system claims to tolerate.

XDP runs before the normal socket stack has paid most of its costs, which is exactly why the program has to prove memory safety with packet bounds checks the verifier can follow. A production parser does not cast the packet to a tower of C structs and hope. It advances explicit offsets, validates data_end before every dereference, handles optional VLAN layers with bounded unrolling, and rejects unsupported layouts early.

The program also separates policy from mechanism. The hot path should not parse a dynamic configuration language or chase a graph of pointers. User space resolves policy into compact map entries: flow key, action, egress set, sequence mode, and recovery-window parameters. The XDP program performs the smallest bounded lookup and update needed to enforce that decision. If the verifier cannot establish the bound, the architecture is asking kernel space to do too much.

1if ((void *)(eth + 1) > data_end)2    return XDP_ABORTED;3 4flow_key = parse_bounded_l2_key(data, data_end);5rule = bpf_map_lookup_elem(&flow_rules, &flow_key);6if (!rule)7    return XDP_PASS;8 9seq = read_sequence(data, data_end, rule);10state = bpf_map_lookup_elem(&recovery_state, &flow_key);11if (!state)12    return initialize_and_accept(flow_key, seq);13 14if (!window_accept(state, seq)) {15    count_drop(flow_key, DROP_DUPLICATE_OR_STALE);16    return XDP_DROP;17}18 19return XDP_PASS;

A recovery window is mutable shared state. The map type decides memory placement, eviction, lookup cost, and contention behavior. A global hash map is simple but can turn a hot stream into a cacheline fight across receive queues. Per-CPU maps remove write contention but make one logical stream visible as several local histories unless steering guarantees affinity. LRU maps bound memory but can evict active state under pressure and silently reset the replay boundary.

There is no universally correct map. The choice follows RSS steering, stream cardinality, failure tolerance, and whether the sequence state must remain coherent across CPUs. The control plane monitors allocation failures and eviction pressure, because a reliability mechanism that loses state without reporting it is worse than an explicit degraded mode. State lifetime must also follow a monotonic clock and a bounded inactivity policy so dead streams cannot pin kernel memory forever.

Frame replication and elimination addresses path or link faults by accepting the first valid copy and suppressing later replicas. It does not create synchronized time, reserve an egress slot, or preempt a large best-effort frame. IEEE 802.1AS supplies time synchronization, 802.1Qbv supplies a time-aware transmission schedule, and 802.1Qbu reduces blocking through frame preemption. Treating any one of them as the entire real-time story produces a system that is correct only in diagrams.

The implementation boundary must preserve those distinctions. The XDP path can classify and replicate or eliminate frames, but schedule conformance and egress timing may live elsewhere in the Linux traffic-control and driver stack. Observability has to correlate them: sequence duplicate rate, recovery-window movement, redirect errors, gate state, clock quality, queue occupancy, and late traffic. A packet dropped by the recovery function is different from a packet that missed a transmission window, even if both appear as application loss.

• 802.1CB: replicate and eliminate redundant copies for reliability.
• 802.1AS: establish a shared time base and expose clock-quality failure.
• 802.1Qbv: schedule traffic classes against that time base.
• 802.1Qbu: reduce blocking by allowing eligible high-priority traffic to preempt lower-priority transmission.
• Linux/XDP control plane: bind stream policy, kernel state, interface topology, and telemetry into one inspectable operating model.

Printing from the data plane is not observability. The hot path should emit bounded counters and compact sampled events, each carrying a reason code the control plane can resolve. Required signals include accepted-first-copy, duplicate, stale-window, parse-reject, missing-rule, state-allocation-failure, redirect-failure, and sequence-reset. Per-CPU counters can keep writes local while user space aggregates them at a cadence that does not turn monitoring into the bottleneck.

The test matrix is sequence-heavy: in-order copies, replica inversion, maximum tolerated reordering, one step beyond the window, wraparound, state expiry, CPU migration, map pressure, malformed headers, and control-plane rule replacement while traffic is active. Performance evidence is also bounded by truth: report the kernel, NIC, queue topology, packet size, CPU placement, map type, and traffic shape or do not publish a headline rate. A fast-path number without its execution environment is not evidence.