Skip to main content

Simplifying Development and Deployment of High-Performance, Reliable Distributed Systems

Final Report Summary - PROPHET (Simplifying Development and Deployment of High-Performance, Reliable Distributed Systems)

It is crucial to have reliable networks, and this requirement does not change with the introduction of Software Defined Networking (SDN). Unfortunately, as the network programmability enhances and software plays a greater role in it, risks that buggy software may disrupt an entire network also increase. The centralized programming model, where a single controller program manages the network, seems to reduce the likelihood of bugs. However, the system is inherently distributed and asynchronous, with events happening at different switches and end hosts, and inevitable delays affecting communication with the controller.

This summary presents an overview of efficient, systematic techniques for testing the SDN software stack at both its highest and lowest layer produced by the PROPHET ERC project. Our goal is to: (1) decrease the chance of encountering bugs in the deployment of OpenFlow controller applications and switches, and (2) enable faster adoption of OpenFlow/SDN due to accelerated switch interoperability testing. Combined, our tools should increase the confidence in SDN as a whole.

Our NICE (No bugs In Controller Execution) tool tests unmodified controller programs by subjecting them to automatically generated carefully-crafted streams of packets under many possible event orderings. To use NICE, the programmer supplies the controller program, and the specification of a topology with switches and hosts. The programmer can instruct NICE to check for generic correctness properties such as no forwarding loops or no black holes, and optionally write additional, application-specific correctness properties (i.e. Python code snippets that make assertions about the global system state). By default, NICE systematically explores the space of possible system behaviors, and checks them against the desired correctness properties. The programmer can also configure the desired search strategy. In the end, NICE outputs property violations along with the traces to deterministically reproduce them. The programmer can also use NICE as a simulator to perform manually-driven, step-by-step system executions or random walks on system states. Our NICE prototype tests unmodified applications written in Python for the popular NOX platform. A release of NICE is publicly available at

SOFT: Testing the Interoperability of OpenFlow Switches

An aspect that is mostly going unnoticed is that OpenFlow switches also run software, which must behave correctly. In practice, a real OpenFlow deployment likely has switches from multiple vendors managed by one or more controllers. To ensure correct network operation, all switches must work properly. If failures start occurring in OpenFlow deployments, the hard-earned ability to innovate in the networking space will be severely hampered by mistrust. Towards achieving exhaustive testing, we proposed SOFT (Systematic OpenFlow Testing), an approach to interoperability testing that leverages the multiple, existing OpenFlow implementations and herein identifies potential interoperability problems by crosschecking their behaviors.

Monocle: Dynamic, Fine-Grained Data Plane Monitoring

Monocle allows network operators to simplify their network troubleshooting by providing automatic data plane correspondence monitoring. Monocle transparently operates as a proxy between an SDN controller and network switches, verifying that the network view configured by the controller (for example using OpenFlow) corresponds to the actual hardware behavior. To ensure that a rule is correctly functioning, Monocle injects a monitoring packet (also referred to as a probe) into a switch, and examines the switch behavior. During reconfiguration, Monocle closely monitors the updated rule(s) with millisecond precision, and provides a service to the controller which informs when the rule updates sent to the switch finished being installed in hardware. This information could be used by a network controller to enforce consistent updates. In steady-state, Monocle periodically checks all installed rules and reports rules which are misbehaving in the data plane.