CAFLnet: a network protocol fuzzing framework based on selection algorithm with enhanced contextual information

Abstract Network protocol fuzzing is a critical method for detecting vulnerabilities in network protocol programs. However, traditional selection algorithms used in network protocol fuzzing often fail to accurately select effective states and seeds. To address this limitation, this paper proposes a fuzzing framework called Contextual AFLnet (CAFLnet), which employs a selection algorithm that utilizes enhanced contextual information. This framework introduces key metrics, such as state in-degree, state out-degree, and trace-adjacent call count, to enhance contextual information. The selection algorithm is divided into two parts: (1) a state selection algorithm based on the linear upper confidence bound, which optimizes the balance between exploration and exploitation by utilizing enhanced contextual information, and (2) a tri-factor seed selection algorithm, designed to utilize contextual information such as seed labels, execution information, and session information to thoroughly and effectively evaluate seed value in the selection process. We evaluated our framework and AFLnet using eleven benchmark programs from ProFuzzBench and the real-world. The results demonstrate that our framework outperformed AFLnet by an average of 6.86% in terms of branch coverage, with a notable increase of 18.79% on PureFTPD. In addition, our framework slightly outperformed AFLnet in state discovery and exhibited superior performance in vulnerability detection, triggering known vulnerabilities earlier and more frequently and successfully exposing a previously unknown vulnerability.