Artifact of Fuzz3: Entropy as a Third Oracle

Artifact of Fuzz3: Entropy as a Third Oracle (2026) Abstract. Abstract. Fuzz testing finds security issues and improves robustness; however, it has only two implicit test oracles: timeout and crash. Information theory gives a third: entropy, which is generic, low-cost and widely applicable. Fuzz3 is SBSE blackbox and programming language agnostic. Applied to 6 open source software, it found 4 bugs, 2 already fixed. GitHub: https://github.com/karineek/Fuzz3 Methodology Supported Mutator Operators. The mutators Fuzz3 currently supports are as follows: Mutator Description Text Numeric Code bit_flip Flips a random bit in the input V V V delete_line Removes a random line V V delete_char Removes a random character V V V duplicate_line Duplicates a random line V V duplicate_char Inserts a random character V V V crazy_indentation Random whitespace and punctuation edits V V insert_block_comment Inserts a synthetic C-style comment V none Returns input unchanged (baseline) V V V add_one Increments a random numeric token V add_one_mixed_tokens Increments a numeric substring in mixed input V sub_one Decrements a random numeric token V equilateral Forces three equal numeric values V isosceles Forces two equal numeric values V cmutation_assignment Mutates assignment expressions (GrayC's AST-based) V cmutation_conditional Mutates conditional expressions (GrayC's AST-based) V cmutation_duplicate_stmt Duplicates statements (GrayC's AST-based) V cmutation_jump Mutates control flow jumps (GrayC's AST-based) V cmutation_unary Mutates unary operators (GrayC's AST-based) V Bug Found We fuzzed with Fuzz3 the following SUTs: SUT Name Description Experimental Setup Git Repository / Link flaky_triangle Target described in the original TSE 1976 paper. See the Fuzz3 publication. https://doi.org/doi:10.1109/TSE.1976.233835 LLVM/clang Fuzzed to find C compiler bugs. LLVM 23.0.0git (64e75b1), Ubuntu 20.04, Python 3.10. https://github.com/llvm/llvm-project clang-format Fuzzed to find code formatting bugs. LLVM 23.0.0git (64e75b1), Ubuntu 20.04, Python 3.10. https://github.com/llvm/llvm-project Pytest with Cirq Fuzzed cirq-core/ test suite via pytest (no mutators). Pytest 9.0.2, Cirq 1.6.1 (9888067), Ubuntu 20, Python 3.11.11. https://github.com/pytest-dev/pytest and https://github.com/quantumlib/Cirq httpcore Python library for low-level HTTP requests. Fuzzed GET request command on Windows OS, Python 3.14. https://github.com/encode/httpcore We found several bugs using Fuzz3: Bug #1 – Crash in clang-format caused by a NUL character inside a comment We found a bug in clang-format. With `--dry-run` it crashes on an input file that contains a NUL byte (i.e. `^@`) inside a block comment: #if defined(AAA) && (BBB + 3) > 9 /*^@comment */ && !defined(CCC) int f(void) [ return 1; } #elif defined(DDD) || defined(EEE) int f(void) { return 2; } /* trailing */ #else int f(void) { return 3; } We reported the bug in clang‑23. Tracing it back showed it existed already in clang‑10. The report was acknowledged within one hour and fixed by the LLVM developers on the same day [1]. --- Bug #2 – Hang in clang-format caused by a non‑printable character in a comment `clang-format test.c` hangs on the following input because it contains a non‑printable ASCII escape character (hex `0x1B`, also shown as `^[`): int32_t f[][1]; // g^[]; void main() {} The escape character began an incomplete terminal‑control sequence. We did not file a bug report with the LLVM developers, as the issue seems more general OS matter (it also makes the `cat` utility hang). Redirecting the output to a file avoids the hang. --- Bug #3 – Failures in the CI pipeline of Cirq due to missing LaTeX dependencies We fuzzed the Cirq test suite with pytest. Because we used an empty list of mutators, the only expected source of output variation was the total runtime (in seconds). At the start of fuzzing, while collecting initial input‑output entropy, the measured output entropy was 2.585, larger than expected from runtime variation. The root cause turned out to be missing LaTeX libraries, which produced errors such as: cirq-core/cirq/contrib/quantikz/circuit_to_latex_render_test.py:41: AssertionError !!! pdflatex failed on run 1 (exit code 1) !!! ! LaTeX Error: File `standalone.cls' not found. ... E pylatex.errors.CompilerError: No LaTex compiler was found Consequences: Two tests in `cirq.contrib` failed instead of being skipped when external dependencies are absent. Additionally, the error messages were not printed in a deterministic order, inflating the initial output entropy. We reported the issue (including the logs) to the Cirq developers [1]. --- Bug #4 – Improper URL validation and wrong response codes in httpcore The bug in httpcore concerns malformed URLs being accepted as valid. An example (anonymised) is: ```http://www.xxxx/xxxx/\nhttp://www.xxxx/xxx/``` Instead of returning a 404 or raising an exception, the library returns a 301 redirect, and the redirect URL itself is malformed: ```http://www.xxxx/xxxx/http:/www.xxxx/xxxx/``` Similar malformed inputs also produced 301 responses with redirects that point to syntactically valid but incorrect domains. We reported this bug to the developers [1]. --- References 1. Anonymous, *2026* – internal bug‑report identifiers in this Zenodo record (see PDF files) ````