Fuzzing a C Extension with Atheris and ASan

Pure Python code cannot corrupt memory. A C extension can, and when it does, the symptoms are rarely a clean crash at the faulty line. A read one byte past a buffer returns whatever happens to be there; a write past the end overwrites a neighbouring object that fails, much later, in unrelated code; a use-after-free works perfectly until the allocator reuses the block. Unit tests pass. Production occasionally segfaults somewhere else.

Coverage-guided fuzzing with Atheris finds the inputs that reach those paths, and AddressSanitizer makes the errors visible the instant they happen. Together they turn "occasional segfault in production" into a report naming the exact line, the size of the overflow, and where the memory was allocated and freed. The setup takes more steps than fuzzing pure Python — the extension must be rebuilt with instrumentation, and the sanitizer runtime must be loaded into an interpreter that was not built with it — but each step is mechanical.

Prerequisites

• Linux, clang >= 16, atheris >= 2.3, Python 3.11 or later.
• An extension built from source with setuptools or scikit-build.
• The basics from Writing your first Atheris fuzz target.

Solution

# 1. Build the extension with ASan and libFuzzer coverage instrumentation.
export CC=clang CXX=clang++
export CFLAGS="-fsanitize=address,fuzzer-no-link -g -O1 -fno-omit-frame-pointer"
export CXXFLAGS="$CFLAGS"
export LDSHARED="clang -shared"
pip install --no-build-isolation --force-reinstall -e .

# 2. Locate the ASan-enabled libFuzzer runtime shipped with Atheris.
ASAN_LIB=$(python -c "import atheris; print(atheris.path())")/asan_with_fuzzer.so

# fuzz_decode.py
import sys
import atheris

with atheris.instrument_imports():
    import fastcodec                      # the C extension under test

def TestOneInput(data: bytes) -> None:
    fdp = atheris.FuzzedDataProvider(data)
    mode = fdp.ConsumeIntInRange(0, 2)
    payload = fdp.ConsumeBytes(fdp.remaining_bytes())
    try:
        if mode == 0:
            fastcodec.decode(payload)
        elif mode == 1:
            fastcodec.decode_stream(payload, chunk=fdp.ConsumeIntInRange(1, 64))
        else:
            fastcodec.validate(payload)
    except fastcodec.DecodeError:
        pass                              # documented rejection is fine

atheris.Setup(sys.argv, TestOneInput)
atheris.Fuzz()

# 3. Run with the runtime preloaded and leak detection off.
LD_PRELOAD="$ASAN_LIB" ASAN_OPTIONS=detect_leaks=0,allocator_may_return_null=1 \
  python fuzz_decode.py corpus/ -max_total_time=600

Why this works

-fsanitize=fuzzer-no-link inserts libFuzzer's coverage callbacks into every branch of the extension without linking a main, so Atheris receives edge coverage from C as well as from the instrumented Python. That matters because the interesting paths of a decoder — length fields, escape sequences, chunk boundaries — are in C, and without C coverage the fuzzer would be mutating blindly as far as those branches are concerned.

-fsanitize=address adds shadow memory around every allocation and checks every load and store. An access one byte outside a heap buffer, a read from freed memory, a double free — each triggers an immediate abort with a report showing the faulting stack, the allocation stack and, for use-after-free, the free stack. Atheris treats the abort as a crash, writes the input that caused it to crash-<hash>, and stops.

The preload step exists because the interpreter itself is uninstrumented. The ASan runtime must be initialised before any instrumented code runs, and it must intercept malloc and free for the whole process; loading it as the first shared library achieves both. Atheris ships a runtime built for exactly this purpose.

Reading an ASan report

A typical first finding looks like this, trimmed:

==4121==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x6020000000b1
READ of size 1 at 0x6020000000b1 thread T0
    #0 in read_varint fastcodec/varint.c:41
    #1 in decode_field fastcodec/decode.c:118
    #2 in fastcodec_decode fastcodec/module.c:72
0x6020000000b1 is located 0 bytes after 1-byte region [0x6020000000b0,0x6020000000b1)
allocated by thread T0 here:
    #0 in malloc
    #1 in PyBytes_FromStringAndSize

The first line names the error class. The access stack shows where: read_varint at line 41 read one byte beyond the input. "0 bytes after 1-byte region" says the buffer was one byte long and the read was immediately past it — a varint decoder that did not check for input ending mid-value. The allocation stack confirms the buffer was the Python bytes object passed in. The fix is a bounds check in the loop; the crash input becomes a regression test.

Seeding the corpus and giving the fuzzer a dictionary

A fuzzer started from an empty corpus spends its first minutes discovering the most basic structure of the format — that a valid message starts with a particular magic byte, that a length prefix must roughly match the payload. For a binary codec that can waste most of a short CI budget. Two cheap inputs shorten that phase dramatically.

The first is a seed corpus: a directory of real, valid inputs. Take a few dozen payloads from the codec's own test fixtures, or encode a handful of representative values with the Python-level API, and write each to its own file in corpus/. libFuzzer starts by running every seed, records the coverage each one reaches, and mutates from there. Valid seeds put the fuzzer past the parser's front door on the first iteration, so mutation time goes into the interesting branches: truncation, oversized lengths, nested structures, unusual tags.

The second is a dictionary: a text file of tokens the format uses, passed with -dict=codec.dict. Each line is a quoted byte string — magic numbers, tag values, keywords. libFuzzer inserts dictionary entries during mutation, which lets it produce a multi-byte magic value in one step instead of guessing it byte by byte. For text-like formats, the keywords and delimiters belong in the dictionary; for binary formats, the tag bytes and boundary lengths such as "\xff\xff\xff\x7f" do.

After a long run, minimise the corpus with -merge=1 into a fresh directory. Merging keeps the smallest set of inputs that preserves total coverage, which makes the corpus quick to replay at the start of every future run and small enough to commit or cache as a CI artefact.

From crash file to fixed release

A crash is only half a result. The workflow that follows it decides whether the same bug can come back.

First, reproduce outside the fuzzing loop. Passing the crash file as the only argument — python fuzz_decode.py crash-1a2b3c with the same LD_PRELOAD — runs just that input and prints the same report. That confirms the crash is deterministic and gives a fast loop for testing the fix.

Second, write a plain pytest regression test that feeds the exact bytes to the extension. Without ASan, the test may pass even before the fix, because the overflow does not necessarily crash an uninstrumented build. That is why the test belongs in a job that also runs the suite against the sanitizer build — a small matrix entry that builds once with ASan and runs the extension's tests with the runtime preloaded. The regular job still runs the test, which documents the input; the sanitizer job enforces it.

Third, add the crash input to the seed corpus. Future fuzzing runs then start from it, and mutations of it explore the neighbourhood of the bug, which is where related bugs tend to cluster — a missing bounds check in one field decoder often has siblings in the others.

Finally, decide whether the bug is a security issue. An out-of-bounds read in a decoder that parses untrusted network input is, and it deserves an advisory and a patched release rather than a quiet fix in the next version.

Edge cases and failure modes

• ASan runtime does not come first in initial library list. The runtime was not preloaded, or another library was preloaded before it. Put the ASan library first in LD_PRELOAD.
• Flood of leak reports at exit. CPython does not free everything at shutdown. Set detect_leaks=0 for fuzzing runs.
• Extension built without coverage. If only ASan flags were used, the fuzzer finds crashes only by luck. Confirm fuzzer-no-link is present in the compile commands printed by the build.
• Crashes inside CPython. A report whose top frames are in the interpreter usually means the extension corrupted an object earlier — a reference count or a buffer it did not own. Look at the allocation and free stacks, not just the access.
• Optimised-away bugs. High optimisation can remove the faulting read. Build fuzzing binaries at -O1 with frame pointers, as above.

Frequently Asked Questions

Why do I need AddressSanitizer to fuzz a C extension? Without it, many memory errors — reading one byte past a buffer, using freed memory — do not crash. They corrupt memory silently and the fuzzer never notices. ASan turns each of those errors into an immediate, precise report at the faulting instruction.

Why must the ASan runtime be preloaded? The Python interpreter itself is not built with ASan, so the sanitizer runtime is not loaded when the extension is imported. Preloading the runtime library with LD_PRELOAD makes it available before any instrumented code runs.

Should I disable ASan leak detection? Usually yes, with detect_leaks=0. CPython intentionally keeps objects alive until exit, so leak detection reports many false positives. Use a dedicated leak-hunting run with suppressions if leaks are the concern.

Related

• Coverage-Guided Fuzzing with Atheris — how Atheris works.
• Writing Your First Atheris Fuzz Target — target structure and corpora.
• Running Fuzz Targets in CI with Time Budgets — running this setup on a schedule.
• Structure-Aware Fuzzing with Atheris and Protobuf — valid-by-construction inputs.

← Back to Coverage-Guided Fuzzing with Atheris