CS 5963/6963: Applied Software Security Testing


Final Project (collected via Canvas)

Instructions: Using your skills from Labs 1–3, team-up in groups of two to four students to hunt bugs in a real-world software of your choice! Upon choosing a target, your team must figure out how to (1) harness it, (2) fuzz it, and (3) triage any discovered bugs. You may select any target you like (e.g., popular APIs, video games, kernels), provided it has not been fuzzed before—or has demonstrably not been fuzzed effectively. Failure to do your due dilligence will cause your team to lose points—or worse yet—have to find a different target!

Halfway through the semester, your team will present a 5-minute project lightning proposal to the class outlining your chosen target, your proposed approach, and the significance of your work. At the semester's end, you will prepare and deliver a final in-class presentation along with submitting a final project report outlining your ultimate approach, findings, and any discovered bugs.

Heilmeier's Catechism will serve as the high-level rubric for your proposal, presentation, and report—so be ready to answer why your project idea matters! But most importantly, get creative and have fun. Besides Heilmeier's Catechism, other important criterion include:

  • Responsible Disclosure: Discovered bugs must be disclosed to developers! Include any bug report links (e.g., GitHub issues) in your final report. If bug reports are not public, you may include screenshots of your correspondence (e.g., emails) with developers.


Project signup sheet is available: here (accessible only via your @gcloud.utah.edu account).

Working Solo: Solo projects are permitted only in special circumstances and require prior instructor approval. Consult the Schedule for the solo project approval deadline and meet with the instructor ahead of time. Be prepared to justify why your project is doable by you alone.

Lab Exercises (collected via Canvas)

Instructions: There will be three introductory fuzzing exercises that will count for 45% of your course grade (15% each).

Unless otherwise indicated, you must work solo. You may consult general reference material, but you may not collaborate with other students. The material you turn in must be entirely your own work, and you are bound by the Student Code.

Assignment Deadline (by 11:59PM)
Wednesday, January 29
Overview: In preparation for the semester project, this lab will familiarize you with AFL++ (the world's most popular and extensible fuzzing platform). You will test several of AFL's features in fuzzing an application of your choice.

Learning Objectives:

  • Compiling a C/C++ program.
  • Inserting AFL++’s instrumentation.
  • Initiating fuzzing with AFL++.
  • Interpreting AFL++’s results.


Your task: Select three of AFL++'s user-configurable features and evaluate their impacts on fuzzing:

  • What led you to explore these fundamental features and why?
  • How do these features impact speed, coverage, and crash discovery?
  • Do certain features work better in tandem, or individually?
  • Do these features perform as you expected, or unexpectedly?
  • As a target application, you are welcome to use the FuzzGoat fuzzing benchmark, or select a program from any other publicly-available fuzzing benchmark corpora (e.g., FoRTE-FuzzBench, HexHive's Magma).


Other Notes:

  • Information on AFL++'s available features can be found in its documentation.
  • Linux is recommended. You are welcome to use the Lubuntu VM from CS 4440.
  • For issues troubleshooting AFL++, you can ask for help on the Course Piazza, or reach its authors via GitHub or the #aflplusplus-issues-questions channel in the Awesome Fuzzing Discord. It is recommended that you start early.


Recommended Readings:


What to Submit:

Submit a 1–3 page report detailing your experimental findings. There are no "right" or "wrong" answers—your work will be assessed by your overall effort. You have full creative liberty—feel free to use images, tables, etc.

Wednesday, February 05
Overview: In this lab, you will leverage AddressSanitizer (ASAN)—today's state-of-the-art memory error detection tool—to perform post-fuzzing crash triage for crashing test cases you found during your Lab 1 fuzzing campaign.

Learning Objectives:

  • Compiling a C/C++ program with ASAN.
  • Generating and deduplicating ASAN reports.
  • Interpreting and triaging fuzzer-found crashes.


Your task: Replay the fuzzer-found crashing test cases you found during Lab 1 on an ASAN-instrumented version of your target program, and collect and analyze their resulting crash report information:

  • What sorts of bugs did your fuzzer find?
  • Are you seeing lots of "unique" crashes map to only a few bugs?
  • Or lots of bugs with a few crashes each?
  • What issues do you see, and what insights do you have?


Other Notes:

  • AFL's fuzzer-found crashes are stored in sub-directory within its output folder. Locate this before proceeding!
  • See LiveOverflow's blog and the Readings below for tips on using ASAN for post-processing fuzzer-found crashes.
  • Instead of re-running test cases by hand one-by-one, you can save lots of time by scripting (e.g., with Bash, Python).
  • Didn't find any crashes in Lab 1? Perform some quick fuzzing on the FuzzGoat benchmark (it has lots of crashes).
  • For issues troubleshooting ASAN, you can ask for help on the Course Piazza, or on the #crash-analysis channel in the Awesome Fuzzing Discord. It is recommended that you start early.


Recommended Readings:


What to Submit:

Submit a 1–3 page report detailing your experimental findings. There are no "right" or "wrong" answers—your work will be assessed by your overall effort. You have full creative liberty—feel free to use images, tables, etc.

Monday, February 24
Overview: In this lab, you will learn the art of writing fuzzing harnesses for a real-world software API (libArchive).

Learning Objectives:

  • Writing and testing API fuzzing harnesses.
  • Compiling and linking fuzzing harnesses.
  • Understanding harnessed API vs. application fuzzing.
  • Analyzing harness correctness from fuzzer results.


Your task: Develop a correct (i.e., upholds the API's intended usage) and AFL-compatible (i.e., accepts file inputs via AFL's @@ command-line directive) fuzzing harness for the libArchive archive-processing library API:

  • Which libArchive API functions did choose to harness, and why?
  • Does API code coverage steadily increase? What about crashes?
  • Is your harness correctly calling the API? How did you assess this?
  • Did you make refinements to improve your harness? What happened?


Other Notes:

  • Work together! You may team-up in groups of up to two students, or work solo.
  • Review libArchive's documentation to understand its API.
  • Don't use ChatGPT as a crutch—harnessing is a critical skill for your final project!
  • You are allowed to get inspired from libArchive's libFuzzer harness.
  • Research how others are using libArchive's API outside of fuzzing.
  • You can ask for help on the CS 5963/6963 Piazza. It is recommended you start early.


Recommended Readings:


What to Submit:

Submit a 1–3 page report detailing your experimental findings (including your harness code). We'll especially scrutinize how you "prove" the correctness of your harnesses—so be sure to thoroughly show that it works correctly. As usual, you have full creative liberty—feel free to use images, tables, etc. Be sure to append your harness code as the final page of your report!

Research Paper Presentations (collected via Canvas)

Instructions: Select one paper from the course Schedule to present to the class in a 10–15 minute presentation, totaling 10% of your course grade. You are free to borrow visual aides from existing slide decks such as the authors' own presentation, but the overarching presentation must be in your own words. Following each presentation will be a 5–10 minute audience discussion.

Paper signup sheet is available: here (accessible only via your @gcloud.utah.edu account; requests from other emails will be denied).

  • CS 5963 students must team-up and present in pairs of two.
  • CS 6963 students must deliver their presentation solo.


Presenters: your responsibility is to teach us the paper:

  • Prepare a 10-15 minute oral presentation on your own with an accompanying slide deck.
  • Outline the paper's context, main contributions, key assumptions, results, and pros/cons.
  • Do your own research on the field—you are expected to be "the expert" on your paper!
  • Minimize background information to only what's new—don't re-teach us what fuzzing is!
  • Following your presentation, submit your slides in PDF format to Canvas by 11:59PM that day.


Audience: you are not required to read the paper, but you must participate in the class discussion following each paper presentation.

Participation (collected via PollEverywhere)

Attendance: Regular attendance in this class is required as the lectures and discussions are designed to prepare you for success in the final course project. To monitor attendance, we'll issue attendance codes in-class via PollEverywhere for five randomly-chosen lectures throughout the semester, totaling 10% of your overall course grade.

We'll drop one absence at zero penalty. We'll track this automatically—you don't need to notify us about using any of your absences.