1. Make sure that your msdscript
executable always exits with code 0 or 1, where 0 means success and 1 means an error or test failure. From now on, that's a requirement for msdscript
.
If your parser or interpreter can raise an exception, then probably main
should be something like
int main(int argc, char **argv) {
try {
.....
return 0;
} catch (std::runtime_error exn) {
std::cerr << exn.what() << "\n";
return 1;
}
}
so that any exception is caught, its message is printed, and the exit code is 1.
2. Implement a test_msdscript
program for random testing. This program's implementation should be in your msdscript
repo, so make test_msdscript
should build test_msdscript
, while just make
or make msdscript
still builds msdscript
. The implementation of test_msdscript
should not use any of the msdscript
implementation, since it's job is to independently test the msdscript
implementation.
Your test_msdscript
program should work in two modes:
msdscript
program, and test_msdscript
should test that msdscript
implementation on randomly generated programs.msdscript
program, and test_msdscript
should try to find a program and mode (--interp
, --print
, or --pretty-print
) where the two programs disagree.Naturally, test_msdscript msdscript
for your msdscript
should not find any errors. Also, test_msdscript msdscript msdscript
should not find any differences between your msdscript
and itself. But test_msdscript
may uncover bugs in your msdscript
to be fixed. In class, you'll run test_msdscript
with your msdscript
and another student's msdscript
.
Here are some broken msdscript
implementations (as Mac Intel executables; you may need to use chmod a+x msdscript0
, etc., to make each file executable after downloading, and you may need to use xattr -d com.apple.quarantine msdscript0
, etc., to make each executable trusted):
The msdscript0
and msdscript1
implementations are so broken that you should be able to find inputs where they fail using just test_msdscript msdscript0
and test_msdscript msdscript1
. To find inputs where the others fail, you'll probably need to use, say, test_msdscript msdscript msdscript2
to compare against your msdscript
implementation.
You don't have to find inputs that produce incorrect results all of the above implementations, but your test_msdscript
program should be able to automatically find inputs that trigger incorrect results for most of them (comparing against your correct msdscript
implementation). Keep in mind that it's also possible that comparing against one of the broken implementations will expose a bug in your msdscript
, since most of the broken implementations above work on at least some programs.
As you are implementing test_msdscript
, remember to use good programming practices:
You are excused from making test cases for test_msdscript
itself. Testing a random generator can be tricky — but, obviously, it should not incorrectly blame a program that is working correctly.