"If that doesn't seem doable you might be taking on a task that doesn't have a good set of requirements." Or it might have perfectly good requirements which are very hard to write automated tests for.
Consider (for instance) a program to translate ABC music format to proper sheet music. It's easy to say the basic requirement: "The program has to accurately translate ABC format to legible and easy to read sheet music." But even a start at automating a test for that would require converting a graphical representation of sheet music back to the basic notes, and that problem is at least an order of magnitude harder than writing the original program, without factoring in the "easy to read" bit at all. (PS This is a real issue for me, if someone knows of a decent open source sheet music to MIDI/ABC convertor I'd love to hear about it.)
The mistake here is that what you're describing is a functional test, not a unit test.
A unit test for a piece of code like this might be "Given that the time signature for the music is 3:4, the software puts 3:4 in the appropriate place on each line of output".
You then might write a variety of cases testing that it deals correctly with (say) a changing time signature at some point in the piece.
The upside of this is when you try and fix another bug which has a knock on effect on this bit of code, lots of your tests are going to fail- immediately identifying where the problem is (or at least letting you know there is one!)
> A unit test for a piece of code like this might be "Given that the time signature for the music is 3:4, the software puts 3:4 in the appropriate place on each line of output".
Don't you still have the same problem colomon described here, though? Testing your stated condition "the software puts 3:4 in the appropriate place on each line of output" still implies some form of image recognition on the graphical output.
But okay, is it really so much easier to write a test which just tests changing the time signature? It's still going to require doing OCR. And if I'm really testing it, I've got to make sure the time signature change comes at the correct point, which requires OCR on the notes around it. Also the bar lines (to makes sure the time signature change is reflected in the notes per bar and not just by writing a new time signature out and otherwise ignoring it).
Now, it's perfectly reasonable to have unit tests that the code correctly recognizes time signature changes in ABC format. (And it looks like I forgot to write them: I see inline key changes in the tests but not inline time signature changes.) But that's only testing half of the problem; and it's the easier half by far.
Actually, I've written tests for half of that problem: Parsing ABC text into an internal format. That is a very clear problem and one just needs a representative set of input files (I now have around 500 of them -- the tests still run in less than a minute. It's true that I haven't been able to figure out a useful way to have an automated test for the drawing part. Here's my project: http://code.google.com/p/abcjs/
Consider (for instance) a program to translate ABC music format to proper sheet music. It's easy to say the basic requirement: "The program has to accurately translate ABC format to legible and easy to read sheet music." But even a start at automating a test for that would require converting a graphical representation of sheet music back to the basic notes, and that problem is at least an order of magnitude harder than writing the original program, without factoring in the "easy to read" bit at all. (PS This is a real issue for me, if someone knows of a decent open source sheet music to MIDI/ABC convertor I'd love to hear about it.)