Everything that our group has done so far has been in Java, so I'll be doing a bunch of low-level JVM hacking using BCEL to instrument the programs that we're using for test cases. There's also an automated tool that was written a few years ago for 1.2, but it won't run under 1.4; fixing it may wind up on my "to do" list. Aside from a bunch of compiler tools, the main application that's been used as a guinea pig so far has been a Jabber client that a group of undergrads wrote last year, and my advisor wants me to find at least one more example.
What I need is a sizeable database of both successful and unsuccessful executions, which means that I need to find something that's popular, but still buggy enough to give me enough failure cases. (Actual crashes don't count, since you get a stack trace -- we're concerned with weird and subtle bugs, the sort that are hard to track down.) My first impulse was to use PCGen, since I have a captive audience for it in my D&D campaign, and the version that I need is still in beta, but we only use it once or twice a week, so I'm not sure that I'd get enough data from it. Ideally, I'd like to find something that a sizeable user community uses every day that's in Java, and that isn't of rock-solid quality. Can anyone think of a good example?