Polyglot compatibility with Java 1.4

We evaluated the correctness of Polyglot by testing it against the Jacks test suite. Polyglot fails 207 test cases out of 4863 in the test suite. 40 of the failed test cases involve an ambiguity in the JLS that Polyglot chose to resolve differently from Jacks's expectation. 90 of the failed test cases either contradict the JLS, or test features of the compiler other than its language implementation. 15 of the failed test cases are outside the control of Polyglot.

Excluding such anomalous test cases, Polyglot passes 98.7% (4656/4718) of the Jacks tests. For comparison using the same criteria, javac (1.7.0) passes 99.2% of the Jacks tests.

We have classified the failed 207 test cases as follows.

Updated: 11/02/2015