Geoffrey Hoffman

March 13, 1998

MEng. Progress Report #3

The Beautify project is coming along fairly well. A lot of ground has been made in the last week or two, and the results seem to be pretty good. The first phase is pretty much complete: the formatter itself. It does a pretty good job of formatting code. There are a few minor cases that are not covered, but it is a matter of time to fix those. The next phase will be to implement the formatter into the Codewarrior development environment. This is a couple of tasks. The first thing is to figure out how to have a C/C++ program invoke a Java program, as Codewarrior and the plug-in APIis written in C++. The next task will be to find out how to get Codewarrior to invoke this application from the development environment. Another task would be to create a settings panel, and integrate that into the Codewarrior environment, which may not be simple.

So far:

There have been a number of changes to the Beautify code since the last update. Probably the most noteworthy is changing the temporary string that stores the code into a StringBuffer. This has one immediate benefit, as well as an additional unexpected benefit. The biggest different is that since the StringBuffers are mutable, and the Strings are not, it significantly faster. A StringBuffer can be modified, while a String can not, and String concatenation is actually the creation of a new String. Since Beautify needs to concatenate about two times for each token, it is a large increase in speed. Some tests show that it can Beautify a 4400 line code chunk in only a few seconds. The secondary benefit to the StringBuffers is that they have the insert() function, which allows me to add the feature of extra lines between functions. Basically, when it finds a “{“ in a class, then it can go back to the beginning of that line and insert another line.

Additional improvements include implementing support for conditionals that are not enclosed in brackets. Comments have also been made more robust. The lexer can now tell the difference between “//” and “/*” comments. This allows the proper recreation of these comments. It also means that multi-line “/*” comments can be be indented with the rest of the code around it. The lexer will recreate the comment line-by-line as until it finds the “*/” and it now will insert an indented new line, not just a new line. It has been designed to support many forms of Switch-Case statements. It also support Do-While loops, both single line and bracket-encased. There are a number of other minor improvements that are in response to errors or bugs that were found while testing.

Problems, Future plans:

The list of problems is dropping, but still present. There are some minor problems with nested Do-While loops, particularly if some of them are single line bodies and some are not. This will be worked out in time. Another minor bug is with “/*” comments; if it finds another /* for some reason, the lexer will remove the /* and start a new line. This involves rebuilding the lexer, and it is a rare instance, so it is low on the priority. At the moment, there is no functionality for line length, which is non-trivial, but still a problem, as it will put all of the cases in an if on one line, or all of the values of a pre-defined array. There are plans to do a similar kind of insertion technique for the pre-defined arrays, but that is also not simple, so that will need to be designed in full. There is no way for the lexer to know that the comments that it sees are on the same line as code or not. It will just put them when it sees them, which is usually after a semicolon or a right curly bracket, so in the end, it will almost always put them on the same line.

As for the other aspects of the project, the plan is to take the time to get familiar with Codewarrior, and learn the ways that code can be altered, and the ways that a Java function can be invoked. This means that there may be a fair amount of work just learning the methodology, before physical work can be done.

Project Structure:

If you open the workspace in MSDev, there are actually a number of projects gathered together. Here is a brief description of each project, and it sub parts:

Beautify:

This is the main project, and contains all of the classes to run the Beautify Applet.

About.java:

This is a simple pane for an about box.

Beautify.java:

This is the main class that creates a new UI.

Original.java:

This is the window that contains the source of the beautify process.

TokenType.java:

This contains the list of token constants.

UI.java:

This contains all of the GUI and the IO components. It extends frame, and then adds all of the AWT components. It also handless the event, and calls the updateCode function when the user requests and passes it all of the necessary settings from the GUI.

UpdateModule.java:

This is the main class that does the interpretation of the tokens, and recreates the code, as per the settings it gets passed. This is mainly a large Switch statement, and will alter what it adds to the code, depending on what tokens it sees. It will basically either add a space or a new line with indent.

Yylex.java:

This is the result of Jlex. This is the parser, which will sequentially return all of the tokens that it finds in the code that it is passed.

Jlex:

This is the generator for the lexer. It takes in the Yylex file, and creates Yylex.java, the lexer class. It only contains one file.

Main.java:

This is the main class for Jlex. You run it and pass it the grammar file as the argument, and it produces the lexer.

Lexer:

This is just a sample lexer that I used as an example, and contains no necessary code.

Unnecessary:

This was a project that I created, just so I could have easy access to certain files, and yet MSDev would not try to compile them.

Sample.lex:

This is a sample grammar file.

Scrap.txt:

A scratch pad for copy and pasting test cases.

Yylex:

The grammar file used by Jlex.

Usage:

To generate a new grammar file, a few steps need to be performed. First, set Jlex as the active project. Make all of the changes to the grammar file, and then run Jlex. It will produce Yylex.java in it's directory, so copy that into the beautify directory. Then set Beautify as the active project and run it.