24. Graphical User Interfaces
So far, the programs that we have seen and written in the course have had straightforward user interfaces. Some of these programs (for example, the Huffman coding exercise that you completed in Discussion 9) have been batch applications, reading input data from a file at the start of their execution and writing output data to a different file at the end. This paradigm is indicative of the very earliest days of computing, when programs and their inputs were cut onto punch cards that were loaded into a computer. After the computer finished its computation, it would punch its output onto a different card to be interpreted by the user. Similar in spirit are programs whose only opportunity for user input comes through supplying program arguments; once the actual computation begins, the user has no way to pass additional information to the program.
Beginning around the 1960s, programs began to be more interactive. Rather than requiring all of their inputs up front, applications could now pause in the middle of execution to prompt for additional inputs on a command line. An execution of the program typically involved interaction between the user and the application on the console. The user prompts affected the next sequence of operations performed by the application (and potentially also which prompts they would receive next). This is the paradigm that we have used for most of the course, using Java objects like Scanners to process command-line inputs, and using statements like System.out.println() to write messages to the console. The interactive nature of these command-line applications provides a much better user experience and unlocks the ability to effectively use computers for many more tasks.
Around this same time, computer scientists began to focus on questions of human-computer interaction, asking how we could enable richer interactions between users and applications that would expand the utility of computers in our everyday lives. One groundbreaking example of this research came in 1968, with the so-called Mother of All Demos. In this 90-minute presentation at the ACM/IEEE Joint Computer Conference, Douglas Engelbart introduced the idea of “Personal Computing”. He conceptualized many technologies that remain popular to this day, including computer mice, file menus, hyperlinks, word processing, copy-paste tools, document sharing, and teleconferencing. Most notably, Engelbart demonstrated the power of graphical applications, pushing computers past simply processing lines of text to laying out multiple widgets on the screen that the user could choose to interact with. Of course, these graphical user interfaces (or GUIs) have become the dominant paradigm for almost all software we interact with today, from websites to phone apps to other software applications.
Over the next two lectures, we will give a brief overview of writing graphical, interactive programs. As we’ll see, graphical applications are significantly more complicated than their text-based counterparts; many more choices must be made as we design their interfaces. In addition, the presence of multiple widgets enables the user to interact with the program in different, less predictable ways, and we must be prepared to correctly process these interactions. In today’s lecture, we’ll focus on the visual portion of these graphical applications, how we lay out various components and control their appearance on a screen. The next lecture will focus on processing user interactions through event-driven programming.
Structure of Graphical Applications
Over the course of the next two lectures, we will build a GUI Tic-Tac-Toe application that allows the players to make moves by clicking on cells in the grid.
What is needed for this application?
This is a lot of stuff to consider! To better conceptualize all of the considerations for GUI applications, we typically divide them into three different aspects: the application’s model, view, and controller.
Of these, the model should be the aspect with which you are most familiar.
The model of a graphical application consists of the objects that represent the underlying state of the application at any point during its execution, along with the methods that access or modify this state.
Just as we have throughout the course, we should ask ourselves, “What information will I need to keep track of to make the program work?” and use this to decide on the state representation of the model. In our Tic-Tac-Toe example, the model will keep track of whose turn it is, which cells contain which symbols, and which cells can be chosen on the next turn. It will also include the logic for determining when the game has ended and which (if any) player has won. In fact, we already wrote a model class for Tic-Tac-Toe earlier in the semester, when we first introduced state representations and class invariants, the TicTacToe class.
|
|
|
|
Previously, we wrote a command-line application, TicTacToeConsole, that used the console to prompt the players for the locations of their next moves. Now, we’ll develop a graphical application that uses this same TicTacToe class as its underlying model. Notice that the model only represents the “behind-the-scenes” state of the application; it does not contain any variables that represent what actually appears on the screen. The visible windows and widgets constitute the application’s view.
A widget is any graphical component that appears on the screen when a graphical application is running. Widgets are typically used to display information about the state of the program or provide an interface through which user data can be collected through interactions.
The view of a graphical application describes the windows and widgets drawn by the application, including information about their individual appearances and how they are arranged together on screen.
We will spend most of today’s lecture discussing how to design, initialize, and update the view of an application. Finally, and central to the goal of designing programs that offer the users the opportunity for richer interaction, graphical applications have a controller.
The controller of a graphical application contains the logic that responds to user inputs (or events) by appropriately updating the model and/or view to accurately reflect the application's new state.
Writing the controller, an example of event-driven programming, is the focus of the next lecture. While the design of all graphical applications will include creating a model, a view, and a controller, there is no paradigmatic way to organize these in our code. For example, sometimes it will be convenient to have separate classes that are responsible for managing the model (such as our TicTacToe class) and have our view’s widget objects query this model class to understand how they should be drawn. Other times, it may be better to decentralize the model, integrating pieces of it within different view objects to allow easier access to connected parts of the view and the application state.
GUI Frameworks
Because of the tight coupling and varied forms of interaction between these different aspects of a graphical application, we almost always rely on frameworks when developing graphical applications.
A framework is a large software library that includes base implementations of many different view components and utility classes for managing their layout, rendering, and event handling.
For example, web developers use frameworks like React, Angular, and Spring to build upon the core HTML/CSS/JavaScript languages that form the foundation of many websites.
Frameworks can help to abstract away many of the lower-level details of graphical applications. For example, these libraries interface with the operating system to process hardware events such as mouse clicks and keyboard input. They also rely on core routines of your system to manage how windows are drawn and managed, and (as we will discuss more soon) manage the application threading model that allows computation, event handling, and rendering to be scheduled to execute simultaneously across multiple cores of your machine. Since we don’t need to worry about these details, we can focus our attention on writing code that describes how we want our application to look and behave.
There are many different frameworks in Java. The first of these was the Abstract Window Toolkit (AWT), which allowed users to build up GUIs using native (to a particular operating system) elements. Since it was tied to operating-system-specific functionality, it could only offer features that were supported by all platforms.
The next iteration of Java’s internal framework was Swing, which came out in 1998. In Swing, all of the drawing and event handling is done by Java, rather than relying on the operating system to do much of this work. This allowed more functionality to be supported and built on top of the AWT abstractions. Since Swing continues to be packaged with current Java releases, it offers a lower barrier to entry to new developers and will be our framework of choice in CS 2110.
More modern alternatives to Swing, such as JavaFX – the framework of choice in CS 2112 – utilize web technologies to offer an enhanced set of features and better rendering. These frameworks are not packaged with Java by default, and often have a steeper learning curve, but their enhanced features make them preferable to more experienced developers.
Building an Application View
The first two steps to write a graphical application are deciding on its model (i.e., state representation) and designing its view. For our Tic-Tac-Toe example, we have already completed the model, so we’ll focus on the view.
JFrames
A graphical application is contained within one or more windows, rectangular portions of the screen that provide a designated space for that application’s widgets. Therefore, the first step to building an application’s view is constructing a window. Swing uses the JFrame class to model an application window decorated with a title bar and close buttons. If you open the JFrame documentation, you’ll see that it includes hundreds of different methods for controlling various aspects of the window. We will discuss only a few of the most important features.
Since we want to leverage a lot of existing functionality in our graphical applications, we will make heavy use of inheritance. For example, our main TicTacToeGraphical class (that models our full graphical application) will extend JFrame; in a sense, our application is synonymous with the window in which it is running. Within the constructor for this class, we’ll use some JFrame methods to initialize its appearance and behavior.
- We’ll set the title of the frame to be “Tic-Tac-Toe”. This text will appear in the top bar of the frame window.
- We’ll specify the behavior that should happen when the user closes the window. A good option for this is
JFrame.DISPOSE_ON_CLOSE(a constant that theJFrameclass has defined to signal a desired close behavior), which will gently terminate the application. - We’ll set other properties of the window. For this application, we don’t want the user to be able to resize the window by dragging its edges. We can specify this by calling
setResizable(false). Later today, when we introduce layout managers, we’ll see how the initial size of the window is determined.
|
|
|
|
Now, we’d like to make our application run and show this window. As usual, the entry point of our application will be a main() method, which we can add to the TicTacToeGraphical class. In this main() method, we’ll introduce a standard piece of code to initialize a Swing application.
TicTacToeGraphical.java
|
|
|
|
Here, the static method SwingUtilities.invokeLater() takes in a Runnable object as its parameter. Notice that Runnable is a functional interface since it declares a single method, run(). Therefore, it can be instantiated with a lambda expression.
The lambda expression that we wrote encapsulates code to construct a new TicTacToeGraphical object and then make it visible on the screen, so that the user can interact with its application window. We use this SwingUtilities.invokeLater() method (rather than constructing the TicTacToeGraphical object directly) to hand over control to the Swing framework to execute this code on its terms. For Swing to function directly, it needs all of its components (i.e., view elements) to be constructed on something called the event dispatch thread (we’ll discuss threads in greater detail in a couple of lectures). This ensures that the framework can manage the application’s controller and update the application’s state correctly in response to events. We’ll continue our discussion of the event dispatch thread in the next lecture. For today, the main takeaway is to always write the code initializing the main application JFrame within a call to SwingUtilities.invokeLater().
When we run our main() method, we see that a new window is created; however, it looks rather small and unimpressive.
We haven’t yet added any widgets to the application. The window is empty, so it is rendered at the smallest possible size, too small to even read its title. Next, we’ll add the other widgets to the frame to build out its view.
Other Widgets
In addition to Windows, such as the JFrame that we just introduced, the Swing framework includes many different types of widgets, which are subtypes of the JComponent class (you can think of component as a synonym for widget). We’ll introduce a few common JComponent subtypes here, but encourage you to explore the Swing tutorial for a more thorough documentation of what is available. Overall, we can visualize (a chunk of) the Swing type hierarchy as follows:
The Component and Container classes (from the older AWT framework) offer higher levels of abstraction that manage many common component behaviors (such as managing layout, which we will discuss shortly). As far as the primary JComponent subtypes:
JPanels act as intermediary containers that can help arrange multiple other components in a particular configuration.JLabels are used to display (decorated) text within the application window.JButtons allow the user to click on a specific portion of the screen to trigger a particular action in the controller.
We will use all three of these JComponent subtypes in our application. We’ll construct these components within the TicTacToeGraphical constructor.
At the top of the window, we’ll have a JLabel that displays the message of whose turn it is. At the start of the game, it’s the “X” player’s turn. We can initialize this message by selecting a JLabel constructor that accepts it as an argument. In addition, this constructor takes in the horizontal alignment of the text, which will allow us to center the message within the label. After constructing the JLabel object, we can use some of its other methods to further control its appearance.
|
|
|
|
We don't expect you to memorize all of the methods for controlling various visual features of JComponents. Rather, we expect you to be familiar with some of the main ones (like add(), pack(), and repaint() that we'll discuss soon) and be comfortable searching through the documentation to find other methods that you need.
At the bottom of the window, we’ll have a JButton that will be used to reset the game. We’ll construct this JButton, passing in its label text to its constructor. Then, we’ll call a method to increase the font size of this label.
|
|
|
|
In the middle of the screen is the Tic-Tac-Toe board. We intend to have the users click on the Tic-Tac-Toe cells to make their moves, so it makes sense for these cells to be nine separate JButtons. We’ll want a way to group together these nine buttons into a single component, and a JPanel is a good candidate for this. To improve the modularity of our code, we’ll model this as a separate class, a TicTacToeGrid that extends JPanel. We’ll return to the definition of this class shortly.
The Component Hierarchy
Now that we have an idea of all of the widgets that we need for our application, we’ll need a way to connect them all together within the application window. We do this by establishing parent-child relationships between various Components. A parent component visually contains its child components. For example, both the JLabel turnLabel and the JButton resetButton will be children of the main JFrame, since these widgets sit directly in the frame. The TicTacToeGrid panel will also be a child of the JFrame, and it will, in turn, have its nine cell buttons as its children. If we draw all of the parent-child relationships of the various Components in our view, we end up with a tree structure rooted at the application JFrame:
Here, we're using the Swing component names rather than our custom subclass names to make it clearer which types of widgets we're using in our application.
We call this tree the component hierarchy of the application.
The component hierarchy of a graphical application is the tree that models the parent-child relationships of all the components of its view.
To establish the parent-child relationships in the component hierarchy, we call the add() method on the parent component, passing in a reference to its child component.
Layout Managers
Once we know which widgets will be the children of which other widgets, we still need to specify how these widgets should be arranged (or laid out) within their parent container. This is the responsibility of the container’s LayoutManager. Swing provides many default layout managers. We will introduce two of them here.
BorderLayout:
The BorderLayout is the default layout manager of the JFrame. It includes five regions that can each contain up to one child widget: its CENTER, NORTH, SOUTH, EAST, and WEST regions.
When we add() widgets to a JFrame with a BorderLayout, we pass in an argument to specify in which region they should be placed. In our Tic-Tac-Toe application, we want the JLabel to be in the NORTH region, the JButton to be in the SOUTH region, and the JPanel (i.e., the TicTacToeGrid) to be in the CENTER region (which is the default).
We call the pack() method on a JFrame after adding all of its components to inform the layout manager (and recursively inform the layout managers of all child components) that we are ready for it to lay out the components on the screen.
|
|
|
|
GridLayout:
A GridLayout arranges its child components in equally-sized cells within a grid with a specified number of rows and columns (and optionally with a specific amount of hgap and vgap spacing between these cells). This layout will be perfect for the buttons in our TicTacToeGrid.
We’ll make the background of the TicTacToeGrid panel black so the spacing between the buttons forms the lines of the Tic-Tac-Toe grid. We’ll model the cells with a custom JButton subclass, TicTacToeCell, and use some JButton methods within the constructor to modify the buttons’ appearances.
TicTacToeGrid.java
|
|
|
|
Within a GridLayout, the child components are, by default, added in row-major order, meaning that the first row of cells is filled from left to right before moving on to subsequent rows.
TicTacToeCell.java
|
|
|
|
If we re-run our main() method, we’ll see that our application view now resembles what we were targeting.
Connecting the Model
Now that we have a working view and a working model for our Tic-Tac-Toe game, we need to combine them. To do this, we’ll associate a TicTacToe model object with one of our component classes. A natural choice is to associate the model with the TicTacToeGrid, as the grid is the component whose view (a grid of nine buttons) most naturally aligns with the model’s state (which keeps track of the symbols in each of the nine cells).
We’ll add a TicTacToe model field to our TicTacToeGrid class, and we’ll initialize this field by calling the TicTacToe constructor from within the TicTacToeGrid constructor.
TicTacToeGrid.java
|
|
|
|
We have a bit more work to do in the TicTacToeCell class. Each cell will need a way to keep track of its state (which symbol, if any, it contains). This can be accomplished by giving it a char symbol field.
Technically, this field is redundant. The TicTacToe class already stores (and makes accessible) the content of all the cells. We could refactor the code to remove this redundancy, relying on the observer pattern that we will discuss next lecture to notify the cells when the game state has changed so they can re-render to reflect their updated state. We'll use the observer pattern for a different part of the application (updating the turn label), and we'll use this symbol field to demonstrate another approach.
We’ll maintain the class invariant that the symbol is either ' ' (indicating an empty cell), 'X', or 'O'. Upon construction, each cell is empty. We’ll add methods that will allow the TicTacToeGrid class to update these symbols when it detects a click (more on this next lecture). Within these methods, we’ll also use the setEnabled() method to prevent the user from clicking on a cell that has already been claimed.
TicTacToeCell.java
|
|
|
|
Custom Painting
To wrap up our discussion of our application’s model and view, we’ll discuss custom painting, a way to take greater control of the styling of our application’s widgets. We can use custom painting to decorate widgets with different colored shapes, lines, and/or text. To perform custom painting, we override the paintComponent() method, which accepts a Graphics object as its parameter. We always begin this method definition with a call to super.paintComponent() to make sure the re-rendering executes correctly.
You can think about a Graphics object as behaving like a paintbrush. To draw something onto this component, we first select the color that will next be used with the Graphics.setColor() method, “dipping” its brush in that color of paint. Then, we dictate what should be drawn by using one of the fill() or draw() methods in the Graphics class. These methods take in the coordinates of various shapes, placing a grid over the component whose (0,0) coordinate point is its upper left corner.
For example, the following paintComponent() method shown on the left would result in the pixel coloring on the right (with the solid illustrations representing the ideal shapes and the translucent shading showing the coloring at the pixel granularity).
|
|
|
|
We’ll use custom painting to draw the symbols onto our Tic-Tac-Toe cells. Trace through the following code to see how we build up these “X” and “O” shapes. Note that we use a switch expression to determine which color to make the background of the cell button based on its symbol.
TicTacToeCell.java
|
|
|
|
paintComponent() is called once at the beginning of the application when we first make the frame visible. If we want to trigger a re-painting of a component at a later point (e.g., to reflect a change in the model), we never call paintComponent() directly. Rather, we use the repaint() method to ask the Swing framework to schedule a repaint at the next convenient time. This helps to make sure the event handling of our application proceeds smoothly.
In the case of our Tic-Tac-Toe application, we’ll want to repaint() a cell from within its addSymbol() method.
TicTacToeCell.java
|
|
|
|
For now, we’ll hard-code some calls to the addSymbol() method at the end of our TicTacToeGrid constructor so we can test out our custom painting:
|
|
|
|
Re-running our TicTacToeGraphical application, we see that its view exactly matches the image from the start of the lecture.
In the next lecture, we’ll talk about how to add the controller to this application to make this an actual responsive game.
Main Takeaways:
- Over time, programs have evolved to become more interactive, transitioning from batch applications to command-line applications to modern graphical applications.
- We often divide the design of a graphical application into three separate aspects: its model (underlying state representation), view (windows and widgets), and controller (interaction logic).
- GUI frameworks, such as Java's Swing framework, package together many classes for writing graphical applications.
- The component hierarchy describes the parent-child (nested containment) relationships of different members of an application's view.
- Layout managers are responsible for arranging components on the screen and are requested using the
pack()method. - To perform custom painting on a
JComponent, override thepaintComponent()method. We never call this method directly; rather, we request that it be executed by callingrepaint().
Exercises
JComponent that should be rendered as the CS2110 logo. Which method should you override to perform the drawing commands to create this logo?TicTacToe to track the wins per player and the number of tie games. You may need to modify the constructor and checkForGameOver() to maintain the class invariant for your new fields.
reset() method to our TicTacToe class to do so.
reset() method in TicTacToeGrid that invokes TicTacToe.reset() and clears the symbols from all the grid cells.
Implement a TicTacToeScoreboard class that will display the stats. The constructor should define 3 JLabels that display each player’s wins and the number of draws.
|
|
|
|
TicTacToeScoreboard to the right of the main grid.
TicTacToePlayerCustomizer, extending JPanel that handles the view for this personalization. Add two JComboBoxes to this panel that allow each player to select a shape to represent themself. Peruse the documentation for JComboBox.
JRadioButton and this tutorial useful.
TicTacToeCell.paintComponent() to accommodate these new changes. You may want to add new states to this class.
TicTacToePlayerCustomizer to the left of the main grid.
paintComponent()
JPanel with dimensions 100 \(\times\) 100. Draw what these panels will look like after a call to repaint().
|
|
|
|
|
|
|
|
|
|
|
|
TicTacToeGraphical, define a class Connect4Graphical. This should set up the frame and have a main() method that can run the application.
To model the board, we’ll represent each column as a button. The button will maintain a state of the chips in its column, which will inform the rendering logic.
|
|
|
|
paintComponent(). A 0 should be painted black, 1 red, and 2 yellow. Remember that the positive y direction is downwards.
Implement the following add() method, which adds a chip to this column.
|
|
|
|
We can now use this column component to build our board. Implement Connect4Board to construct a 6 \(\times\) 7 board with a model.
|
|
|
|
Connect4Board to Connect4Graphical.