Testing Swing Application

gowebing

I had to recently add UI tests for an application implemented with Swing library for the Posmulten project. The GUI does not do any rocket science. It does what the Posmulten project was created for, generating DDL statements that make RLS policy for the Postgres database, but with a user interface based on Swing components. Now, because the posmulten is an open-source project and the CI/CD process uses GitHub action, it would be worth having tests covering the UI application's functionality. Tests that could be run in a headless environment.

Testing Framework

As for testing purposes, I picked the AssertJ Swing library. It is effortless to mimic application users' actions. Not to mention that I could, with no effort, check application states and their components. 

Below is an example of a simple test case that checks if the correct panel will show up with the expected content after entering text and clicking the correct button.

Java
  
    @Test
    public void shouldDisplayCreationScriptsForCorrectConfigurationWhenClickingSubmitButton() throws SharedSchemaContextBuilderException, InvalidConfigurationException {
        // GIVEN
        String yaml = "Some yaml";
        ISharedSchemaContext context = mock(ISharedSchemaContext.class);
        Mockito.when(factory.build(eq(yaml), any(DefaultDecoratorContext.class))).thenReturn(context);
        List<SQLDefinition> definitions = asList(sqlDef("DEF 1", null), sqlDef("ALTER DEFINIT and Function", null));
        Mockito.when(context.getSqlDefinitions()).thenReturn(definitions);
        window.textBox(CONFIGURATION_TEXTFIELD_NAME).enterText(yaml);

        // WHEN
        window.button("submitBtn").click();

        // THEN
        window.textBox(CREATION_SCRIPTS_TEXTFIELD_NAME).requireText("DEF 1" + "\n" + "ALTER DEFINIT and Function");
        // Error panel should not be visible
        findJTabbedPaneFixtureByName(ERROR_TAB_PANEL_NAME).requireNotVisible();
    }


You can find the complete test code here.

Posmulten

The library for which the GUI application was created is generally a simple DDL statement builder that makes RSL policy in the Postgres database. The generated RLS policies allow applications communicating with the Postgres database to work in Mutli-tenant architecture with the shared schema strategy. 

For more info, please check below links:

Maven Configuration

It is worth excluding UI tests from unit tests. Although tests might not be fully e2e with mocked components, it is worth excluding from running together with unit tests because their execution might take a little longer than running standard unit tests.

XML
  
<profile>
            <id>swing-tests</id>
            <activation>
                <activeByDefault>false</activeByDefault>
            </activation>
            <build>
                <plugins>
                    <plugin>
                        <groupId>org.codehaus.gmavenplus</groupId>
                        <artifactId>gmavenplus-plugin</artifactId>
                        <version>1.5</version>
                        <executions>
                            <execution>
                                <goals>
                                    <goal>compile</goal>
                                    <goal>testCompile</goal>
                                </goals>
                            </execution>
                        </executions>
                    </plugin>
                    <plugin>
                        <groupId>org.apache.maven.plugins</groupId>
                        <artifactId>maven-surefire-plugin</artifactId>
                        <version>2.22.1</version>
                        <configuration>
                            <includes>
                                <include>**/*SwingTest.java</include>
                            </includes>
                        </configuration>
                    </plugin>
                </plugins>
            </build>
        </profile>


Full Maven file.

To run tests locally, on an environment with a Graphics card, you need to execute tests with a maven wrapper, like below.

Shell
  
./mvnw -pl :openwebstart '-DxvfbRunningTests=true' -P !unit-tests,swing-tests test


GitHub Action

Now, moving to the GitHub action, running the UI test on the environment with a Graphics card seems easy. However, there might be situations when some UI windows with a WhatsApp or MS Teams notification appear on the Desktop on which UI tests are executed, and our tests will fail. Tests should be repeated in such cases, but that is not the problem.

Many more problems can occur when we try to execute tests on a headless environment, which is probably the default environment for every CI/CD pipeline. And we still need to run those tests and ensure they will pass no matter if they are executed in such an environment.

When we ask how to execute UI tests in a headless environment, the first suggestion on the internet is to use the Xvfb. However, the contributors of AssertJ Swing suggest a different approach.

Our tests maximize windows and do other stuff the default window manager of xvfb doesn't support. TightVNC makes it easy to use another window manager. Just add gnome-wm & (or the window manager of your choice) to ~/.vnc/xstartup and you're ready to run.

GitHub

So, I followed suggestions from the contributors' team and used the Tightvncserver. I had some problems with adding the gnome-wm. Instead, I used the Openbox.

Below, you can see the step that runs UI tests.

The full GitHub action file can be found here.

The script files used to configure CI can be found here.

YAML
  
  testing_swing_app:
    needs: [compilation_and_unit_tests, database_tests, testing_configuration_jar]
    runs-on: ubuntu-latest
    name: "Testing Swing Application"
    steps:
      - name: Git checkout
        uses: actions/checkout@v2

        # Install JDKs and maven toolchain
      - uses: actions/setup-java@v3
        name: Set up JDK 11
        id: setupJava11
        with:
          distribution: 'zulu' # See 'Supported distributions' for available options
          java-version: '11'
      - name: Set up JDK 1.8
        id: setupJava8
        uses: actions/setup-java@v1
        with:
          java-version: 1.8
      - uses: cactuslab/maven-toolchains-xml-action@v1
        with:
          toolchains: |
            [
            {"jdkVersion": "8", "jdkHome": "${{steps.setupJava8.outputs.path}}"},
            {"jdkVersion": "11", "jdkHome": "${{steps.setupJava11.outputs.path}}"}
            ]
      - name: Install tightvncserver
        run:  sudo apt-get update && sudo apt install tightvncserver
      - name: Install openbox
        run:  sudo apt install openbox
      - name: Copy xstartup
        run:  mkdir $HOME/.vnc && cp ./swing/xstartup $HOME/.vnc/xstartup && chmod +x $HOME/.vnc/xstartup
      - name: Setting password for tightvncserver
        run:  ./swing/setpassword.sh
      - name: Run Swing tests
        id: swingTest1
        continue-on-error: true
        run:  ./mvnw -DskipTests --quiet clean install && ./swing/execute-on-vnc.sh ./mvnw -pl :openwebstart '-DxvfbRunningTests=true' -P !unit-tests,swing-tests test
        #https://www.thisdot.co/blog/how-to-retry-failed-steps-in-github-action-workflows/
#        https://stackoverflow.com/questions/54443705/change-default-screen-resolution-on-headless-ubuntu
      - name: Run Swing tests (Second time)
        id: swingTest2
        if: steps.swingTest1.outcome == 'failure'
        run:  ./mvnw -DskipTests --quiet clean install && ./swing/execute-on-vnc.sh ./mvnw -pl :openwebstart '-DxvfbRunningTests=true' -P !unit-tests,swing-tests test


Retry Failed Steps in GitHub Action Workflows

As you probably saw in the GitHub action file, the last step, which executes the UI tests, is added twice. After correctly setting up Tightvncserver and Openbox, I didn't observe that the second step had to be executed, except at the beginning during deployment when there were not a lot of UI components. I used Xvfb, and sometimes, the CI passed only after the second step. So even if there is no problem with executing the test the first time right now, then it is still worth executing those tests the second time in case of failure. To check if the first step failed, we first have to name it. In this case, the name is "swingTest1". In the second step, we use the "if" property like the below:

YAML
  
if: steps.swingTest1.outcome == 'failure'


And that is generally all for running the step a second time in case of failure.

Check this resource if you want to check other ways to execute the step a second time.

Summary

Setting CI for UI tests might not be a trivial task, but it can benefit a project with any GUI. Not all things can be tested with unit tests. 

 

 

 

 

Top