The OHT Plus Framework

An Application Framework for Non-Intrusive Testing Tools

The OHT+ Framework is completely open and portable. Its design describes a flexible architecture which can be adapted to many development and testing environments; even, this framework is also agnostic to the operating system and the windowing system. The framework described here is based in more than fifty percent on adaptable modules, allowing testers and developers to practically build any GUI Testing Tool on top of it, and easily fit that tool to the current testing environment in which is used. Another important feature is that the OHT+ Framework provides non-intrusive GUI introspection by using the DLL injection technique, which allows to easily extract information and data from GUI elements and GUI events.

The OHT+ Framework can be downloaded from SourceForge. Click  here to download.

The OHT+ Product Family for Usability Assessment

The OHT+ Product Family for Usability Assessment is composed of a set of open-source prototypes whose aim is measuring or evaluating several of the most important usability attributes. They are a User Interaction Recorder, a Usability Inspection Tool, and a User Experience Analyzer, that are adapted to Qt windowing system as the default GUI System in which to perform the different usability assessment processes. Anyway, these prototypes could be easily adapted to fit them to different testing environments, or to extend its functionality to support additional features. The following products (available at SourceForge.net) are open-source projects that have been written in standard C++, and use Qt library version 4.x for the user interface; so they can be built and run in Linux, Windows, or MacOs.

OHT+Behavior

The OHT+Behavior Prototype includes the basic functionality corresponding to the abstraction of the user interaction. The aim of this tool is to capture the mouse clicks and keyboard interaction on a GUI (i.e. it captures the events generated by these actions); then, this data is abstracted and used to create a user-interaction report composed of a heat map (shows how often the areas of a GUI are interacted by using colors) and a path map (shows a set of ordered steps that summarizes the user interaction; it also distinguishes between mouse click and key press events.) The GUI events captured by this tool provide excellent data for quantitatively characterizing user behavior.

This data is abstracted as a basic representation of mouse and keyboard events, and then used to draw, over a real snapshot of the tested application, the abstract diagrams obtained from the abstraction processes. It allows the usability experts to evaluate, in a very abstract level, the behavior of a set of users on a GUI: what elements were interacted, the sequence of performed actions, what areas were interacted more frequently, etc. This prototype could be extended in many ways, as it uses an abstract representation of the GUI events that can be extended to include other contextual information which may be very useful in testing or evaluation processes. It could be also easily adapted to other windowing systems by extending only the event capture module.

The OHT+Behavior tool can be downloaded from SourceForge. Click  here to download.

The following video shows how the user creates a new project associated to a binary file to be tested. Then, the user creates a new test-case and starts the interaction with the application. When finished, the OHT+Behavior ask for a location to store the two maps generated as a result of the abstraction process applied to the just performed user interaction. Finally, the resulting maps are showed.

OHT+Inspector

The OHT+Inspector Prototype is a Usability Evaluation Tool whose aim is to perform a reviewing process over a system or a GUI based on a set of constraints and guidelines. Some examples of these constraints are, for example, that all the widgets in a GUI have to follow a tab-order, that all the tooltip texts have to be filled, etc. This process is usually conducted manually by a usability expert (or a group of them), who focus his/her efforts on a list of areas of the GUI design that have proved to be troublesome for users. Once the problems are discovered, the experts make a set of reports including recommendations for resolving these issues.

The aim of this prototype is to encapsulate the knowledge of these usability experts and automate the inspection process; thus, it works as follows: first, the tool analyses the properties of a GUI by using GUI instrospection; then, these properties are used to create an abstract GUI using an abstraction process in which the GUI elements are tagged with roles; finally, a list of previously chosen constraints and guidelines are checked against the abstract GUI using a set of usability evaluators. The result of the evaluation process is a set of reports including the errors/warnings discovered, and in some cases advices to fix them. These reports are stored in a usability inspection project for further analysis. The tool also allows the tester to export the obtained reports to a PDF document and other open formats.

This prototype could be easily extended in order to add new usability evaluators to check new GUI properties or more complex constraints; it also could be extended to fit it to other testing environments where, for example, other windowing system is used.

The OHT+Inspector tool can be downloaded from SourceForge. Click  here to download.

The first of these videos shows how to create a new project and add a new binary to be evaluated. Then, it shows how to select a group of evaluators that are used to evaluate some properties and constraints in the GUI of the tested application. Finally, the obtained report containing all the inspection results is showed.

The second video starts with an existing project. Once the project is opened, the video shows how the user opens the usability-evaluators dialog and selects the evaluators he/she needs to create a new report. Once the evaluators are selected, the tester launches a new inspection process and provides the name of the reports is going to be generated. Finally, the obtained results are analyzed and the inspection project is saved to the disk.

The last videos shows the post-inspection actions that can be performed with the OHT+Inspector tool. First, we can see in the video how the tester loads an inspection project from the disk. Once the project is opened, the tester analyses all the results obtained from previous usability inspections performed on the tested application. Finally, the video shows how a report can be exported to some document formats. The different formats to which a report can be exported are PDF (Portable Document Format), ODF (Open Document Format), and plain text.

OHT+Experience

The OHT+Experience Prototype is a usability testing tool whose aim is to extract the user experience from the interaction with a software. User experience highlights the experiential, affective, meaningful and valuable aspects of Human-Computer Interaction (HCI) and product ownership. It also covers the perceptions of practical aspects such as utility, ease of use and efficiency of the system.

User Experience represents one of the most important usability attributes both for software developers and project managers, and is composed of impressions, comments, opinions, and suggestions from the users. It is very useful for usability experts and developers to measure user satisfaction level both with the GUI and the overall behavior of a software. This prototype allows the testers to perform most of the whole process of a usability test, as it allows to design the briefing stage (composed of a set of test cases to be performed by the users), guide the user during the test-case performing, store the comments and impression of the users and show them to the expert.

Once the expert has designed a set of test-cases to be performed by the users, during the test case execution, the OHT+Experience automatically conducts the test session telling the users what to do, and collecting their impressions, comments, and suggestions. Finally, both the test cases and the impressions of the user are stored in a project for further analysis. This prototype also includes a test conduction assistant, which shows the elements the user has to interact with by using visual help (i.e. the tool denotes with red color the elements that have to be interacted; it uses green color to denote the elements already interacted with).

This tool could be also used in many ways, as for instance to ease communication between experts and developers, as the usability experts can play the user role and use this tool to evaluate a GUI (both the GUI aspect and the use cases) and include all the comments and corrections for the development team to improve the GUI. Another way to use this tool is for designing operator guides. Before the software deployment, the development team can design the use cases using the OHT+Experience. Then, these use cases can be used to train the final user at the beginning of the software deployment phase. This prototype could be easily extended to enhance its features, or also to fit it to other testing environments.

The OHT+Experience tool can be downloaded from SourceForge. Click  here to download.

The first video shows how to design a usability test briefing by defining a set of use-cases associated to a software. Once the tester has created a project and selected a binary, he/she starts to design the use-case structure the user will perform. In the video, a new use-case is created and further information (i.e. descriptions) is added. A use-case is composed of a set of stages; for each stage it is necessary to add a description about what to do (for the briefing process) and select a set of widgets that have to be interacted with. Finally, the project is saved to the disk.

The second video depicts the user perspective. First, the user opens an existing project which is composed of a set of recorded use-cases (each one including the briefing information). Once the project is opened, the user select one of the available use-cases and starts performing it. As we can see in the video, the widgets that have to be interacted are denoted with red color; widgets already interacted are denoted with green color. When a stage of the use-case is completed, the OHT+Experience goes to the following automatically. At the end of each use-case (and also during the performance) the user can add a global rate (which represents its user-satisfaction level) and a set of comments or impressions.

The third video returns to the usability expert perspective. Once the project is opened, the usability expert can find the use-cases he designed along with the comments and ratings provided by the users during the usability-test execution. Thus, the usability expert can perform an analysis of the information provided by the users without the need of being present during the usability-test. Additionally, the expert can export the reports to PDF, ODF and plain text formats.

---

These prototypes are real software that have been tested in real industrial scenarios. They have been developed by the Cátedra SAES Team as Open Source contributions, and its source code and additional information are available at the Cátedra SAES website (www.um.es/catedrasaes). This work is supported by the Cátedra SAES Team, the SAES Company and the University of Murcia.

For further information, you can contact Fran San Pedro at  fjsanpedro@gmail.com or Pedro Mateo at  pedromateo@um.es