The RevEngE Data Collection Engine, built by Claire Taylor at the University of Arizona for her PhD dissertation, consists of three main components: reCord, reView, and reSolve. These are designed, respectively, to generate traces of individuals doing difficult tasks, visualize the data from multiple users for manual analysis and annotation, and aggregate and lift that data to higher representations with automated tools. You can read more about RevEngE here.
Users download the reCord endpoint software from the RevEngE webapp. The installer automatically provisions authentication data to the users, and the software itself records keystrokes, mouse input, active window information, running process information, and screenshots, all of which captures the human device interaction. Users can optionally enter annotations into a UI as they complete tasks. The data is streamed back to the webapp and stored in a database on the server.
The user data stored on the server is served into reView, a browser-based visualization. The visualization presents timeline data from all of the users’ active windows, allows playback of user activity, graphs process data, and contains an annotation engine to label all of the data.
Once annotated, the data can be analyzed by reSolve, contained in the visualization. These algorithms distill the annotation data into Petri Nets of what users are doing to complete tasks, combining the data from multiple users in order to generate a broad view of how users solve difficult reverse engineering problems.