Assignment 8: User behaviour prediction
- 5 November: added Step 3.
This assignment familiarises you with a simple yet widely used model in human-computer interaction. The task consists of two steps:
- First you'll find parameters for the model so that the model will fit the given observed user behaviour.
- Then, after fitting the model, you will use the model to investigate contrafactual ("what if") scenarios. In other words, the model will let you predict how other types of users would use the system.
The model is based on Fitts' law, which was featured in Lecture 8. The use context in this assignment is one finger mobile phone texting. The assumption of one finger makes it easy to model the users' behaviour with a single Fitts' law. Further, we assume that the user does not need to do visual search of buttons, or proof reading of the already typed text.
The source code (fitts-model.py) contains all the code that is required for completing the assignment. The code is documented, and you should read the in-line comments to understand what the functions and function calls do. For simplicity, all the data is hand-crafted for the purpose of this assignment and is not authentic user data.
You can do one of the following:
- Download the Python code from the link above and work with it on your own computer.
- Similarly with Assignment 4, run the code in Microsoft Azure notebook by cloning it from this repository to your Azure workspace, editing it there, and running it in the cloud. See the more detailed instructions on cloning and Azure notebooks from Assignment 4.
- Work with Excel tables that replicate the code's behaviour.
Step 1. Parameter estimation
The source code contains log-data based text message typing times from two users: data_user_u1 and data_user_u2. These variables contain sentences and how many seconds it took from a user to type those sentences. Your task is to find Fitts' law parameters a and b for these two users. For any two parameters a1 and b1, you can check how well the Fitts' law, equipped with these parameters, corresponds to the user data by calling the function prediction_error() with a1, b1, and the target user data as parameters.
Your goal is to find such values for a and b, separately for users u1 and u2, that result in as small a prediction error as possible.
Tip: if you don't wish to try values of a, b one by one, you can use function grid_search(), which accepts a list of a and b values, and iterates through all combinations, calculating the prediction error for them. The source code's last row shows how you can call the function and print its results.
In your report, present the best values for a and b separately for both u1 and u2, and describe how you found out these parameters. In addition, check the lecture slides, p. 15, and try to identify which user group u1 and u2 probably represent, given that the Table's "Parameter of interest", "Cycle time of the motor processor" can be matched to parameter a, and "Fitts' law slope constant" can be matched to parameter b.
Note. Fitts' law is a fairly simple model. As discussed in the lecture, the reality of typing behaviour is much more extensive than what can be captured with one equation and two parameters. If you are interested, you should follow up on the papers cited in the lecture and considering enrolling to User research course at Aalto the next time it is lectured.
Step 2. Forward modelling
"Forward modelling" refers to using a model for predicting user behaviour of hypothetical ("counter-factual") users.
For this task, take a look at the specification of a keyboard layout (in Python, see the keyboard variable in the source code; in Excel, see the table titled "Keyboard layout"). Using the existing layout as your example, create a new keyboard layout that follows the Dvorak simplified keyboard layout. You only need to replicate the lowercase letters a-z and the spacebar.
Your task is to predict the typing time for the sentence "how fast can you come to my place" for both users u1 and u2, with both Qwerty and Dvorak layouts. Use the user-specific parameters for the Fitts' a and b that you uncovered in Task 1 and the call fitts_typing_time within print function to obtain results.
In your report, present a table like this with the typing times in seconds entered in the empty cells:
User 1 User 2 Dvorak Qwerty
Note. In this assignment, we assume that the users are experts with both Qwerty and Dvorak layouts. For more realistic predictions, you would need some sort of a visual search model and keyboard layout learning model. See the lecture slides for an introduction to them.
Step 3. Reflecting the value of modelling
Provide an example of a situation where a model of a user and an ability to use it to predict user's behaviour can be beneficial. The model can, for instance, help in improving system's usability or providing useful system features. Your example can be also from some other situation than text typing.
Save your report in PDF and submit it at the latest on Sunday 11 November at 23:55.