CS-E5210 - Usability evaluation, 26.02.2019-14.05.2019

Topic outline

• Groups' reports
  • 

    Instructions for customer meeting File

    PDF document
  • 

    Assignment 1: Research plan
  • 

    Assignment 2: Expert evaluation results
  • 

    Assignment 3: Empirical evaluation results
  • 

    Assignment 4: Final presentation slides

    Upload here the slides about your group's final presentation.

    Short instruction for the presentation: Your 10-minute presentation should focus on your research process: how you narrowed down your problem to manageable research questions, how you chose your methods, how you focused your expert evaluation and empirical evaluation, etc. Telling about the process will help you learn from each other's presentations. Tell also something about your findings, but don't attempt to cover all of them.

    The presentation will earn you max 5 points, and it will be based on your presentation, not these slides only. 

    The final report will be max 15 points.

  • 

    Assignment 5: Final report

    In the final report the maximum is 15 points.

    Use the following template and section headings.

    Cover page

    1. Introduction (1-2 pages)

    1.1. Description of the system

    1.2. Customer’s wishes

    1.3. Analysis of the needs for evaluation: based on the assessment of the system and the customer wishes, what parts of the system needed evaluation most?

    2. Research questions (1 page)

    2.1. Expert evaluation – Based on the analysis that you explained above, what RQs did you focus on?

    2.2. Empirical evaluation – Based on the analysis and findings from the expert evaluation, what what RQs did you focus on?

    2.3. Additional matters that affected the choice of RQs

    3. Methods (2-5 pages)

    3.1. Expert evaluations – presented like in the table below

    
    

    3.2. Empirical evaluations

    Present a similar table as the one above, and add details to Appendix B.

    3.2.1. Participant recruitment – Who did you seek to recruit? Who did you eventually got for the studies (present participant details in a table)

    3.2.2. Technical setup – Where did you run the evaluations?

    3.2.3. Other details – Use this to highlight any other important matters about the methods

    4. Findings (you can decide the length)
    

    4.1. Expert evaluations – 3-6 most important findings, with screenshots or other evidence (put other relevant findings to Appendix C)
    

    4.2. Empirical evaluations – 3-6 most important findings, with screenhots, video frames, transcriptions etc as evidence

    5. Discussion (you can decide the length)
    

    5.1. Limitations – What issues related to different types of validity

    5.2. Design suggestions – What do you recommend as a solution to 1-2 most important problems?

    Appendix A – Expert evaluations
    
    Cover here the methodological details:
    

    • How did you carry out these evaluations? Explain the methods in the level of detail that another usability expert would be able to carry out largely the same evaluation too only by reading your method description.
      
    • Did you use heuristics, and if yes, which ones, and how did you choose those ones in particular? 
      
    • Did you use scenarios, and if yes, which ones, and why those in particular?
      

    Appendix B – Empirical evaluations
    
    Cover here the methodological details:
    

    • How did you carry out these evaluations? Also here, explain the method in the level of detail that another usability expert would be able to carry out largely the same evaluation too only by reading your method description.
      
    • Did you use scenarios to contextualise and focus your data collection, and if yes, which ones, and why those in particular? Describe those scenarios
      
    • Did you have interviews? Before or after the study? What answers did you try to get? List the questions that you asked from the participants.

    Appendix C – Expert evaluation findings
    

    • All relevant usability problems and findings

    Appendix D – Expert evaluation findings
    

    • All relevant usability problems and findings