Educator Dashboard User Testing Completed!

Educator Dashboard User Testing Completed! 

By TzuChin Chen

This summer, I joined Killer Snails’ learning team as a research intern, focusing on user research. As I worked towards my Master’s degree in Games for Learning at NYU, my goal for this internship was to gain hands-on, industry experience in educational games-related user research. The Killer Snails team offered me the opportunity to design and conduct a user study on the usability of the educator dashboard for WaterWays, Killer Snails’ latest project. The educator dashboard seeks to provide teachers with information regarding their students’ progress, responses, questions, and comments. 

The research began with the learning team carefully parsing out the usability goals of the educator dashboard for it to serve its intended purpose. These goals include easy access to different classes’ dashboards, individual students’ work in WaterWays, and the students’ questions and comments. With the usability goals in mind, we put on our user goggles to test the educator dashboard ourselves and identified areas that need further improvement. Based on the first round of internal testing, we designed and developed a prototype of an alternative version of the educator dashboard that presented different information and navigation options compared to the existing version. 

To test the current design, and see if the alternative design might be a better  match with teachers’ expectations, we invited 5 elementary science teachers with varying levels of experience with WaterWays and other online learning management systems to participate in this user research study.

Over the span of one week, we met with each of our participants individually via Zoom to conduct the testing sessions. The testing session itself was comprised of three parts: 

  1. Complete four tasks such as viewing a class’ collective progress and specific students’ responses in a module on the existing educator dashboard. 
  2. Answer semi-structured interview questions about the existing dashboard.
  3. Complete the same four tasks and a short interview about the alternative dashboard design. 

As we were conducting the testing sessions, we were delighted to find out that our participants loved the information provided on the current dashboard! One participant commented that the current educator dashboard was “a lot less clunky than other educator dashboards.” Another participant stated, “I didn’t have any negative experiences [with the educator dashboard].” The visual presentation also received a lot of compliments. Overall, our participants expressed both satisfaction and excitement about WaterWays’ educator dashboard. 

Nevertheless, some usability issues were observed. Specifically, there were issues with navigating from one view to another on the educator dashboard, which caused some confusion and made some information more difficult to access than expected. Our participants pointed out features on the alternative design that were helpful to them and kindly proposed other potentially useful features that were not available on the current educator dashboard. 

Based on the user testing results, our learning and development team are working together to finalize the design of the educator dashboard to address teachers’ needs: 

  1. Guiding materials that introduce teachers to the features and functions of the educator dashboard. 
  2. Supportive features to guide intuitive navigation on the educator dashboard, including consistent color-coding of icons, placement of the ‘back’ button, and a user profile menu with shortcuts to important pages. 
  3. Improving access to view students’ responses, including more clearly labeling students’ progress within a module, redesigned icons and labeling to show read and unread messages with students, and a redesigned downloadable spreadsheet of students’ responses to better suit teachers’ needs for grading and assessment. 

Killer Snails is super excited to take our innovative educator dashboard to the next level! 

Our amazing intern graduating!