Home > About the eNLVM Project > User Testing

User Testing

User testing is an important method for evaluating online instructional tools and content such as the eNLVM. User testing involves observing target users (teachers or students) interacting with a product. Typically, a single user is observed as opposed to a group of users. Often the user is encouraged to speak out loud what they are thinking in order to gather as much data as possible about their experience. This data is then typically analyzed and summarized, and suggestions are made for improving the product. Using this method, a lot can be learned from just a few users.

In addition to being an effective method for evaluating tools and content, user testing can provide a rich source of data for understanding student learning and supporting teacher professional development. Observation data can be shared to communicate effective and alternative methods for using online materials and to help illuminate common student misconceptions and developmental stages.

One approach to recording student and teacher interaction with online resources is to record screen capture video and audio. A variety of software tools are available for this purpose. One of these tools for Microsoft Windows is CamStudio. While a complete archive of a session may be sufficient for user testing, if you want to share it on the web, it will probably need to be edited. Screen capture video files can easily be quite large and thus prohibitive to download. In addition, a complete archive typically contains lots of dead space or time that is probably not useful for teacher training.

The following videos are a few examples of user test data. Please note the file sizes and consider your connection speed.

Motivation or What? (22 sec, .9 M)

This activity was intended to help students see a motivation for place value by giving them a choice to be paid a $1.23 with the fewest bills and coins possible or by 123 pennies. See how the plan backfired. How could you revise it to accomplish the intended goal?

Finding the Trash (6 sec, .4 M)

In the base blocks applet, dropping blocks in the trash can worked fine with the small blocks. See how a student has problems trying to put blocks that are bigger than the can. How is that possible anyway?

Renaming Down (3.5 min, 7.9 M)

Watch a discussion between a teacher and a 2nd grade student about renaming during subtraction. Look for ways that the base blocks applet helps and hinders student understanding. How could you modify the design of the applet to better support understanding?

Questioning (5.5 min, 12 M)

Watch a 4th grader explore numberline and place value concepts using a numberline virtual manipulative. What role did questioning have in helping the student clarify their understanding? How could you achieve this in writing?