Skip to content

Latest commit

 

History

History
65 lines (40 loc) · 4.13 KB

index.md

File metadata and controls

65 lines (40 loc) · 4.13 KB
title
Classes*v2 Mobile Usability

{% include JB/setup %}

Case Study

In January 2010, Yale's Center for Media and Instructional Innovation conducted a survey of mobile devices used to access Classes*v2, the University's learning management system built on the open-source Sakai platform. The results of this survey and a series of related discussions prompted further development, and in 2011 the Classes*v2 team completed an alpha version of a mobile web interface. A decision was made to open it up to a select group of pilot participants during the Spring 2012 semester. Our goal was to gather student and faculty feedback to inform ongoing development efforts. With no formal usability training, but plenty of gritty do-it-yourself determination, the team set about conducting a large-scale usability study.

The process of selecting and gathering feedback from the pilot participants was a multi-phased approach:

1. Open call for volunteers

Upon logging into Classes*v2, a message invited interested users to apply to be part of the pilot.

  • Sample language we used:
    • "Help us pilot a new mobile interface for Classes*v2!"
    • "Got a mobile device? Help us pilot a new mobile interface for Classes*v2!"
    • "The Classes*v2 team needs your help to pilot a new mobile interface!"

2. Selection of pilot participants

Pilot applications simply required the user's identity as well as which Classes*v2 tools they used regularly. Since the alpha version of the mobile web interface only worked with some tools, the team wanted to ensure a good fit between these parameters and the usage habits of potential pilot participants.

3. The call to action

We sent accepted pilot participants an introductory email:

"Thanks for your interest in the Classes*v2 mobile pilot. As a first step in optimizing Classes*v2 for mobile devices, we invite your feedback on an alpha release, which includes views of several key tools.

To access the mobile view, please visit https://the.alpha.url

We'll be sending you a quick survey in several weeks to learn about your experience. Your input on what works, what's missing, and what needs to be tweaked will be incredibly helpful as we continue building the mobile view.

We appreciate your participation in the alpha pilot and look forward to your feedback."

4. Gathering post-pilot feedback

After the month-long pilot period ended, we sent participants a survey about their experience. Out of 648 participants, 220 (34%) responded to the survey.

"Thank you for your participation in the pilot of the Classes*v2 mobile alpha. Please take a few minutes to complete a quick survey on your experience with the pilot. We're grateful for the feedback, and confident that it will help us further develop and improve the interface."

(The final survey was individualized for tracking responses.)

Here are the results of the survey, if you're curious. Included are some lessons learned and signposts that we used to prioritize development efforts.

Outcomes

  • Well over 500% mobile activity increase.
  • Over 300% increase in pages per visit during mobile sessions.
  • Over 300% increase in average time on site during mobile sessions (from roughly 20 seconds to about 90)
  • 25% increase in new mobile visitors
  • 57% decrease in the mobile bounce rate (a good thing!)
  • A focus group with student interns from the Instructional Technology Group to further refine the feedback we received
  • Coverage in the Yale Daily News
  • Users of phone-sized devices are now automatically redirected to the mobile web interface when logging into Classes*v2.

Lingering Questions

  • How might we have streamlined the process?
  • Was there anything missing from our approach, bearing in mind the scale of the effort?
  • What should we do when frequently recurring suggestions are seemingly impossible to deliver?