PoP – Test Report

A journey through the design thinking process

The last mode on the amazing journey through the design thinking process is the test mode. It’s an opportunity to test prototyped solutions and gain valuable feedback from the users of your product and their interaction with your solution(s).

The Context & Users

As I worked through each of the modes in the design thinking process up to the final test mode, I have discovered that there are many variables that may affect student retention in online courses. As I mentioned in an earlier blog post, the student population at the college where I teach is very diverse and includes many of the following variables, usually with multiple variables applied to a single student: taking a full-time or part-time course load, working full-time or part-time, holding one or multiple jobs, family responsibilities (children), caretaker for parents, grandparents or other family member(s), Federal Aid recipients, various income levels, varying GPA scores (we are an open enrollment college), commuter college (no on campus housing), various ethnic backgrounds, language barriers and special needs requirements. Given this information, the user group for the test mode for my problem of practice (increasing student retention in online courses) focuses on students within my online basic web programming course. This is an introductory course and the first course within the web programming degree program at the college.

Prototypes Tested & Why

Two different prototypes were tested during the test mode. The first prototype included a video that walked students through creating their first web page. The second prototype included the process of using an online collaborative coding tool called Collabedit that allows for real-time collaboration among multiple users during a single session. A tutorial PDF document was created that explained how to access and use Collabedit and its features so the user has a basic understanding of how to join and participate in a collaboration session prior to joining the session.

Within both prototype solutions, I was testing the effectiveness of the tool, ease of use, ease of access, and its functionality to determine if these solutions would be beneficial to online students and if the solutions would provide the students with additional support options that could be used for learning and collaborating with both the instructor and their classmates within their online classroom. I also wanted to determine if these solutions provided value and support to the user and assisted them in successfully learning the course material.

The Test Protocol(s) & Design

The testing period for both prototypes ran from March 27 – April 2 respectively.  I designed my tests to be available over a week long period in order to give users enough time to participate. I also designed the tests in a way that they would be available in an online setting to simulate a true user experience since the audience includes users participating in an online course. The design intentions were to make access to the prototypes and providing feedback as easy, convenient, and user-friendly as possible.

The test protocol for the video included the following:

  • A video was created that walks through the process of creating a basic web page using HTML. This video uses information from Chapter 2 with the course textbook as a reference point.
  • The video was uploaded to YouTube and initially set to “Unlisted”, students were provided with the URL to access the video.
  • An extra credit assignment was created within my online web programming classroom that offered students points for participation in reviewing and providing feedback based on the video walk-through.
  • The assignment link within my online classroom provided the user with some basic background information, instructions for accessing the video, and a series of some general questions regarding the video.
  • The users were given a deadline for submitting their feedback (one week).
  • Feedback was submitted through a drop box within the online classroom and assembled and documented within a Word document at the end of the submission period.

This slideshow requires JavaScript.

The test protocol for the online collaborative coding session process using Collabedit included the following:

  • A multipage tutorial was created that provides the user with basic information on how to access, use, join, and initiate an online collaborative coding session using Collabedit.
  • The tutorial provides instructions and screen shots documented the process.
  • Prior to joining a collaborative session, the user is supplied with the Collabedit tutorial document so he/she can prepare for the session.
  • I initiated the Collabedit session and invited the participant via email with some basic instructions and a URL that the user can copy/paste into their web browser to join the session.
  • The Collabedit sessions contained the HTML code the student participant needed assistance with and was present in the browser window upon joining the session.
  • The user was instructed to modify the HTML code within the session window and engage in conversation via the chat window during the session while we collaborated in real-time. The session lasted for approximately 45 minutes.
  • Feedback was submitted through email following the session and was based on a series of questions regarding the use of Collabedit.
  • Feedback was assembled and document within a Word document.

This slideshow requires JavaScript.

Outcomes

I received very valuable and useful user feedback. I had approximately half of my online basic web programming course students participate and provide feedback regarding the video walk-through that I created. I was able to conduct an online collaborative session using Collabedit with one of my web programming students. A few other students that were interested in testing Collabedit contacted me but due to scheduling conflicts, they were not able to participate.

Some feedback regarding what users liked included:

  • I love the fact that the video is broken down line by line explaining in detail the function of each tag.
  • I liked that in the end a resource was provided that would help you if you had any more questions on different topics.
  • I do find videos very useful to help me understand concepts within programming I’m finding difficult.
  • I enjoyed this video! Videos are most useful when concise and focused on one topic. I think that makes it much easier to refer back to them when necessary.
  • I’m a visual and kinetic learner so reading doesn’t help as much as hands on or visual instruction.
  • This youtube video takes it a step further and shows you how to create elements within the instructions.
  • I like tips and tricks, for example when you mention white space in HTML code for debugging.
  • I liked how you went step by step and introduced the basics of the class.
  • The language used was easily understood and presented well.
  • I absolutely would watch videos like this where a topic is challenging.
  • While using this tool (Collabedit) I thought it was very useful and nice to converse with you while figuring out where my errors were.
  • I liked the fact that we could, literally converse back and forth and I knew where you were talking when trying to help me, it made this learning experience easier and was very beneficial.

Some feedback for improvement included:

  • I would prefer that videos be entirely optional and cover topics that are historically harder to comprehend and not on every single topic we cover.
  • I do feel that this video was a little long nut it is packed with information. My suggestion may be to separate it into many parts maybe, like mini videos that way they are more specific to one topic.
  • After watching the whole video to the end my one complaint is the text front is a little small.
  • I feel as if you could zoom or make the text font larger it would be more user-friendly and helpful
  • The use of the yellow cursor bubble was distracting and caused a blur on the parts of the code when you hovered over them.  A different pointer might work better.
  • Maybe add a definition of terms of the code tags that you used  at the end just as a reference.
  • I think the length of the video is acceptable, but so is shorter. As I said, I find these types of videos useful, but they are most useful when concise and focused on one topic.

Reflecting on the Process

After wrapping up the testing mode and reviewing the user feedback, I learned a great deal from the process. The first and foremost lesson I learned, is that it’s absolutely necessary to get feedback from your users – the individuals that are actually using and interacting with what you created. It has to fulfill their needs and “work” for them. It should provide added value and make it worth their time. I also learned that in a few instances, I missed the obvious and a feature that I thought would be beneficial may be become distracting if it’s used throughout the entire video. All the user feedback that I received overwhelmingly indicated and verified that the two support options that I prototyped and tested are definitely resources that students see as valuable, supportive, time worthy, and assists them in their understanding of the materials and topics within the course.

Going forward, I will continue to create short, instructional videos for topics that students struggle with and include a video link within each module of my 8-week online web programming course. To accompany the instructional video, a PDF will be provided that highlights the specific HTML tags and web page elements that are discussed within the video along with any additional topic-related references to additional online resources. I will also be posting the Collabedit tutorial PDF document within my online classroom and will make a formal announcement to my students and encourage the use of Collabedit for real-time collaboration with classmates and instructor assistance in an online environment. I want to create a very positive and supportive online learning experience for my students with the goal of increasing student success, confidence, motivation, and student retention in online courses.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s