Problem of Practice Final Design Report

The culmination of the work completed in CEP 817 entailing the use of the Stanford d.school Design Thinking Process is included within my Problem of Practice Final Design Report for Increasing Student Retention in Online Courses for review: Final Design Report – CEP 817. It has been a fantastic journey!

Advertisements

PoP – Test Report

A journey through the design thinking process

The last mode on the amazing journey through the design thinking process is the test mode. It’s an opportunity to test prototyped solutions and gain valuable feedback from the users of your product and their interaction with your solution(s).

The Context & Users

As I worked through each of the modes in the design thinking process up to the final test mode, I have discovered that there are many variables that may affect student retention in online courses. As I mentioned in an earlier blog post, the student population at the college where I teach is very diverse and includes many of the following variables, usually with multiple variables applied to a single student: taking a full-time or part-time course load, working full-time or part-time, holding one or multiple jobs, family responsibilities (children), caretaker for parents, grandparents or other family member(s), Federal Aid recipients, various income levels, varying GPA scores (we are an open enrollment college), commuter college (no on campus housing), various ethnic backgrounds, language barriers and special needs requirements. Given this information, the user group for the test mode for my problem of practice (increasing student retention in online courses) focuses on students within my online basic web programming course. This is an introductory course and the first course within the web programming degree program at the college.

Prototypes Tested & Why

Two different prototypes were tested during the test mode. The first prototype included a video that walked students through creating their first web page. The second prototype included the process of using an online collaborative coding tool called Collabedit that allows for real-time collaboration among multiple users during a single session. A tutorial PDF document was created that explained how to access and use Collabedit and its features so the user has a basic understanding of how to join and participate in a collaboration session prior to joining the session.

Within both prototype solutions, I was testing the effectiveness of the tool, ease of use, ease of access, and its functionality to determine if these solutions would be beneficial to online students and if the solutions would provide the students with additional support options that could be used for learning and collaborating with both the instructor and their classmates within their online classroom. I also wanted to determine if these solutions provided value and support to the user and assisted them in successfully learning the course material.

The Test Protocol(s) & Design

The testing period for both prototypes ran from March 27 – April 2 respectively.  I designed my tests to be available over a week long period in order to give users enough time to participate. I also designed the tests in a way that they would be available in an online setting to simulate a true user experience since the audience includes users participating in an online course. The design intentions were to make access to the prototypes and providing feedback as easy, convenient, and user-friendly as possible.

The test protocol for the video included the following:

  • A video was created that walks through the process of creating a basic web page using HTML. This video uses information from Chapter 2 with the course textbook as a reference point.
  • The video was uploaded to YouTube and initially set to “Unlisted”, students were provided with the URL to access the video.
  • An extra credit assignment was created within my online web programming classroom that offered students points for participation in reviewing and providing feedback based on the video walk-through.
  • The assignment link within my online classroom provided the user with some basic background information, instructions for accessing the video, and a series of some general questions regarding the video.
  • The users were given a deadline for submitting their feedback (one week).
  • Feedback was submitted through a drop box within the online classroom and assembled and documented within a Word document at the end of the submission period.

This slideshow requires JavaScript.

The test protocol for the online collaborative coding session process using Collabedit included the following:

  • A multipage tutorial was created that provides the user with basic information on how to access, use, join, and initiate an online collaborative coding session using Collabedit.
  • The tutorial provides instructions and screen shots documented the process.
  • Prior to joining a collaborative session, the user is supplied with the Collabedit tutorial document so he/she can prepare for the session.
  • I initiated the Collabedit session and invited the participant via email with some basic instructions and a URL that the user can copy/paste into their web browser to join the session.
  • The Collabedit sessions contained the HTML code the student participant needed assistance with and was present in the browser window upon joining the session.
  • The user was instructed to modify the HTML code within the session window and engage in conversation via the chat window during the session while we collaborated in real-time. The session lasted for approximately 45 minutes.
  • Feedback was submitted through email following the session and was based on a series of questions regarding the use of Collabedit.
  • Feedback was assembled and document within a Word document.

This slideshow requires JavaScript.

Outcomes

I received very valuable and useful user feedback. I had approximately half of my online basic web programming course students participate and provide feedback regarding the video walk-through that I created. I was able to conduct an online collaborative session using Collabedit with one of my web programming students. A few other students that were interested in testing Collabedit contacted me but due to scheduling conflicts, they were not able to participate.

Some feedback regarding what users liked included:

  • I love the fact that the video is broken down line by line explaining in detail the function of each tag.
  • I liked that in the end a resource was provided that would help you if you had any more questions on different topics.
  • I do find videos very useful to help me understand concepts within programming I’m finding difficult.
  • I enjoyed this video! Videos are most useful when concise and focused on one topic. I think that makes it much easier to refer back to them when necessary.
  • I’m a visual and kinetic learner so reading doesn’t help as much as hands on or visual instruction.
  • This youtube video takes it a step further and shows you how to create elements within the instructions.
  • I like tips and tricks, for example when you mention white space in HTML code for debugging.
  • I liked how you went step by step and introduced the basics of the class.
  • The language used was easily understood and presented well.
  • I absolutely would watch videos like this where a topic is challenging.
  • While using this tool (Collabedit) I thought it was very useful and nice to converse with you while figuring out where my errors were.
  • I liked the fact that we could, literally converse back and forth and I knew where you were talking when trying to help me, it made this learning experience easier and was very beneficial.

Some feedback for improvement included:

  • I would prefer that videos be entirely optional and cover topics that are historically harder to comprehend and not on every single topic we cover.
  • I do feel that this video was a little long nut it is packed with information. My suggestion may be to separate it into many parts maybe, like mini videos that way they are more specific to one topic.
  • After watching the whole video to the end my one complaint is the text front is a little small.
  • I feel as if you could zoom or make the text font larger it would be more user-friendly and helpful
  • The use of the yellow cursor bubble was distracting and caused a blur on the parts of the code when you hovered over them.  A different pointer might work better.
  • Maybe add a definition of terms of the code tags that you used  at the end just as a reference.
  • I think the length of the video is acceptable, but so is shorter. As I said, I find these types of videos useful, but they are most useful when concise and focused on one topic.

Reflecting on the Process

After wrapping up the testing mode and reviewing the user feedback, I learned a great deal from the process. The first and foremost lesson I learned, is that it’s absolutely necessary to get feedback from your users – the individuals that are actually using and interacting with what you created. It has to fulfill their needs and “work” for them. It should provide added value and make it worth their time. I also learned that in a few instances, I missed the obvious and a feature that I thought would be beneficial may be become distracting if it’s used throughout the entire video. All the user feedback that I received overwhelmingly indicated and verified that the two support options that I prototyped and tested are definitely resources that students see as valuable, supportive, time worthy, and assists them in their understanding of the materials and topics within the course.

Going forward, I will continue to create short, instructional videos for topics that students struggle with and include a video link within each module of my 8-week online web programming course. To accompany the instructional video, a PDF will be provided that highlights the specific HTML tags and web page elements that are discussed within the video along with any additional topic-related references to additional online resources. I will also be posting the Collabedit tutorial PDF document within my online classroom and will make a formal announcement to my students and encourage the use of Collabedit for real-time collaboration with classmates and instructor assistance in an online environment. I want to create a very positive and supportive online learning experience for my students with the goal of increasing student success, confidence, motivation, and student retention in online courses.

PoP – Test Mode

Time to test!

My problem of practice continues to focus on exploring ways to improve student retention in online courses. During three of the prior modes in the design thinking process (empathize, define, and ideate), I have discovered a number of obstacles that may inhibit student success and decrease retention for students enrolled in online courses. One of the obstacles is the lack of support and assistance in the online environment where the instructor is not standing in front of the class lecturing, demonstrating, and available to immediately answer questions and provide assistance within a synchronous setting. For the prototype mode, I created two prototypes that focused on providing additional support and resources to students when it comes to learning weekly and fundamental course topics and offering assistance – the prototypes were tested by online students currently enrolled in my web programming course.

Approximately half of the students in my course participated in the testing process and provided feedback on the video that I created which walked students through creating their first web page (based on a hands-on exercise within chapter 2 of their course textbook). I participated in an online collaborative coding session with one of my students where we tested the use of an online tool called Collabedit for collaboration between students and students and their instructor. Prior to the collaborative coding session, I supplied the student with a tutorial PDF document that I created for accessing and using Collabedit and its features so the student had a basic understanding of how to join and participate in the collaboration session.

The purpose of testing the prototypes that I created was to determine if these solutions would be beneficial to online students and if the solutions would provide the students with additional support options that could be used for learning and collaborating with both the instructor and their classmates within the online classroom. If students know that assistance and additional support is available, they may be more likely to stay enrolled in the course, successfully learn the course material, and complete the course with the appropriate knowledge to continue on with their education with confidence and assurance.

Ideas, ideas, & ideas!

Problem of Practice: Ideate Mode

It was all about brainstorming and incubation during the ideate mode. Engaging in ideation over the past few weeks, with the focus being on increasing student retention in online courses (my PoP),  involved multiple activities which included gathering ideas during a brainstorming session, letting the ideas incubate, maintaining an incubation journal, and finally, reflecting on those ideas.

As a quick side note, I recently read the following statement (shown below) related to brainstorming in the latest edition of the Runner’s magazine. A perfect example of a “mind break”.

“When I run, I have my most unbridled thoughts. It’s a brainstorming session, as well as a time to process any issue that may be presenting in my work as food analyst and activist.” – Robyn O’Brien (Food industry analyst and author)

Part 1 of Ideation: Do a Brainstorm Session

Brainstorming notes during brainstorming sessions.

I conducted two formal brainstorming sessions and one informal session. By “informal” I mean that the session was an impromptu session based on the participants availability and I used Facebook and text messaging – I thought I would try something a bit different.

The formal brainstorming sessions included teachers both inside and outside of my discipline. The informal brainstorming session included students and parents.

The following ideas came out of the brainstorming sessions:

  • Online workshops that promote student success and preparedness
  • Create online tutorials demonstrating code, techniques, methods
  • Create class assignments that focus more on real-life scenarios
  • Have the students build something “real”
  • Prerecorded online webinars that students can watch on their own time
    • Post them to the online classroom
  • Create hands-on assignments that include demonstrations
  • Take an abstract concept and apply it to something “real”
    • e.g., use robots to teach coding concepts and coding languages
  • Make use of online simulators
    • Write a program and then deploy it through the use of simulator to bring a real-life experience to an assignment/project
  • Run an extra credit contest and award extra points for the best designed website, best content, best graphics, etc. – motivate and engage students
  • “Early turn in” – offer the students extra points for submitting their work prior to the deadline in order to motivate and engage students
    • e.g., if work is due on Sunday,  you get 5 extra points for turning it on Friday and 3 extra points for turning it on Saturday
  • Assign a group website final project rather each individual working on their own separate website
    • Promotes collaboration
  • For help discussion forums, offer extra points to students based on how many students they help
    • Gets students involved with helping each other, teaching and learning
  • Job shadowing different IT departments at the college that help students gain experience
    • Run a raffle of all interested students, choose 2-3 students to job shadow during the semester, maybe 5-10 per week, rotate in different IT areas: web development, networking, help desk, etc.
    • Motivates students
    • Engages students
    • Builds their resume
  • Assign a “pre-quiz” worth 10 points that asks questions to find out if students are properly prepared to take an online course
I also noted interesting topics of conversation during the brainstorming session that involved students and parents (read below). 
  1. Interesting activities and good teacher engagement. I’d imagine a lot of people’s biggest problem with online classes is communication to their teacher, or motivation to actually get the work done. Probably reply time would be most important. And i usually use email with my teachers.
  2. My take on being successful in online vs traditional classrooms has to do with having many overlapping variables that drive success. The method of engaging in the class might appear radically different, but I don’t see it that way. A student might be “shy” in a classroom and not raise their hand or they could be “shy” in an online forum and feel they have nothing to add. Seriously, in a case like that, it’s part of the instructor’s job to create an environment that encourages engagement and a clear communication of the expectations of the course. I could think of dozens of others, but I still go back to the same point…NOT that many differences. If a student has an unsupportive environment, the form of receiving the coursework doesn’t matter. Part of it is being able to navigate life and having the time for the prep and necessary work away from the classroom to succeed. Not to sound overly-pessimistic, but it’s never a one-way street for the student nor the instructor. There is give and take and responsibilities on both sides.
  3. I tend to drop a online class when it seems to be so overwhelming. Under week 1 there is 200 items that need to be completed by next week. To me it feels like I have no support in this class. I have yet to have an online class with a lot of instructor involvement. One class after the instructor “introduced” himself, we didn’t hear from again. I was really worried thinking maybe he died and no one knew. I even emailed him with no response. I dropped the class. Everyone taking an online class knows there is a lot of work involved but the professors have to realize we have other classes, work full time and have a family. I personally rather take a brick and mortar class because i feel i get more out of it because of the professor interaction.
  4. I’ve never taken an online course, though I know people who swear by them. Part of my problem would definitely be the family responsibilities – hard to sit at a computer with laundry baskets around you waiting to be folded and dinner waiting to be made. My problem is that I don’t feel I’m “tech savvy” enough to make it through an online course, like I wouldn’t even know where to begin, so that’s why I would shy away from even trying one. But if I knew it was for every level of computer knowledge, even one like mine, I might be more willing to at least try it. One of my biggest questions would be how do I contact the instructor if I needed help with anything, and would they be willing to help me?

Part 2 of Ideation: Keep an Incubation Journal

Following the brainstorming sessions, I maintained a journal where I kept personal notes, a record of my thoughts, and any new interesting ideas that came to mind. I even had one of the brainstorming participants come up to me the next day and give me a few more ideas which I included in my journal. In order to easily maintain the journal and to be able to access it anywhere online across devices, I used Evernote. My journal kept getting more and more detailed as the days passed. I also included a “PoP Notes” section in Evernote to keep track of additional notes not related to the brainstorming session but notes that are important to my PoP.

A slideshow of my incubation journal entries in Evernote:

This slideshow requires JavaScript.

The PoP Notes that I’m maintaining for my own information as an additional part of my incubation journal:

Part 3 of Ideation: Reflect on the Ideas

I have learned a great deal through the process of ideation, brainstorming, and incubation. One of the most difficult parts of the process for me was letting the ideas incubate – it seemed that this part was never-ending for me. I’m not sure if it’s meant to be like that but I found myself constantly thinking and coming up with ideas after brainstorming, so much that I kept thinking that I was losing my focus. I don’t necessarily feel that that’s a bad thing that ideas keep flowing, but given the scope of the project, it started making me anxious simply because there is a deadline associated with the project.

I found that working through one of the Da Vinci processes based on the chapter reading for this topic, was very interesting. It allowed be to try something different and use a technique/process that I had never tried before – it brought about a new way of looking at an existing problem. I honestly did not think the process would generate any ideas but in the end I was amazed at the results! I also thought the incubation journal rocked! It proved to be extremely valuable not only in documenting my thoughts, but as a way to help me visually “see”  issues/ideas/solutions.

In my incubation journal, I summarized my thoughts regarding the ideas and areas that I would like to pursue and keep (shown below).

I am strongly considering the following idea(s) to implement within the next course sections:
  • Create videos demonstrating code, techniques, and methods associated with weekly topics
    • Supports learning and student support
  • Use online tools with students to review source code in a collaborative environment
    • collabedit, CodePen (online, free, and works through the web browser – no need to download any software)
    • Supports learning, collaboration, and student support
    • Connecting with students
  • Adding links within my online classrooms
    • student resources, job announcements, internship opportunities, and workshops

As final thought, I think the ideas that I listed above are all doable. Adding the resources to my online classrooms can be implemented immediately and will require a minimal time commitment. Creating the videos for instruction will involve more of a time commitment, but if I initially focus on the areas of the greatest need, it will be the most beneficial choice for the students and will impact them immediately. Using online collaboration tools for supporting students is something that will occur more so on an individual basis as students need assistance.

Onward to prototyping and testing! The journey continues!

PoP Define Mode: Part 2

Problem Statement

As an online web programming teacher, my PoP (problem of practice) focus will be on examining student retention in online courses and to identify the various obstacles that may inhibit success. I am using the Introduction to Web Programming online course that I teach as the focal point and the students within that course, as my target audience.

This semester I am teaching three sections of the introduction course with a total of approximately 63 students. The student demographics include ages ranging from high school juniors to adults in their fifties/sixties, a variety of different ethnic groups, and the male students greatly out number the female students in all three sections (19 females to 44 males respectively) – this is very common in the IT related courses at the college and the industry itself. The student population includes many of the following variables, usually with multiple variables applied to a single student: taking a full-time or part-time course load, working full-time or part-time, holding one or multiple jobs, family responsibilities (children), caretaker for parents, grandparents or other family member(s), Federal Aid recipients, various income levels, varying GPA scores (we are an open enrollment college), commuter college (no on campus housing), various ethnic backgrounds, language barriers and special needs requirements.

The number of possible variables that can play a role in student retention in online courses makes it difficult to pinpoint an exact single cause as to why some students are successful and some students are not successful in an online environment. I believe that being adequately prepared, possessing strong time management and organizational skills, goal setting and having a strong support system both at home and at the college play an important role in success. The direction I want to pursue with my PoP is to examine the different obstacles students are facing in online courses and focusing on those obstacles that I as a teacher, can assist with overcoming. Areas that I want to examine are: providing online support with course content (e.g., how-to videos, help discussion forums, etc.), providing online office hours (examine online video conferencing tools), develop an online support community in each course where students can share with each other (discussion forums), encourage study groups on campus, and look into possible course offerings and/or workshops that may provide students with information on study skills and time management techniques. My goal is help students feel prepared for their online course and to make the online classroom environment portray a feeling of community and support so that everyone can be successful and succeed in achieving their goals.

Problem of Practice – CEP 817

Online Learning

Currently, I’m a member of the LMS/Online Advisory Committee at the college and often the topic of student retention comes up in conversation. Student retention is not only important in courses that are taught on campus in a traditional classroom, but also in the online environment as well. It seems that retention in online courses is an ongoing issue and we have yet to find a single viable solution. Is it the technology, the instructor, the teaching method, the student, the course? The list of questions goes on.

For some time now, I have been interested in exploring different ways to improve student retention in online courses so I thought this would be a great opportunity and an ideal problem of practice to explore. The Web Programming curriculum at the college where I teach is taught entirely online but I am going to focus on a single 8-week “Introduction to Web Programming” course (ITWP 1000) within the curriculum. This is a core introductory course within the program and is also a course that is required in many of the other IT programs at the college as well. I typically run 4 sections of the ITWP 1000 course in the Fall and Winter semesters at the college with full enrollment (26-28 students).

I have noticed that within the first week of the course; typically anywhere from 1-5 students drop the course and half way through, additional students may drop or withdraw from the course. For example, a course that begins with 26 enrolled students may end with only 18 and out of the 18, only 15 may have earned a passing grade. Unfortunately, I have no way of knowing the reason a student may drop or withdraw from a course unless the student chooses to share it with me and on occasion some students do. Time commitment, health issues, a family emergency, and finances are some of the reasons that students choose to share. Those types of issues are part of life of which we can’t control, but what about other issues such as their comfort level with technology and computers, reliable Internet access, course workload, lack of confidence, lack of motivation, or not understanding the course material? All of these are obstacles that may inhibit success.

Target Audience and Preliminary Thoughts

The students are going to be the primary target audience during my exploration into improving student retention in online courses (ITWP 1000) but I am very interested in the thoughts and opinions of my colleagues as well so they will play an additional role in my research. Some preliminary ideas of exploration (there may be more) include examining the following:

  • The use of “how-to” videos or video instruction within the course at crucial points.
  • The idea of hosting a “meet and greet the instructor” session online during the first few days of the online class.
  • The idea of hosting online sessions for some of the discussion forum activities for the course.
  • The use of online “tutoring” sessions and what technology could be used to accomplish it.
  • Establishing online office hours specifically for online students.
  • The idea of hosting 2-3 half hour (approximately) on campus sessions during the first few days of class where students can drop in to watch me demonstrate how to set up a connection to the web server and then attempt it themselves with me standing by if they need assistance.
  • The use of video conferencing software for student interaction.
  • Establishing common obstacles that may get in the way of success.

I’m very excited to dive into and explore the issues and obstacles surrounding the student retention rate in the ITWP 1000 course that I teach and the possible solution(s) to increasing student retention in the course.

References:

SpliteShire. (2014, August). [Computer/Communication] [Image]. Retrieved from https://pixabay.com/en/macbook-notebook-apple-device-407127/

Reflecting on the “Networked” approach

This was an amazing experience! I learned how to create a basic iOS app using Xcode and Swift and deployed it to my phone. I have been wanting to do this for over a year now. There were a few reasons for not doing this sooner but two of the major reasons were time (the lack of) and fear of the unknown. I conquered both! And in the process, I successfully created two apps and they now both reside on my phone. Success! Whew!

There weren’t a shortage of online resources, but it did take some time locating the most relevant, quality resources and those that would best suit my needs based on my experience level (beginner). Before starting the project, I spent approximately 2-3 hours searching the web for the exact type of resources that I wanted to use – bookmarking and documenting each of them as I went along.

I originally had planned on creating a single mobile app (but actually created two). I figured it was an attainable goal that I could accomplish within 2 weeks. I used an Apple tutorial located on the Apple Developer website to create my first app. The online tutorial was very detailed and included step-by-step instructions along with screen shots every step of the way. It was a perfect starting point! As I progressed through the tutorial and gained some confidence, I started to tweak things. Quickly I learned that that was not the best idea and parts of the app were not working correctly. I had to repeat several sections of the tutorial multiple times to fix the “bugs”. The positive side of that was that I learned a lot through repetition and having a resource to always refer back to was invaluable.

For the second app, I used only YouTube videos. Being able to watch a coding demonstration and hearing it explained, created a slightly different learning experience than simply reading instructions and viewing screen shots. The YouTube videos were more dynamic, but I learned equally from using both methods. I didn’t realize how much I had learned from the online Apple tutorial until I started watching the YouTube lessons and realized that I actually had a basic understanding of what was being demonstrated. Cue the aha moment! On the flip side, I also learned new concepts and techniques from the YouTube videos that I didn’t get from reading the online tutorial. The bottom line was that each method complimented the other.

I will continue using the “networked” approach for learning. I think that having resources that you can reference repeatedly during the learning process, is very important and helps to reinforce learning. Not only can this approach be used for acquiring a new skill set, it can also be used to update or refresh previous skills as well as teach, share, and collaborate within an online environment.

I currently encourage my students to incorporate and use this approach for their learning and most definitely will continue to do so. I see it as a necessary supplemental approach to traditional learning both online and in the classroom. In the field of technology in which I teach, it moves very quickly and in order to keep current with what’s needed in this industry and the skills and knowledge that it demands, learning new skills and refreshing old skills, never ceases. A “networked” approach supports the industry’s needs. However, I will continue to stress the importance of time management, motivation, self-learning, and patience that this method requires.