The culmination of the work completed in CEP 817 entailing the use of the Stanford d.school Design Thinking Process is included within my Problem of Practice Final Design Report for Increasing Student Retention in Online Courses for review: Final Design Report – CEP 817. It has been a fantastic journey!
A journey through the design thinking process
The last mode on the amazing journey through the design thinking process is the test mode. It’s an opportunity to test prototyped solutions and gain valuable feedback from the users of your product and their interaction with your solution(s).
The Context & Users
As I worked through each of the modes in the design thinking process up to the final test mode, I have discovered that there are many variables that may affect student retention in online courses. As I mentioned in an earlier blog post, the student population at the college where I teach is very diverse and includes many of the following variables, usually with multiple variables applied to a single student: taking a full-time or part-time course load, working full-time or part-time, holding one or multiple jobs, family responsibilities (children), caretaker for parents, grandparents or other family member(s), Federal Aid recipients, various income levels, varying GPA scores (we are an open enrollment college), commuter college (no on campus housing), various ethnic backgrounds, language barriers and special needs requirements. Given this information, the user group for the test mode for my problem of practice (increasing student retention in online courses) focuses on students within my online basic web programming course. This is an introductory course and the first course within the web programming degree program at the college.
Prototypes Tested & Why
Two different prototypes were tested during the test mode. The first prototype included a video that walked students through creating their first web page. The second prototype included the process of using an online collaborative coding tool called Collabedit that allows for real-time collaboration among multiple users during a single session. A tutorial PDF document was created that explained how to access and use Collabedit and its features so the user has a basic understanding of how to join and participate in a collaboration session prior to joining the session.
Within both prototype solutions, I was testing the effectiveness of the tool, ease of use, ease of access, and its functionality to determine if these solutions would be beneficial to online students and if the solutions would provide the students with additional support options that could be used for learning and collaborating with both the instructor and their classmates within their online classroom. I also wanted to determine if these solutions provided value and support to the user and assisted them in successfully learning the course material.
The Test Protocol(s) & Design
The testing period for both prototypes ran from March 27 – April 2 respectively. I designed my tests to be available over a week long period in order to give users enough time to participate. I also designed the tests in a way that they would be available in an online setting to simulate a true user experience since the audience includes users participating in an online course. The design intentions were to make access to the prototypes and providing feedback as easy, convenient, and user-friendly as possible.
The test protocol for the video included the following:
- A video was created that walks through the process of creating a basic web page using HTML. This video uses information from Chapter 2 with the course textbook as a reference point.
- The video was uploaded to YouTube and initially set to “Unlisted”, students were provided with the URL to access the video.
- An extra credit assignment was created within my online web programming classroom that offered students points for participation in reviewing and providing feedback based on the video walk-through.
- The assignment link within my online classroom provided the user with some basic background information, instructions for accessing the video, and a series of some general questions regarding the video.
- The users were given a deadline for submitting their feedback (one week).
- Feedback was submitted through a drop box within the online classroom and assembled and documented within a Word document at the end of the submission period.
The test protocol for the online collaborative coding session process using Collabedit included the following:
- A multipage tutorial was created that provides the user with basic information on how to access, use, join, and initiate an online collaborative coding session using Collabedit.
- The tutorial provides instructions and screen shots documented the process.
- Prior to joining a collaborative session, the user is supplied with the Collabedit tutorial document so he/she can prepare for the session.
- I initiated the Collabedit session and invited the participant via email with some basic instructions and a URL that the user can copy/paste into their web browser to join the session.
- The Collabedit sessions contained the HTML code the student participant needed assistance with and was present in the browser window upon joining the session.
- The user was instructed to modify the HTML code within the session window and engage in conversation via the chat window during the session while we collaborated in real-time. The session lasted for approximately 45 minutes.
- Feedback was submitted through email following the session and was based on a series of questions regarding the use of Collabedit.
- Feedback was assembled and document within a Word document.
I received very valuable and useful user feedback. I had approximately half of my online basic web programming course students participate and provide feedback regarding the video walk-through that I created. I was able to conduct an online collaborative session using Collabedit with one of my web programming students. A few other students that were interested in testing Collabedit contacted me but due to scheduling conflicts, they were not able to participate.
Some feedback regarding what users liked included:
- I love the fact that the video is broken down line by line explaining in detail the function of each tag.
- I liked that in the end a resource was provided that would help you if you had any more questions on different topics.
- I do find videos very useful to help me understand concepts within programming I’m finding difficult.
- I enjoyed this video! Videos are most useful when concise and focused on one topic. I think that makes it much easier to refer back to them when necessary.
- I’m a visual and kinetic learner so reading doesn’t help as much as hands on or visual instruction.
- This youtube video takes it a step further and shows you how to create elements within the instructions.
- I like tips and tricks, for example when you mention white space in HTML code for debugging.
- I liked how you went step by step and introduced the basics of the class.
- The language used was easily understood and presented well.
- I absolutely would watch videos like this where a topic is challenging.
- While using this tool (Collabedit) I thought it was very useful and nice to converse with you while figuring out where my errors were.
- I liked the fact that we could, literally converse back and forth and I knew where you were talking when trying to help me, it made this learning experience easier and was very beneficial.
Some feedback for improvement included:
- I would prefer that videos be entirely optional and cover topics that are historically harder to comprehend and not on every single topic we cover.
- I do feel that this video was a little long nut it is packed with information. My suggestion may be to separate it into many parts maybe, like mini videos that way they are more specific to one topic.
- After watching the whole video to the end my one complaint is the text front is a little small.
- I feel as if you could zoom or make the text font larger it would be more user-friendly and helpful
- The use of the yellow cursor bubble was distracting and caused a blur on the parts of the code when you hovered over them. A different pointer might work better.
- Maybe add a definition of terms of the code tags that you used at the end just as a reference.
- I think the length of the video is acceptable, but so is shorter. As I said, I find these types of videos useful, but they are most useful when concise and focused on one topic.
Reflecting on the Process
After wrapping up the testing mode and reviewing the user feedback, I learned a great deal from the process. The first and foremost lesson I learned, is that it’s absolutely necessary to get feedback from your users – the individuals that are actually using and interacting with what you created. It has to fulfill their needs and “work” for them. It should provide added value and make it worth their time. I also learned that in a few instances, I missed the obvious and a feature that I thought would be beneficial may be become distracting if it’s used throughout the entire video. All the user feedback that I received overwhelmingly indicated and verified that the two support options that I prototyped and tested are definitely resources that students see as valuable, supportive, time worthy, and assists them in their understanding of the materials and topics within the course.
Going forward, I will continue to create short, instructional videos for topics that students struggle with and include a video link within each module of my 8-week online web programming course. To accompany the instructional video, a PDF will be provided that highlights the specific HTML tags and web page elements that are discussed within the video along with any additional topic-related references to additional online resources. I will also be posting the Collabedit tutorial PDF document within my online classroom and will make a formal announcement to my students and encourage the use of Collabedit for real-time collaboration with classmates and instructor assistance in an online environment. I want to create a very positive and supportive online learning experience for my students with the goal of increasing student success, confidence, motivation, and student retention in online courses.
Time to test!
My problem of practice continues to focus on exploring ways to improve student retention in online courses. During three of the prior modes in the design thinking process (empathize, define, and ideate), I have discovered a number of obstacles that may inhibit student success and decrease retention for students enrolled in online courses. One of the obstacles is the lack of support and assistance in the online environment where the instructor is not standing in front of the class lecturing, demonstrating, and available to immediately answer questions and provide assistance within a synchronous setting. For the prototype mode, I created two prototypes that focused on providing additional support and resources to students when it comes to learning weekly and fundamental course topics and offering assistance – the prototypes were tested by online students currently enrolled in my web programming course.
Approximately half of the students in my course participated in the testing process and provided feedback on the video that I created which walked students through creating their first web page (based on a hands-on exercise within chapter 2 of their course textbook). I participated in an online collaborative coding session with one of my students where we tested the use of an online tool called Collabedit for collaboration between students and students and their instructor. Prior to the collaborative coding session, I supplied the student with a tutorial PDF document that I created for accessing and using Collabedit and its features so the student had a basic understanding of how to join and participate in the collaboration session.
The purpose of testing the prototypes that I created was to determine if these solutions would be beneficial to online students and if the solutions would provide the students with additional support options that could be used for learning and collaborating with both the instructor and their classmates within the online classroom. If students know that assistance and additional support is available, they may be more likely to stay enrolled in the course, successfully learn the course material, and complete the course with the appropriate knowledge to continue on with their education with confidence and assurance.
Problem of Practice: Ideate Mode
It was all about brainstorming and incubation during the ideate mode. Engaging in ideation over the past few weeks, with the focus being on increasing student retention in online courses (my PoP), involved multiple activities which included gathering ideas during a brainstorming session, letting the ideas incubate, maintaining an incubation journal, and finally, reflecting on those ideas.
As a quick side note, I recently read the following statement (shown below) related to brainstorming in the latest edition of the Runner’s magazine. A perfect example of a “mind break”.
“When I run, I have my most unbridled thoughts. It’s a brainstorming session, as well as a time to process any issue that may be presenting in my work as food analyst and activist.” – Robyn O’Brien (Food industry analyst and author)
Part 1 of Ideation: Do a Brainstorm Session
I conducted two formal brainstorming sessions and one informal session. By “informal” I mean that the session was an impromptu session based on the participants availability and I used Facebook and text messaging – I thought I would try something a bit different.
The formal brainstorming sessions included teachers both inside and outside of my discipline. The informal brainstorming session included students and parents.
The following ideas came out of the brainstorming sessions:
- Online workshops that promote student success and preparedness
- Create online tutorials demonstrating code, techniques, methods
- Create class assignments that focus more on real-life scenarios
- Have the students build something “real”
- Prerecorded online webinars that students can watch on their own time
- Post them to the online classroom
- Create hands-on assignments that include demonstrations
- Take an abstract concept and apply it to something “real”
- e.g., use robots to teach coding concepts and coding languages
- Make use of online simulators
- Write a program and then deploy it through the use of simulator to bring a real-life experience to an assignment/project
- Run an extra credit contest and award extra points for the best designed website, best content, best graphics, etc. – motivate and engage students
- “Early turn in” – offer the students extra points for submitting their work prior to the deadline in order to motivate and engage students
- e.g., if work is due on Sunday, you get 5 extra points for turning it on Friday and 3 extra points for turning it on Saturday
- Assign a group website final project rather each individual working on their own separate website
- Promotes collaboration
- For help discussion forums, offer extra points to students based on how many students they help
- Gets students involved with helping each other, teaching and learning
- Job shadowing different IT departments at the college that help students gain experience
- Run a raffle of all interested students, choose 2-3 students to job shadow during the semester, maybe 5-10 per week, rotate in different IT areas: web development, networking, help desk, etc.
- Motivates students
- Engages students
- Builds their resume
- Assign a “pre-quiz” worth 10 points that asks questions to find out if students are properly prepared to take an online course
- Interesting activities and good teacher engagement. I’d imagine a lot of people’s biggest problem with online classes is communication to their teacher, or motivation to actually get the work done. Probably reply time would be most important. And i usually use email with my teachers.
- My take on being successful in online vs traditional classrooms has to do with having many overlapping variables that drive success. The method of engaging in the class might appear radically different, but I don’t see it that way. A student might be “shy” in a classroom and not raise their hand or they could be “shy” in an online forum and feel they have nothing to add. Seriously, in a case like that, it’s part of the instructor’s job to create an environment that encourages engagement and a clear communication of the expectations of the course. I could think of dozens of others, but I still go back to the same point…NOT that many differences. If a student has an unsupportive environment, the form of receiving the coursework doesn’t matter. Part of it is being able to navigate life and having the time for the prep and necessary work away from the classroom to succeed. Not to sound overly-pessimistic, but it’s never a one-way street for the student nor the instructor. There is give and take and responsibilities on both sides.
- I tend to drop a online class when it seems to be so overwhelming. Under week 1 there is 200 items that need to be completed by next week. To me it feels like I have no support in this class. I have yet to have an online class with a lot of instructor involvement. One class after the instructor “introduced” himself, we didn’t hear from again. I was really worried thinking maybe he died and no one knew. I even emailed him with no response. I dropped the class. Everyone taking an online class knows there is a lot of work involved but the professors have to realize we have other classes, work full time and have a family. I personally rather take a brick and mortar class because i feel i get more out of it because of the professor interaction.
- I’ve never taken an online course, though I know people who swear by them. Part of my problem would definitely be the family responsibilities – hard to sit at a computer with laundry baskets around you waiting to be folded and dinner waiting to be made. My problem is that I don’t feel I’m “tech savvy” enough to make it through an online course, like I wouldn’t even know where to begin, so that’s why I would shy away from even trying one. But if I knew it was for every level of computer knowledge, even one like mine, I might be more willing to at least try it. One of my biggest questions would be how do I contact the instructor if I needed help with anything, and would they be willing to help me?
Part 2 of Ideation: Keep an Incubation Journal
Following the brainstorming sessions, I maintained a journal where I kept personal notes, a record of my thoughts, and any new interesting ideas that came to mind. I even had one of the brainstorming participants come up to me the next day and give me a few more ideas which I included in my journal. In order to easily maintain the journal and to be able to access it anywhere online across devices, I used Evernote. My journal kept getting more and more detailed as the days passed. I also included a “PoP Notes” section in Evernote to keep track of additional notes not related to the brainstorming session but notes that are important to my PoP.
A slideshow of my incubation journal entries in Evernote:
The PoP Notes that I’m maintaining for my own information as an additional part of my incubation journal:
Part 3 of Ideation: Reflect on the Ideas
I have learned a great deal through the process of ideation, brainstorming, and incubation. One of the most difficult parts of the process for me was letting the ideas incubate – it seemed that this part was never-ending for me. I’m not sure if it’s meant to be like that but I found myself constantly thinking and coming up with ideas after brainstorming, so much that I kept thinking that I was losing my focus. I don’t necessarily feel that that’s a bad thing that ideas keep flowing, but given the scope of the project, it started making me anxious simply because there is a deadline associated with the project.
I found that working through one of the Da Vinci processes based on the chapter reading for this topic, was very interesting. It allowed be to try something different and use a technique/process that I had never tried before – it brought about a new way of looking at an existing problem. I honestly did not think the process would generate any ideas but in the end I was amazed at the results! I also thought the incubation journal rocked! It proved to be extremely valuable not only in documenting my thoughts, but as a way to help me visually “see” issues/ideas/solutions.
In my incubation journal, I summarized my thoughts regarding the ideas and areas that I would like to pursue and keep (shown below).
- Create videos demonstrating code, techniques, and methods associated with weekly topics
- Supports learning and student support
- Use online tools with students to review source code in a collaborative environment
- Adding links within my online classrooms
- student resources, job announcements, internship opportunities, and workshops
As final thought, I think the ideas that I listed above are all doable. Adding the resources to my online classrooms can be implemented immediately and will require a minimal time commitment. Creating the videos for instruction will involve more of a time commitment, but if I initially focus on the areas of the greatest need, it will be the most beneficial choice for the students and will impact them immediately. Using online collaboration tools for supporting students is something that will occur more so on an individual basis as students need assistance.
Onward to prototyping and testing! The journey continues!
As an online web programming teacher, my PoP (problem of practice) focus will be on examining student retention in online courses and to identify the various obstacles that may inhibit success. I am using the Introduction to Web Programming online course that I teach as the focal point and the students within that course, as my target audience.
This semester I am teaching three sections of the introduction course with a total of approximately 63 students. The student demographics include ages ranging from high school juniors to adults in their fifties/sixties, a variety of different ethnic groups, and the male students greatly out number the female students in all three sections (19 females to 44 males respectively) – this is very common in the IT related courses at the college and the industry itself. The student population includes many of the following variables, usually with multiple variables applied to a single student: taking a full-time or part-time course load, working full-time or part-time, holding one or multiple jobs, family responsibilities (children), caretaker for parents, grandparents or other family member(s), Federal Aid recipients, various income levels, varying GPA scores (we are an open enrollment college), commuter college (no on campus housing), various ethnic backgrounds, language barriers and special needs requirements.
The number of possible variables that can play a role in student retention in online courses makes it difficult to pinpoint an exact single cause as to why some students are successful and some students are not successful in an online environment. I believe that being adequately prepared, possessing strong time management and organizational skills, goal setting and having a strong support system both at home and at the college play an important role in success. The direction I want to pursue with my PoP is to examine the different obstacles students are facing in online courses and focusing on those obstacles that I as a teacher, can assist with overcoming. Areas that I want to examine are: providing online support with course content (e.g., how-to videos, help discussion forums, etc.), providing online office hours (examine online video conferencing tools), develop an online support community in each course where students can share with each other (discussion forums), encourage study groups on campus, and look into possible course offerings and/or workshops that may provide students with information on study skills and time management techniques. My goal is help students feel prepared for their online course and to make the online classroom environment portray a feeling of community and support so that everyone can be successful and succeed in achieving their goals.
Wikipedia describes the Socratic method as “form of cooperative argumentative dialogue between individuals, based on asking and answering questions to stimulate critical thinking and to draw out ideas and underlying presumptions.” (“Socratic Method,” 2016, “Definition,” para. 1).
In Chapter 2 of “A More Beautiful Question“, William Deresiewicz and Sebastian Thrun make mention of Socratic teaching. In relation to the online world, Thrun hopes that this method of teaching makes a comeback. The topic of Socratic teaching sparked my interest not only because it poses the question of being able to “teach ourselves to question” but also because it references the online environment which makes up 100% of both my personal learning education and professional teaching career. Food for thought: Is it easier for online learners to question simply because of the online environment? Are college institutions becoming more technocratic when it comes to education? Can we teach ourselves to be a better questioner? Is the Socratic method of teaching making a comeback?
Deresiewicz believes that the college education students are receiving now is more technocratic in nature. He states that, “They’re trained to develop expertise in a particular area–trained to solve problems that are particular to that area. It’s about jumping through hoops, and mastering what’s on the test” (Berger, 2014, p. 67). Instead, students don’t have the opportunity to stop and actually think about what and why they’re doing it let alone ask the “big questions about values and meaning and purpose” (p. 67).
According to Deresiewicz, professors that can inspire students to ask the big questions are rare. How did his favorite professor and mentor “spark inquiry”? Deresiewicz noted that his professor had “an ability to reframe things–to ask questions that got at something fundamental” (Berger, 2014, p. 68). Do teachers always have to the answers to every single question? How do learners perceive a teacher that doesn’t have the answer? Essentially, the Socratic method doesn’t always find the answers to every question, but rather may provoke additional questions for consideration, “…students find it really liberating t have a teacher say, ‘I don’t know the answer–so let’s figure this out together.'” states Deresiewicz (p. 68).
Sebastian Thrun, professor and founder and CEO of Udacity, is trying to bring the Socratic method of teaching as described by Deresiewicz, to online teaching. A Udacity course isn’t about broadcasting lectures while a learner absorbs information, it’s about interjecting “thoughtful questioning at critical junctures” (Berger, 2014, p. 68) so that it provokes the learner to really think about what they’re learning and to ask the big questions that can spark inquiry. Is questioning easier in an online environment? Thrun and one of his partners Irene Au, believe this to be the case and they believe that anonymity plays a role. Although Deresiewicz and Thrun don’t see eye-to-eye on the notion that questioning is easier online, they do both agree that questioning is critically important and necessary and support the Socratic method.
Since 100% of my everyday teaching takes place in an online environment, I am able to observe student interaction and questioning first hand within a higher education setting. A portion of the final grade is based upon discussion forum interaction and asking/answering questions related to the week’s topic. There is also a “help” forum that I highly encourage students to use for asking questions as well as for collaborating and assisting others. I participate within the weekly discussions and respond to students by asking questions about their questions to get them to think outside of the box and “spark inquiry”. Often I find that when I question, it has an avalanche effect and students follow my lead. When this occurs, amazing discussions take place! Whether it’s obvious to the students or not, they are essentially teaching themselves to be better questioner.
I also find that students are more open to asking questions in an online environment, giving credence to the statement made by Thrun and Au that anonymity plays a role. Comparing the discussion forums in my online classrooms to my traditional classrooms from previous years, the amount of complex, out-of-the-box type of questions, are greater in my online classrooms.
Is the Socratic method of teaching making a comeback and replacing more of the technocratic method of teaching in college institutions? If teachers are sparking inquiry within their students and students are critically thinking about what they are doing and the reason(s) why, and are encouraged to ask the “big questions”, then yes, the Socratic method of teaching is making a comeback. As a teacher will you always have the answer to every question? No, but that’s okay.
Altmann, G. (2014, October). [People] [Photograph]. Retrieved from https://pixabay.com/photo-479670/
Berger, W. (2014). A More Beautiful Question: The Power of Inquiry to Spark Breathrough Ideas. New York City, NY: Bloomsbury USA.
Gaba, E. (2005, July). [Portrait of Socrates. Marble, Roman artwork (1st century), perhaps a copy of a lost bronze statue made by Lysippos.] [Photograph]. Retrieved from https://commons.wikimedia.org/wiki/File%3ASocrates_Louvre.jpg
Socratic Method. (n.d.). In Wikipedia. Retrieved September, 8, 2016, from https://en.wikipedia.org/wiki/Socratic_method#cite_note-0