This project was for the course E-Learning Design Principles, taught by Prof. Ken Koedinger at CMU. The objective of this project was to transform the original in-class teaching into online teaching. Our tasks were to redesign the instructional modules and help users learn more efficiently and effectively through e-learning.
My role: Learning Experience Designer
We designed an 1-hour online course which consists of three parts - Pre-test, instruction, and Post-test. For the topic, our group focused on traditional Chinese Music culture. The course has five modules:
Music concepts that can describe Music
Chinese music culture
Comparison between Chinese and Western Music
Chinese music instruments
Chinese music and Kongfu
Who is the course targeting at?
We expect the learner of this course to be interested in music. They are not required to have any prior knowledge with music concepts or even Chinese music, since we will be introducing everything from the basic level.
What are we teaching?
We are planning to teach both concepts and skills regarding music concepts and traditional Chinese music culture.
The music concepts include the key terminologies in the area of music: timbre, pitch, icon/index. These words can be used to describe the sound or the feeling of a piece of music, which the learners will both understand these concepts.
Traditional Chinese music culture consists of the history context, unique Chinese music features with comparison to Western music, and Chinese music instruments. Learners will need to identify the timeline of the origin of Chinese music, understand the unique characteristics of Chinese music.
The skills that learners will perform are based on the conceptual knowledge they understand. The learners will apply the concepts they learned: For example, describe the timbre, pitch, icon of a particular music pieces; differentiate the Chinese music instrument in a music piece; explain the function of music in a Chinese Kongfu movie clip, etc.
Students will be enthusiastic about the Chinese music culture. They will be more willing to apply what they learned from the course to appreciate music.
Why are we teaching?
China has an incredibly long and influential music tradition. Over the centuries, musicians developed various styles of music, they invented and adopted many types of instruments. From this course, students will be able to develop the skills to appreciate Traditional Chinese Music, as well as understanding the history, musical instruments and characteristics of Traditional Chinese Music. We hope that this course could bring valuable music sense and knowledge about Traditional Chinese music to all the students.
The instructional design process followed the big picture as shown on the right: goals guide assessment task guide instruction.
Some instructional principles we applied are:
Multimedia, pretraining (summarize at the beginning of the course), personalization, contiguity (descriptive text is placed with pictures), and modality & redundancy (use audio).
We first conducted contextual inquiry with a professor at University of Pittsburgh teaching eastern Asian music to clarify the knowledge component we want to include (expert Cognitive Task Analysis). We revised our learning objectives after doing the interview to narrow the scope down and added more exploration.
After finishing the first version of course design, we did think aloud with students to identify the content where they struggled with. We then fixed several problems to make the question less vague by bolding the key words, scaffolding more and correcting the wording according to users' confusion in the cognitive task analysis. Following are some notes which guided our second round of iteration.
Observation: Student 1 had trouble matching the name of the instruments to the images, and he did not know how that instrument sounded simply from reading the text.
Iteration outcome: We reduced memory retrieval by requiring students to describe the sound of the instrument and how did he feel when hearing the sound instead of requiring them to remember the name. We assessed more about skills instead of facts. And in our second version, we also applied segmenting by first giving them an overview. Learners could get a grand understanding of different types of instruments before going into details.
Observation: Student 2 was a novice and knew little about music. He found the questions far more challenging for him.
Iteration outcome: We added the first part of instruction teaching about basic music concepts, pitch, timbre and icon, as an entry before going deeper to appreciating the music pieces. We also used interesting and common questions as a hook to get novices more engaged.
Observation: Student 3 said he would like to watch more video clips about the certain music genre.
Iteration outcome: We applied the multimedia principle in our second version by adding more short clips. We also added a goal which introduces the function of music in Kung Fu movies. We realized that observation and appreciation are more important when learning about music. Learners would feel boring and drop out the course instantly when there is only text.
We tested the efficacy of modality and redundancy on one of our learning objectives--differences between Chinese and Western music. The reason why we do this is because we would like to know if it is a good idea to use audio and no text for online learning when there is too much new terminology for students to learn.
For our control group, we applied the modality and redundancy principles by including the short video clip in our instruction. It is hard for students to control their learning at their own pace. And also, since we distributed the course among some non-native English speakers, watching a movie without subtitles may also add to their memory workload.
For our treatment group, we elaborately designed the material to highlight the new terminologies and show the differences between Chinese Music and Western Music with on-screen text. Students could control their learning process based on their own work rate. We did 18 user tests for control group and 12 for the treatment group.
We first calculated the average pre and post test score of the treatment and control group, and it is clearly illustrated that the average improvement of the treatment group, which eliminates the Modality and the Redundancy Rules, is much higher than the treatment group.
Since we only deleted the two principles in one part of our instruction, we want to know if that specific part causes the difference. So we calculated the results of the assessments that test the learning objective of differences between Chinese and Western Music.
From the results, we found that the Treatment group has a higher average improvement than the control group
The results are the same to our guesses that “Applying modality and redundancy principles when a lot of new terms are introduced” is not a good idea. It is important to present certain key words in written text, especially when the words are complex or the learners are not familiar with this area.
During the whole process of the project, I applied the knowledge I learned in the past E-learning class, including think aloud, DFA, refined learning objectives, e-learning design principles etc. to the real project, and got my understanding of these terminologies reinforced.
I also developed a good understanding of the backward design through the two rounds of iteration, with each round making questions and content more explicitly and gradually getting a better understanding of the big picture diagram. We started from goal setting in this project. Next time when I do the same kind of design iteration, I would like to try to start from actual instructional design, so that I could get a better sense of how the instruction develops and do not make the scope too large.