Engaging in Computer Science

The results from the three camps were categorized into themes from field note observations and survey results. The first result set explains the field note observations that were made during the camps, how each successive camp was improved based on these findings, and how those improvements affected future camps. The second result set details the effectiveness of the camps based on the survey (given before the 2019 camp and again after the week-long 2019 camp).

Qualitative Results: Field Note Observations and Improvements by Theme for 2018 Summer Camps

Theme 1: Adjusting for Initial Learning Curves

Many of the students attending the 2018 summer camps had never been exposed to programming or computer science basics. This created a bit of a learning curve during the first few activities since the students were introduced to new hardware, new software, programming, and computer science topics all at once. While some students could liken the Micro:Bit to a Raspberry Pi or Arduino, other students were not familiar with similar technology. For students without this prior exposure, the bridge between programming and actually implementing code on hardware was a completely new concept. Furthermore, since programming benefits from logical yet creative thinking, some students struggled with the initial code. When creativity outweighed logical reasoning, the student had many ideas on how to solve certain real-world problems (e.g., how to advance their robot), but struggled with implementing those solutions programmatically. When logical reasoning outweighed creativity, students were able to create new functionality for their devices, but many of these upgrades were not particularly useful in solving the given problems.

While the introductory activities in the 2018 summer camps taught students how to use the programming environment, it did little to sharpen their problem solving skills. Therefore, introductory Micro:Bit 101 activities were introduced in the summer 2019 camp to address the issues regarding baseline programming knowledge. Although the Micro:Bit website has online tutorials that work with students to create interesting projects, the Micro:Bit 101 activities designed by the researchers were more beneficial to the students than the online tutorials since they were built towards specific learning goals (called level-up guides). Specifically, the tutorials showed the students how to program the devices (e.g., showing which blocks go where in block programming). This approach gives a quick answer and can appease students, but stifles creativity, does not help the students build problem solving skills, and robs them of the opportunity to discuss their solution differences with others. Furthermore, while the tutorials helped students achieve functionality on their Micro:Bits, it did not help students understand the logistics of a given problem, internalize a solution, and synthesize that solution into a program. Since all of the answers were functionally the same, students were unable to use soft skills and communicate to others what made their code unique. Conversely, by providing students with the hand crafted Micro:Bit 101 exercises in the summer 2019 camp, students were able to build their problem solving skills (while still learning the programming environment and Micro:Bit basics).

Theme 2: Maintaining Attention and Engagement

The unplugged activities in the 2018 camps had less engagement than the programming activities. To raise this engagement in 2019, methods were used such as likening activities to real-world applications and keeping the students actively engaged. The students were grouped into two sections based on their self-reported experience level (either novice/intermediate or intermediate/advanced). Each group was instructed simultaneously by a different instructor in a separate room, and then the groups switched rooms. This raised student attentiveness by requiring students to focus for shorter periods of time, provided less competition for open discussions, and allowed the outreach staff to interact with them in more specialized conversations.

Furthermore, although the basic curriculum remained the same for the unplugged activities, it became more interactive in 2019 to stimulate engagement. For example, in 2018 camps, the "telephone game" was played in order to demonstrate how a message could degrade as it was passed from one person to the next, showing a lack of integrity (a principle of computer science information theory). This game was well received as many students had played the game before, and it was a familiar concept to them (pass the message from the front of the line to the back of the line). This introduced an important cybersecurity concept: What if a message is deliberately changed? While opening the door to conversations about a "man-in-the-middle" attack, the conversation was cut short. In 2019, this activity was lengthened and given more focus. Additionally, a lesson on Wi-Fi was developed to further solidify the concepts and relate them back to students' daily lives.

Theme 3: Providing Explicit Real-World Applications

The lessons and activities were all planned around common cybersecurity concepts and computer science challenges facing professionals today, with specific care taken towards relating the topics to student experiences. However, in the 2018 labs, while there were strong parallels between topics and students, the burden for drawing those parallels rested in the hands of the students. For example, one activity involved beacons that sent messages over radio waves and receivers that read those messages, mimicking current beacon technology (used largely for advertising and marketing). While this parallel existed, students were not able to draw this connection on their own. Without properly forming these connections to the real-world, the activity lost quality as students struggled to find ways to make sense of why these beacons degrade security. To solve this issue, in the 2019 labs, camp staff made specific connections to the students' previous knowledge. While the students were all familiar with robotics in general, they were unaware of many of the security concerns associated with this technology. Simply explaining the security vulnerabilities was not sufficient in grounding their knowledge in real-world applications. To fully relate robotics security, the staff explicitly drew comparisons to the real-world. Instead of likening the activity to robotics in general, the staff gave explanations on precisely how they are similar. For instance, the students quickly learned that it was not particularly difficult to control their friend's robot. This demonstrated a common attack vector in unmanned aerial vehicles (UAV) technology or self-driving cars. However, rather than trusting the students to draw these comparisons, specifics were given that directly mapped the activity to these types of issues. Questions were posed such as "How does this relate to self-driving cars?" and "What would happen if this was an actual car and not just a robot?"

Being explicit about the real-world implications of the labs solidified student learning as well as gave students a place to connect the lesson to their life. In other words, clearly explaining connections between lessons and the real-world gave students familiar topics with which to associate the new knowledge. Lastly, consider password strengths. Many people today do not understand why their password should be long and contain different types of characters. To these people, a five character password made up of lowercase letters should be hard to crack since there are 26⁵=11,881,376 total possible passwords and 11,881,376 is a big number. However, 11,881,376 is actually quite a small number considering how fast a computer can generate and test the passwords. Accordingly, when the lessons were grounded in real-world applications (like password strengths), not only was the concept clearly linked to students' existing knowledge, but the proper scaffolding was included to ensure retention.

Theme 4: Enhancing Incentives and Motivation

During all three camps, the students were able to track their progress using their individual badge, and the badges were fit with a ring of light emitting diodes (LEDs) that demonstrated their achievements. This gave the students incentive to work through the labs and stretch to achieve goals that might have seemed out of reach. The students were not required to complete the most difficult levels of the labs or create the best code; instead, they were motivated to solve problems to the best of their ability.

While the badges provided a low-level of motivation in the 2018 camps, the potential incentive was stifled by the inconsistency of the reward. In 2018, the camp staff leveled-up the badges (or increased the score) when they felt the students had demonstrated mastery over a new concept. In 2019, this system was standardized such that activities had specific checkpoints (or level-up guides) that, when reached, granted a student to level-up their badge. This created a direct expectation for levels and better motivated the students by giving them clear goals tied to specific incentives. Furthermore, in the 2018 labs, the students worked to level-up their badges throughout the week (no daily differentiation). However, in 2019, each day marked a new opportunity to level-up their badge. At the end of each day, the students' level was reset to zero meaning that each day they had the opportunity for success regardless of what happened previously in the camp. This better motivated students because they were not as discouraged by earlier mistakes.

Quantitative Results: Descriptive Statistics for the 2019 Summer Camp

The three research questions presented earlier form the basis for the 10 question pre/post survey (see Appendix A.1) that was administered to all the students. For the purpose of this study, the survey is segmented into four distinct areas of "Confidentiality, Integrity and Availability" recall (Q1) and applications (Q2–4), followed by online security (Q9), and finally camp interest (Q10).

Q1: CIA Recall

Student participants gained knowledge and confidence in being able to correctly identify the components of the CIA triad - namely confidentiality, integrity, and availability. The CIA triad is integral to cybersecurity and drives many of the decisions made in the field. The CIA triad was the first concept taught to the students and each lesson related back to these three core principles. Likewise, the first question on the survey was based on recalling this most basic cybersecurity lesson and asked: When talking about information security, what do the following letters represent, C, I, and A? The question asked the students to "select all that apply". When asked this question before the camp, almost half of the students selected that C, I, and A stood for Central Intelligence Agency and only eight students (33%) selected the correct answer (Confidentiality, Integrity, and Availability). After the camp, however, the wrong answer was only selected three times and the correct answer was selected 23 times (96%). These results can be seen graphically in Figure 1 - with only two students answering more than one answer (in both the pre-camp survey and the post-camp survey).

Q2–4: CIA Applications

Student participants identified real-world applications for each component of the CIA triad - with drastic improvement in understanding availability and integrity. Additionally, students generally understood applications of confidentiality and availability at higher rates than the concept of integrity. Questions 2–4 of the survey focus on the application of the CIA concepts. During the camp, each of these concepts were related to real-world issues. For example, confidentiality was linked to media streaming platforms. Many people share their passwords to media streaming platforms. Average users feel safe doing this because they trust the person with whom they are sharing the password. However, many people also use similar passwords across all of their accounts. This means that they are trusting that person to use that password appropriately, as well as trusting them to keep that password a secret at the risk of multiple accounts being compromised. This underscores a significant breach in confidentiality because, while people may share passwords with someone they trust, it becomes difficult to maintain privacy. Students understood this example since they use passwords on a daily basis and are familiar with media streaming platforms. To relate integrity to the real-world, the camp staff asked the students if they had social media accounts and what they would think if they logged on one day to find their account filled with random information. This demonstrated the importance of protecting data from being changed by unwelcome parties. This was also solidified through a lab activity involving the Micro:bots. The students were not told to pick a unique channel for their robot to receive control messages on. This meant that the students were sending control commands to other students' robots and the robots acted erratically. The students learned quickly that ensuring valid data was integral to controlling their robot, and in turn they learned how integrity impacts their day-to-day lives. To relate availability to the students, camp staff asked the students if they played games online, a common pastime for most students. Online games require a connection to a server. If the device (e.g., computer, gaming console) cannot connect to the Internet or the server cannot handle all of the traffic, the game becomes unavailable. Games that are continuously unavailable, or devices that consistently lack functionality, lose value over time and the companies providing them are less likely to be successful. While students did not seem to particularly care about the business aspect, they were able to identify that when services they want are not available, they go somewhere else (e.g., they pick a different game).

To determine the effectiveness of relating the CIA triad to real-world examples, the surveys included three questions that gave an example of a security failure, then asked the students to identify the lapse in security. They were given five multiple choice options and were asked to "select all that apply". The questions were also shuffled such that the answers were not in order with the CIA acronym. If the student selected the correct answer (even if they also selected other answers), the answer was categorized as correct. If the student did not select the correct answer, their answer was classified as incorrect. The results are shown in Figure 2.

While the first question indicated that students were able to recall the CIA triad and what each letter stands for, the next three questions showed that the students were also able to apply those concepts to real-world examples.

Q9: Online Security

Student participants recognized the implications of sharing information online, with a majority successfully identifying the risks associated with certain seemly benign social media activities. All students determined that sharing information has an impact. Other topics beyond the basic CIA triad concepts were also considered. During one activity, the students were shown a common "What _____ character are you?" meme. These memes assign popular characters to people based on some identifying information. For example, it may determine what popular movie character each person would be based on their birthday, the first letter of their middle name, and the name of their first pet. The students reviewed the memes and, when asked what the point of the meme was, most answered that the meme was supposed to be funny. However, these memes are usually created using common security questions and other personal information. When someone posts their character, they are posting the answers to their security questions and breaking confidentiality best practices. During the pre-camp and post-camp surveys, students were asked: Online surveys and "What _____ character are you?" memes are.... Before the camp, the most popular answer was that the memes were funny with 10 (42%) students selecting that answer and seven (30%) students selecting each of the other answers. After the camp, the most popular answer was the correct answer that the memes contain security questions, with 18 (75%) students answering correctly. These results can be seen in Figure 3. This shows that the students understood that the memes contained security questions, but also understood that they are not meant to be funny (they are meant to discover personal information) and that the memes do have an impact on unsuspecting people.

Figure 3. Number of students selecting each answer for the information security question on the pre-camp and post-camp surveys. In the pre-camp surveys, 7 of the 31 selections were the correct answer (23%). In the post-camp surveys, 18 of the of the 26 selections were the correct answer (70%).

Q10: Cybersecurity Interest

Overall, student interest level increased by the end of the week long outreach camp, with all students having a medium to high interest level and with the percentage of students highly interested doubling. Recall that one of the main goals of the outreach activities was to increase interest in computer science and cybersecurity, and the students were asked how interested they were in computer science and cybersecurity both before and after the camp. Before the camp, four (17%) students said that they were only minimally interested in cybersecurity and indicated that they were unsure of cybersecurity. Eleven (46%) students indicated that they were moderately interested in cybersecurity and seven (30%) students reported that they were highly interested in computer science/cybersecurity, finding it fun and exciting. This showed that before the camps, the students were generally unsure of what computer science and cybersecurity entails, but, after the camp, none of the students reported being minimally interested in computer science and cybersecurity. The number of students highly interested in cybersecurity increased to 15 (63%), more than doubling, and seven were moderately (30%) interested. In total, 22 students (92%) were interested in cybersecurity. These results indicate that the camps increased the students interest in cybersecurity and computer science and they are shown in Figure 4.

Camp Rating

During their post-camp survey, in addition to indicating their interest in cybersecurity and computer science, the students were asked to rate their camp experience and write a comment explaining their camp rating. None of the students ranked the camp as a "one" out of five or "two" out of five, indicating that all of the students enjoyed the camp to some extent. Only three of the students (13%) ranked the camp as a "three" out of five. Of these students, one student noted that they would still recommend the camp to others. Six students (25%) rated the camp as a "four" out of five and 15 students (63%) rated the camp as a "five" out of five. These results indicate that the outreach week was engaging and the activities held the interest of the students. They also indicate that some modifications could be made to improve camps in the future for some students. These results are also shown in Figure 5 and the comments are listed in Table 3.

The comments provided a qualitative context in support of the quantitative assessment, and they clarified which components of the camps were well received and which ones were not (from themes discovered in the 2018 camps' field notes).