In 2012, Washington state participated in ShakeOut, an annual, one-day event that encourages residents to practice “drop, cover and hold on” drills for earthquakes and evacuation for tsunamis. To better understand the role of school drills in improving individual and community resilience to disasters, this evaluation examined the effectiveness of the ShakeOut drills in improving or maintaining children’s accurate risk perceptions and adaptive response capacities for earthquakes and tsunamis. Using matched pretest and posttest questionnaires, the analysis examined both population level and individual differences in children’s knowledge and scenario-based knowledge application before and after ShakeOut. Children demonstrated high levels of correct knowledge of protective actions for earthquakes and tsunamis both before and after ShakeOut. However, the findings indicate that significant portions of children have varying levels of knowledge of the causes of injury and approximately a third of children chose an incorrect action or indicated uncertainty in scenarios not commonly practiced in school earthquake drills. Also, more than a quarter of children were not aware they practiced vertical evacuation procedures for a tsunami during ShakeOut. Children would benefit from practice for different scenarios, such as when they are outside or traveling between classes, and explicit lessons on protective actions.