Assessment Feedback: From Quality Control to Quality Assurance

Author’s note: I wrote this position paper for an Assessment class from my MEd program, in which I haven’t really been given grades yet. Ironically this has really bugged me, but there hasn’t been much of any feedback until I turned in this paper. The very next day I received some very positive and specific feedback. Still no grade - guess that is something that takes getting used to. Please also check out Justin’s excellent post on the effects of grades.

Assessment Feedback: From Quality Control to Quality Assurance

Fueled by standardized testing that states have instituted to meet No Child Left Behind (NCLB), assessment in education is big news these days. As individuals and institutions face the repercussions of their test scores, a firestorm of controversy about high stakes testing has erupted. What seems to get lost in the fray is the true power that assessment feedback has on individual student trajectory at school and in their life. Standardized testing has put the focus on quality control, as it attempts to weed out schools and teachers that are underperforming. However, this summative assessment offers students very little in terms of feedback they can use to impact their learning. On the other hand, formative assessment, a quality assurance approach to feedback, offers enormous potential to improve student learning (Leahy, Lyon, Thompson & Wiliam, 2005).

Formative assessment is nothing new, but it has been overshadowed lately, even as ongoing research continues to affirm its important impact on learners. Formative assessment can take a variety of shapes and forms including observation, discussion, exit slips, peer reviews, presentations, pop quizzes, etc. “Formative assessment done well results in student achievement gains of about 26 percentile points” (Ainsworth, et. al., 2007). Not all formative assessment has a positive impact on student achievement. In fact, formative assessment can also have devastating effects. The same study that documented student gains, also “found that in 33% of the students they examined, feedback had a negative impact on achievement” (Ainsworth, et. al., 2007).

Why do we need a shift in focus from quality control to quality assurance in assessment feedback? Partly it has to do with the research that backs up the impact of formative assessment, but mainly it has to do with public perception. Any form of assessment used in the public education arena now, has a whole slew of hyper-vigilant stakeholders including students, teachers, administrators, parents and public policy makers. Despite the research, many adults with influence both inside and outside of the system continue to draw conclusions about assessment primarily from their own prior educational experiences. In his education manifesto, Stop Stealing Dream, Seth Godin writes that “the sanctity of performance / testing / compliance-based schooling is rarely discussed and virtually never challenged” (2012).

Using quality control and quality assurance as metaphors, can help stakeholders connect the impact of assessment feedback on students, with their own real-life experiences. In the business world, quality control is all about weeding out defects from finished products and is therefore a reactive process. Quality assurance is a proactive process that aims to create defect-free products by instituting an iterative process of design, development and testing throughout the life cycle of a product. In the education world, high-stakes testing as well as accountability measures like NCLB and the Common Core Standards represent quality control, while learner-centered formative assessment represent quality assurance.

Former Assistant Secretary of Education, Diane Ravitch, once a strong supporter of NCLB started having doubts about the emphasis on assessment use as quality control. In 2010, she wrote, “I started to doubt the entire approach to school reform that NCLB represented. I realized that incentives and sanctions were not the right levers to improve education; incentives and sanctions may be right for business organizations, where the bottom line — profit — is the highest priority, but they are not right for schools. I started to see the danger of the culture of testing that was spreading through every school, community, town, city, and state” (p. 296). Many states are struggling with the fallout from quality control as they seek waivers because they cannot fulfill NCLB’s mandates. Standardized testing in California is currently in a state of flux as they try to deal with a huge budget crisis and changing standards. In the midst of this turmoil, the state’s Department of Education is looking to change the focus of testing “from measuring results in instruction to improving instruction” (Fensterwald, 2012), essentially shifting from quality control to quality assurance.

The focus on quality control types of assessment can have devastating effects on students. Justin, an 11th grade student, recently wrote about his school experiences: 

We all stress ourselves out to memorize the formulas. We all have had that cram night before the final… School has done a very sad and unfortunate thing. It has placed a number on my learning … The extent we go to thinking and stressing over grades is honestly insane… We have to stop being preoccupied with standardization, we have to stop being preoccupied with “seeing progress” through numbers, we have to let kids be kids. Let me learn how I want to learn (Strudler, 2012).

The recent documentary film, Race to Nowhere, echoes concerns about the amount of stress students face from our highly test-driven culture (George, 2010).

Teacher morale is also being affected, especially as more states incorporate standardized test results into their teacher evaluation systems. “There is broad agreement among statisticians, psychometricians, and economists that student test scores alone are not sufficiently reliable and valid indicators of teacher effectiveness” (Rothstein, et. al. 2010). Even value-added modeling (VAM) which incorporates standardized test results along with other metrics does not pass muster for these researchers. The unfairness baked into the system is not lost on teachers, with almost half of them leaving the profession after just five years. Testing pressure is cited as one of the top five reasons for this teacher turnover (Smolin, 2011).

If the emphasis is to shift from quality control to quality assurance, then it must be quality assurance done well. In software development, quality assurance is part of the process to bring an application to market. The primary purpose is to catch programming bugs and usability issues before the product is released.  If quality assurance is not well planned and well executed, then the software company can expect an unhappy customer dealing with an unstable product. As Seapine Software notes in its white paper about quality, “if the cost of quality is high, the cost of poor quality is still higher” (2009). This axiom holds true for the impact of assessment feedback on student learning and outcomes.

As noted earlier, some kinds of formative assessment feedback can have very negative impact on student learning. For example, one meta-analysis found that there were negative effects on learning when teachers simply told students whether their answers were correct or incorrect. Feedback that explained the correct answer or gave suggestions to help students work it out for themselves, showed a 20% gain in achievement (Ainsworth, et. al., 2007). Several educational leaders have shared their research-based strategies for embedding effective formative assessment feedback into the classroom in their book, Ahead of the Curve. These strategies share many things in common, like providing feedback that moves learners forward, providing students with clear expectations and criteria for success, empowering students to take ownership of the assessment process for themselves and with their peers and most importantly, providing feedback that is timely and accurate (Ainsworth, et. al., 2007).

While some sort of summative assessment feedback may still be necessary to evaluate student mastery of learning, a shift to emphasize the important role that formative assessment feedback has on student learning is required in our current test-obsessed culture. Quality assurance, not quality control, is going to yield the most long-term, positive effects on teaching and learning.

References

Ainsworth, L., Almeida, L., Davies, A., DuFour, R., Gregg, L., Guskey, T., … Reeves, D. (2007). Ahead of the curve: The power of assessment to transform teaching. Bloomington, IN: Solution Tree Press.

Fensterwald, J. (2012). Great uncertainty over direction of state standardized tests. EdSource. Retrieved from http://www.edsource.org.

George, D. (2010). ‘Race to Nowhere’ film highlights stress students face in high-pressure academics. The Washington Post. Retrieved from http://www.washingtonpost.com.

Godin, Seth (2012). Stop stealing dreams: What is school for? Retrieved from http://www.sethgodin.com.

Leahy, S., Lyon, C. & Thompson, M., Wiliam, D. (2005). Classroom assessment: Minute by minute, day by day. Educational Leadership. November 2005, 63 (3), 19-24.

Ravitch, D. (2010). The death and life of the great American school system: How testing and choice are undermining education. Philadelphia, PA: Basic Books.

Rothstein, R., Ladd, H., Ravitch, D., Baker, E. Barton, P. Darling-Hammond, L. … Shepard, L. (2010). Problems with the use of student test scores to evaluate teachers. Economic Policy Institute. Retrieved from http://www.epi.org/publication/bp278.

Seapine Software (2009). Identifying the cost of poor quality. Retrieved from http://downloads.seapine.com/pub/papers/CostPoorQuality.pdf.

Smolin, M. (2011). Five reasons teacher turnover is on the rise. Take Part. Retrieved from http://www.takepart.com.

Strudler, J. (2012). Grades limit my learning. Cooperative Catalyst. Retrieved from http://coopcatalyst.wordpress.com.

re-imagine, re-mix, re-create

I came across HitRecord earlier this week. It has all this cool media (video, sounds, images, text) called records created by its users for use by its users to re-imagine, re-mix and re-create. It is a community in formation as collaborations are encouraged. It is a entrepreneurial business in experimentation as hit records receive profit sharing. I am intrigued and inspired. I want to join in the fun, but I am afraid. After all, I’m not really an artist - am I?

I’m not sure when I decided I wasn’t creative. I remember coming in third place in a poetry contest in middle school. I remember because there’s a newspaper clipping in my scrapbook with my picture and a caption that says I came in third place. Unfortunately, the poem didn’t make it - something about rain, I think. I remember one of my favorite creations in art class was inspired by Superman’s fortress, I don’t think my teacher got it. My horse drawing was horrendous, her disappointment with that was clear. I think it was the B in ceramics in college that finally sealed the deal. If creativity was going to mess with my GPA, then I wasn’t going to be involved and I probably wasn’t very good anyway.

Is that how creativity dies? Ken Robinson says schools kill creativity. Howard Gardner says the creating mind is critical to be productive and fulfilled in this age. He says education needs to protect creativity by creating environments where mistakes, self-reflection and a variety of approaches are valued. Seth Godin says that artists are absolutely essential to the educational system, but the educational system is likely to use all the bureaucracy and fear at its disposal to stamp out the artistry and unique gifts that artists bring to the table. 

I’m not sure where education will come out on this, but I do know that technology is offering many different avenues for creative expression. There are all sorts of individuals and organizations using the internet to re-imagine, re-mix and re-create. They seem to be providing a space for mistakes, self-reflection and a variety of approaches. I think it’s time to join in. Wish me luck! I hope to see you out there too because we’re all creators.

Book Review: Linchpin by Seth Godin

Are you indispensable?  That is the question Seth Godin returns to over and over again in his latest book, Linchpin.  Seth is a product of the information age.  He has written dozens of books and founded dozens of companies, many of which, as he is quick to point out, were failures.  He’s also a best-selling author many times over and a successful entrepreneur.   When Seth asks if you are indispensable, it’s clear that discovering the answer to this question has been his compass, guiding him onward.

Seth’s writing style is fast and furious as he rapidly fires one idea after another at the reader, presumably hoping some of them will stick to their target.  The chapters in this book are loosely affiliated thoughts that read much like someone’s blog.  This can make distilling the big ideas a challenge, but they are there.  One major focus of the book is describing the linchpin.  Linchpins are not some special class of people born with magical talents.  They are people who choose to figure things out instead of wait for instructions, thrive on change instead of defending the status quo, make a difference, invest emotional labor, share their passion and leave their comfort zones.

Two other main ideas that Seth wants to get across is that both nature and nurture have been working against humanity in the pursuit of becoming linchpins.  He starts by arguing that the factory model of education and work has nurtured compliant workers who are most comfortable following rules.  When the rules change, for example with the technology revolution, the factory model does not know what to do.  All it can do is try to make things faster and cheaper using the same old rules.  Linchpins are needed to help draw the new map, provide insights and create.  An even greater force working against the linchpin is within their own brain.  The amygdala or the lizard brain, as Seth refers to it, is the oldest part of the human brain and its job throughout the ages has been survival.  It is afraid and avoids risks and when in conflict with higher brain functions, it always wins.  Seth suggests the way to work through the relentless resistance from the lizard brain is to recognize that it is causing the fear and face it.  The resistance also provides a good clue as to what direction a linchpin should go in next - straight into the resistance. 

Finally, Seth suggests linchpins embrace a new economy, or rather a very old economy that has been lost in the consumer culture, the gift economy.  This is the economy of the tribe that is based on mutual support and generosity.  Linchpins do what they do because they are artists. They need to create and share their work, not for profit, but for respect, connection and to change the world.  The very act of putting their unique work out there results in them being indispensable and sought after by others. 

It seems ironic that Seth makes such a compelling case for linchpins being absolutely essential to the educational system while pointing out that the educational system is likely to use all the bureaucracy and fear at its disposal to stamp out the artistry and unique gifts that linchpins bring to the table.  He argues that what should be taught in school really comes down to two things: solve interesting problems and lead.  This may be overly simplistic for the educator armed with pedagogy and curriculum, but in fact, the reality is that something is broken in education.  Recently, IMPACT, a grassroots justice group in Charlottesville, VA issued a press release about the unemployment rate for people under 30 being three and a half times larger than the rest of the population.  And why is that?  Research into this issue by IMPACT has shown that employers feel many young people do not possess the soft skills - things like working as a team, relating to co-workers or customers, appropriate attire - needed to maintain a job.  What if their unique gifts had been nurtured during school and they had been challenged to solve problems and lead?