More on Assessment

Here is another assessment item - this time concerning general assessment.

This slide presentation was intended as an assessment primer to begin a conversation on best practices for assessment.

*Note that all the original "bells and whistles" placed in the slide presentation do not transfer to this embedded format.

Thanks for viewing!

Standards vs. Standardized Testing: Comparing Apples to Oranges

For the last six months or so, most of my education thoughts have circled around assessment but not standardized testing. So in a continuing effort to educate myself I read the first chapter of a book by Linda McNeil entitled Contradictions of School Reform: the High Cost of Standardized Testing published in 2000. The chapter’s title, “Standardization, defensive teaching, and the problems of control” reflects a critical discussion of the inequities caused by standardization in education.

As a companion reading I chose “Academic Improvement through Regular Assessment” by Patrick J. Wolf for the Peabody Journal of Education (2007). Wolf argues that although there are complaints of “too much testing” in schools, more frequent and regular testing is needed to support academic improvement. Since standardized testing, academic standards, NCLB (No Child Left Behind) and AYP (Annual Yearly Progress) have become divisive and controversial subjects, these two readings seemed like an interesting starting point.

Right away McNeil points out a fundamental problem, one of terminology. The terms “standards”, “standardization”, “standards-based assessment”, and “standardized tests” are being used interchangeably and the result is confusing.

Standards vs. Standardization

When I became a new teacher about ten years ago, the term standard was often used interchangeably with the term expectation. “We have high standards for our students.” In this case it was used to describe a level of quality for work, outcomes, and behavior.

Toward the end of my teacher training as a secondary science and math teacher, I learned that the National Science Teachers Association (NSTA) had recently published a set of “science standards” to guide curriculum. These were basically suggested minimum learning goals in science for each grade level. Suggestions and examples of instructional methods and possible assessments were sometimes included or published in companion documents.

Now when we refer to “standards” in the classroom, we are referring to the Indiana Academic Standards which are published on the Indiana DOE website for all grade levels and core academic subjects. These are also minimum learning goals. The website also includes some resources for suggested instruction and assessment. Hundreds of teachers from all areas of the state had input in writing these goals and resources. So when teachers create a lesson plan, they often required by their district or principal to cite the Indiana Academic Standards which directly relate to the lesson.

In related contrast, the term “standardization” is more general. It is used in all areas of business and culture to mean consistency or sameness. In her book, McNeil argues that “high standards and issues of quality” have become confused with standardization. Her primary example is the state of Texas which became a leader in school reform in the 1980s and early 1990s. In an effort to improve educational quality to hold “schools and school personnel more accountable for their professional practice”, the state government “enacted a set of standardized controls to monitor children’s learning and teacher’s classroom behavior. …The activities they mandated were to be uniform…” The plan to improve quality was to mandate sameness not just in curriculum but instruction and assessment as well. This is the ultimate factory model of education. But because each student brings personal learning variables, it does make sense that sameness in instruction would result in a higher quality of learning. No wonder people are confused.

So we could say that in Indiana we have “standardized our curriculum by legislating that all students in the state of Indiana accomplish or show progress toward the academic goals described by the written standards”. So the sameness is conveyed through the aim that all Indiana students meet these minimum learning goals. The quality pertains first to the quality of the written standard itself (which can be changed or revised) and second to the degree the standard is met, exceeded, or failed. This leads us directly to a discussion of assessment.

Standards-Based Assessment vs. Standardized Tests

To decide if a standard is met, exceeded, or failed, we must figure out a way to measure it. Standards-based assessment simply means that the test items directly refer to the standards learned by the students. Once again this refers to the quality of the assessment. The format of the assessment is not dictated. ISTEP is an example of one form of standards-based testing. This becomes confusing because ISTEP is also a standardized test. It is standardized because it is mandated that all Indiana students (grades 3-10) take the same test at the same time. The sameness of the standardized test insures some comparative data for analysis. However, the sameness of the standardized test does not insure a high level of quality.

These are crucial distinctions to make especially when Wolf in his article defines the two purposes for assessment: diagnostics and accountability. It makes perfect sense.

Teachers and students need diagnostics. These formative assessments should be timely, frequent and relevant (Darling-Hammond, L., 2008) (Schneider, J. 2007). Standards-based assessments are by definition relevant. Thus ISTEP can be used as a diagnostic but it is not considered timely since the results are returned several months later. It is also not frequent since it is only given once per year. So ISTEP could be considered a “limited” diagnostic. Because it is standardized for the state, there is plenty of data to analize for gaps and trends to gauge the big picture.

Since the diagnostic side of assessment is relatively easy to define, the issue of accountability becomes the controversial double-edged sword currently leading the charge for school reform. The current favored evidence for teacher and school accountability are the results of yearly state standardized tests. In science class, you learn early on that you cannot make a conclusion based on one or two measurements. You must have multiple trials, with multiple measurements to analyze and come to a conclusion. So because standardized tests such as ISTEP are a “limited diagnostic” it does not make sense that those once a year results should be the only evidence considered for accountability. So really there aught to be more measurements, more assessments, not less.

“As one principal of an inner-city school once commented at a public forum on No Child Left Behind, ‘Without regular testing teaching is like flying blind’ (World Bank, 2002).” (Wolf, 2007) Why are educators so afraid of assessments? Life is full of tests. Testing is not something we can or should hide from. It is something we should acknowledge as an important part of our life and learn to manage and not fear. (Wolf, 2007)
The educational system, using standards and standards-based assessments, makes logical sense. The controversial issues come to the surface because of the power or control of the source material for the standards, the assessments, and the accountability practices. In the case of Texas, “These controls arose outside the educational system, derived from pressures from the business establishment to fund only those educational expenses that contributed to measurable outcomes.” (McNeil, 2000)

Academic standards and assessments will only be as good as the source materials and the authors who write them. Measurements to determine accountability will only be as good as the measuring instrument. As always, with problems in education, it comes down to the actual implementation. If schools and the world of education had come up with a sound and defendable approach to measuring accountability on their own, then there wouldn’t have been a need for legislation to force the issue.

Resources

Darling-Hammond, L (2008). Powerful Learning - What We Know About Teaching for Understanding. San Fransisco, CA: Jossey-Bass.

Indiana's Academic Standards and Resources. Retrieved February 12, 2009, from Indiana Department of Education Web site: http://www.indianastandardsresources.org/index.asp

McNeil, L. M. (2000). Contradictions of school reform: Educational costs of standardized testing. Critical social thought. New York: Routledge.

Schneider, J. (2007). Chalkbored: What's wrong with school and how to fix it. [S.l.]: Pace of Mind.

Shulman, L. S. (2007).Counting and recounting: Assessment and the quest for accountability. The Magazine of Higher Learning. 39, 20-25.

Wolf, P.J. (2007). Academic improvement through regular assessment. Peabody Journal of Education. 82, 690-702.

World Bank. (2002, October). No Child Left Behind: Implications for Washington, DC – A roundtable discussion with Mayor Anthony Williams. Videotape. Washington, DC: The World Bank

New Book List

I have added another book list in one of the tool boxes on the side. This list contains some of my personal favorites for supplemental reading to use with students. Alice in Quantumland by Robert Gilmore is particularly helpful using imagery to explain quantum theory. I hope to keep adding to this list. Click on the book titles to link to a summary/overview.

Feedback welcome!

Emerging Techology - Go! Animate

How about a little chemistry humor? Just hit play. (make sure your sound is on for full effect)



I really crack myself up. How about another one?

I know, I know...I'll stop now and get on with the discussion.

The research for this particular post has been way too much fun. In fact, don't all teachers need a little more fun? Well if you are looking to insert some fun into your classroom , this technology has it built in! And who couldn't use a little humor or visual boost at a staff or professional development meeting.

Go! Animate.com is very easy to use but a little limited in choices. For each scene of your animation, you choose a background and just insert characters. The manipulation of the characters is easy once you know what you want. There is also the option of using your own face (by photo). For teachers, this could go in several directions (most of them hilarious). In fact I think planning your story and dialogue is the hardest part.

For a middle or high school level science class, I think its educational applications are limited to providing some instructional variety in short spurts (like the jokes I attempted to animate above.) I also began work on a lab safety animation and if I can get it to work I will post it here as well. I should also come up with an ELECTRON joke to complete all the subatomic particles! Most of the time math and science classes take themselves way too seriously (not to mention how seriously mathematicians and scientists take themselves).

Sequencing becomes the key skill that the user needs for putting together these animations. So it is possible that students could use this application to provide visual imagery for a sequence. It could also be used as an option for a project that requires a presentation other than powerpoint slides. Especially for younger students, this could have wonderful applications as a story sequencing tool.

The visual appeal to the students is a no-brainer. The animation could also be used to address issues of behavior (acting out, cheating, bullying, etc.) by bringing them to light and providing alternative solutions.

So I recommend you try it out - it's free and did I mention that it is fun?

Emerging Technology - Google Forms

For my second look at an emerging educational technology, I have chosen Google Forms. In my previous post, I took a look at the other applications in Google Docs. Once I saw what Google Forms could do, I knew it deserved its own post.

It is very simple to use. It allows you to type in questions, choose the format for the answer, and add general directions for the reader. When you are finished, you have a URL address to an electronic form that stores the answers in a spread sheet for you, the creator. Once I discovered this I had to think up an immediate application. So I decided to type up a five question survey to send to some friends of mine that attend an annual social event that I have been hosting for 10 years. My answer formats included: text, paragraph text, and choose from a list. It took maybe 10 minutes to type in the whole thing. I emailed out the URL to my select list of 35 friends, and within 24 hours, I had twenty replies. Not bad at all. There is also an analysis tool included that showed a summary of results for each question. When appropriate there is even a bar graph displayed. And if you prefer, you can export the results data to an excel spreadsheet.

(In fact my husband was bummed when I showed it to him. He had just led a team- building retreat for a not-for-profit and he said that this application could have saved him a few hours if he had used it to replace a paper questionnaire.)

So as an educational tool, can anyone say ----- prior knowledge assessment? In a web cast from the 2008 NECC that I listened to recently, Cheryl Lemke, who spoke on “The Ripple Effect – 21st Century Innovations that Matter”, directly referenced prior knowledge assessment as integral to instructional best practices. As a chemistry teacher I have met so many high school students whose preconceptions lead directly to misconceptions. Abstract learning of chemistry concepts is already hard enough without needing to overcome a variety of misconceptions. And in a study that I recently reviewed in an earlier blog post (Moallem, Kermani, and Chen, 2005), while I disagreed with many of the assertions made by the authors about the effective use of handheld wireless computers in the classroom, they also discussed the benefits of prior knowledge assessment that could be done and quickly submitted via the web prior to class.

I am also thinking that students could potentially use Google Forms to set up data collection for an investigation or science fair project. The simplicity of it is appealing.

Since I am going into elementary schools and helping with their science programs, this tool would be a useful way to briefly survey the teachers with questions like: “How much time per week do you spend on science?”; “What are your favorite and least favorite science topics to teach?”; or “ Is there a science topic for which you would like more support?”. Usually I have to verbally ask the teachers in person because papers just get lost in the shuffle. I think they would be more likely to fill out an electronic form.

As far as drawbacks, student access outside of school is the primary issue. There also could be privacy issues since I believe the data could be accessed. The searches I did to confirm this did not turn up conclusive information on privacy for Google Docs as a whole. I am sure the issue will play out overtime – but as educators we should be aware of what we are using on the web.

So overall, I give a big thumbs up for Google Forms as a potentially useful educational tool.

Now for your participation - fill out the following form and submit. It's that easy!



Lemke, Cheryl. "The Ripple Effect - 21st Innovations that Matter." National Educational Computing Conference. San Antonio, TX. 30 June 2008.

Moallem, M., Kermani, H. & Chen, S. (2005).Handheld, Wireless Computers: Can they Improve Learning and Instruction?. Computers in the Schools. 22, 93-106.

Emerging Technology - Google Docs

Using Google Docs as a successful tool for the high school science classroom, gets the full thumbs up from me!

Let's take a look at my recent exploration of this free and easily accessible software application. Imagine a student using a word processing tool at home to begin a lab report. Then imagine the lab partner accessing the document and adding, revising, or editing it from the comfort of web access at home. In fact both students could both work on the document at the same time without needing to be in the same room. Sound intriguing? I think so.

Now imagine a different scenario... four different chemistry teachers must plan the pacing of the curriculum for the next month and link it with the district standards. They have a week to do this task. Their schedules are such that they have no common planning time or lunch time. So one teacher begins a spreadsheet to chart the pacing with the school calendar. All four teachers can then access and add to the spreadsheet when it is convenient with their schedules.

Did I mention that one tool included in these applications is a recorded revision history so the teacher or the department head can keep tabs on who contributed what? Brilliant!

At the outset, access to this application requires a Google email account. The person who then starts the document (the owner) must designate access (share) with the others allowed to work on the document. It does not appear that there is a limit on the number of shared contributors and all contributors are listed. The word processor itself has similar features to Microsoft Word. There were a few bugs as I worked on the document ---mostly delays in the screen refresh when making edits. But there is definitely the ability to insert data tables. Besides the word processor there are also spreadsheet and slide presentation tools. All the necessary tools for group projects galore.

In this year's publication, Powerful Learning: What we know about teaching for understanding, Linda Darling-Hammond and seven other academic researchers have coalesced the pedagogical research into a handy-dandy reference book. According to Darling-Hammond, et al., not only is collaborative learning one of the "most studied pedagogical interventions in the history of educational research", they report "overall these analyses come to the same conclusion: there are significant learning benefits for students when they are asked to work together on learning activities as compared to approaches where students work on their own (Johnson & Johnson, 1981, 1989)". As a tool to support collaboration, Google Docs has the potential to be a useful tool in the classroom.

The immediate drawbacks are related to student access to the Internet outside of school. While the students in my children's private Catholic school would all have unlimited access to the Internet at home, many students in my former inner-city chemistry class would not. Though I think even for students with limited access at home - time could be built in to the class for group work with some creative planning. There might also be issues for parents with monitoring the google email accounts.

As another plus, Google is reaching out to the education world in a variety of ways. They have specific links for educators and are beginning to build a library of lessons using Google applications submitted by teachers. They also support a professional training opportunity called the Google Teacher Academy. Interested teachers who already use technology creatively in the classroom can submit an application for this one day workshop.

Best of all - as an emerging technology, Google Docs is free. There is no hardware to buy and it is a very small learning curve because the applications are so similar to the familiar MS Office. Anyone with internet access can create an account. So in terms of the technology penalty (Van Horn, 2007) I discussed in my last post, Google Docs gets a gold star from me.

For another blog post that details experience with online collaboration check out the following link:
http://frequanq.blogspot.com/2006/11/collaboration-using-google-docs.html

The fourth application included in Google Docs is Google Forms. I am so excited for the educational applications for this emerging technology that I have decided it needs it's own blog post. So check back for my next emerging technology post!


Darling-Hammond, L., Barron, B., Pearson, P., Schoenfeld, A., Stage, E., Zimmerman, T., Cervetti, G., & Tilson, J. (2008). Powerful Learning - What We Know About Teaching for Understanding. San Fransisco, CA: Jossey-Bass.

Johnson, D. W., & Johnson, R. T. (1981). Effects of cooperative and individualistic learning experiences on interethnic interaction. Journal of Educational Psychology, 73, 444-449.

Johnson, D. W., & Johnson, R. T. (1989). Cooperation and Competition: Theory and research.
Interaction Book company; Edina, MN.

Van Horn, R (May 2007). Technology-The Technology Penalty. Phi Delta Kappan. Vol. 87, No. 9.

Critical Thinking Analysis - The Technology Penalty

For my current graduate course, we were each asked to analyze two journal articles. I have written the following based on that assignment.

Module 3 – Critical Thinking

The relationship between the two articles assigned to our group is an interesting one. In “Handheld, Wireless Computers: Can They Improve Learning and Instruction?”, Moallem, Kermani, and Chen report their attempt to develop an instructional model for the use of handheld computers in a university classroom. The second article, “TECHNOLOGY: The Technology Penalty”, Royal Van Horn lists several instances when educators “do something using technology that you could do easier, quicker, or more efficiently without using technology”. While I appreciate the thoughtful and earnest research approach used by the first group, I find their optimistic conclusions slightly odd since several parts of their report support the technology penalty as described by Van Horn. Though I do not agree wholeheartedly with either article, the juxtaposition of the views makes for an interesting discussion.

As Moallem, et al, designed their study, they made a concerted effort to begin with a solid research based list of principles for good instruction. They reiterate throughout the report the necessity of good instruction before applying technology to the classroom. My recent experience (the last few years) observing in a multitude of classrooms proves this point again and again. We must fix the instruction first. The same philosophy was voiced in a web cast presentation by Stephanie Sandifer from the NECC conference. As a technology guru in her school, Stephanie emphasized beginning with good instructional design and then using technology to fill a gap or solve a problem to further that design. Even though I was introduced to a great variety of instructional strategies during my teacher training eleven years ago, I am just now seeing the beginnings of thoughtful attention to designing instruction in local classrooms. There finally seems to be an interest and even an urgency to implement a new kind of lesson planning and focus on some high-order thinking skills. Whether this is because of a stronger research base that defines authentic learning or because there is now federal accountability for failing schools, it’s about time.

While I completely agree with the authors’ diligence in providing researched-based principles for good instruction, I also struggle with the next section of the study which begins: “HWCs (handheld wireless computers) can facilitate immediate assessment of and access to student prior knowledge, motivation and learning, and cognitive styles through online and ongoing self-assessment tools.” (p.96) And then goes on to assert that HWC’s can facilitate all the other listed principles as well. I understand that these assertions are correct however they are also correct for a regular notebook or PC. There is no information to differentiate the advantages of the HWC over a notebook or PC. I suppose you could make the argument that it is more portable but the HWC also comes with a new learning curve and training time for teachers and students. HWC’s also have an additional cost consideration. The university instructors involved in the study also raised the issue of the time required for designing the unit. So with the idea of Van Horn’s technology penalty in mind, I am beginning to question whether this HWC project was worth all the time and effort.

As I continued to read the results and conclusions concerning the hand held computers, I became increasingly dissatisfied with the content of the article. When the participating students mentioned that they felt the HWCs “increased their motivation by making the unit of instruction more interesting” (p.101), this throws up a red flag to me that should be further explored. Was the instruction more interesting because of the new technology “toy”? Or was the instruction more interesting because it was designed to be more interesting than previous class sessions? Any new (and interesting) technology could soon become uninteresting once the newness wears off. And this then becomes an example of teachers entertaining students rather than facilitating a situation in which the students are motivated and responsible for their own learning. It’s not that I am against a little entertainment, but I have seen students begin to expect it on a regular basis. This can then lead to a passive learner who is completely dependent on the teacher for motivation.

Another interesting aspect to the HWC project, was using the handhelds during instruction to facilitate rapid student response and then adjust the instruction accordingly based on the response results. The implementation of the project states that: “The instructors developed key questions related to the concepts covered in their lecture/demonstration or large group discussions.” (p. 98) I agree that this function is highly dependant on the quality of the questions used but I think it is also dependant on the sincerity of the student responses. In a high school classroom for example, it would not be long before some students would try to manipulate the flow of instruction through their answers to the questions especially if it is an anonymous response. I do not know if this would also be true at the college level as I am not as familiar with undergraduate students. The rapid student response may make more sense in a math classroom to assess computation and problem solving for the whole group.

Another tie in to Van Horn’s technology penalty has to do with his argument that “innovations and good ideas often fail because of the ‘stuff’ requirement. All too often people want teachers to do things that require stuff they either do not have or do not have the time to find, assemble, or make. Laboratory science is a good example of an educational innovation that has an immense stuff requirement.”
His technology example of this goes as follows: “Using technology in the classroom can also have a stuff requirement that ups the technology penalty. For example, consider a mobile laptop computer lab cart. Rolling the cart into a classroom, distributing 30 computers, and attaching a mouse to each one takes time. Returning the laptops to the cart and plugging them all into charging stations takes still more time. For this reason, I am not a fan of these kinds of mobile computer labs.”
Based on this example the use of HWC’s would have a similar penalty. I have been in classes where mobile laptop carts were used very effectively due to the sound instructional design of the lesson. The number of students in the class and the amount of time available for such activities was also a factor. It takes much less time to distribute laptops for ten students than it would for thirty students. It could also be argued that for some classrooms, access to a laptop cart or HWC’s is preferable to no computer access at all.

As Moallem, Kermani, and Chen begin their evaluation of the project, the first discussion point in the evaluation section of the study remarks that:
“Instructors’ and researchers’ reflection and observational notes indicated
that in all three courses, using HWCs for delivering instruction changed
the class dynamics from a primarily lecture-driven, large group discussion
to a more interactive and student-involved learning environment.
Instructors agreed that the applied instructional strategies using HWCs
made them more responsive to students’ understanding of the materials,
helped students become more engaged during instruction, and encouraged
students to participate more actively during discussion and team
activities.” (p.100)
When reading this section it is important to note that it is not necessarily the use of HWC’s that changed the classroom dynamic for the better, but probably the change in instructional strategies compared to a traditional lecture-driven university course. If interest in the new technology of HWC’s was the driving force behind the instructors interest in changing the instructional strategies then we are using a backward model – when new technology drives strategy rather than sound instructional design driving technology application. This seems to be the opposite of what the authors intended at the outset of this project.

To their credit the authors of the HWC study do come back to this later in the report where they conclude:
“In other words, the promise of handheld computers lies in facilitating the improvement of instruction. Both university faculty and school teachers need to be reminded that new and emerging technology tools such as PDAs are only effective if they are used to improve the process of learning and teaching that is established by research.”(p.104)
I also feel the need to comment that using a technology interface for each individual in a group discussion when all participants are physically present in the room seems odd to me. Of course access to information via the web or class materials could be beneficial, but I guess my question refers more to the skills of the instructor in the first place. A skillful instructor can certainly facilitate participation in group discussion and respond to student’s understanding of the material without the use of technology. For all the research and training available, I truly feel that some teachers get it and some teachers don’t get it. My most recent reading has been Powerful Learning – What We Know About Teaching for Understanding by Linda Darling-Hammond, et al. In this book, the research on effective strategies could not be more clear and accessible. The slow adoption of these strategies deprives millions of students of their benefits and our educational system continues to plod along with no clear direction except to avoid the penalties of the No Child Left Behind federal legislation. Although if that is what it takes to get things moving forward, I am all for it.
In the meantime as we try to affect change and navigate our own little corners of the education community, the application of technology to education deserves continued discussion. I appreciate Van Horn’s final comments about the technology penalty:
“Let me state emphatically that I am not discussing the technology penalty because I want to discourage the use of technology. Rather, I am discussing the technology penalty so that readers can take it into account if they are considering integrating technology into the schools at any level. As the old saying goes, ‘Forewarned is forearmed.’"
This quote reminds me of my slogan for this course – If we know better, we should do better. Van Horn is essentially telling his readers to think carefully and gather information about what they are doing and then proceed. That’s what I want for my teaching interns and science students. And that’s what I want for my children – to be thinkers and doers.

Darling-Hammmond, L (2008). Powerful Learning - What We Know About Teaching for Understanding. San Fransisco, CA: Jossey-Bass.

Moallem, M., Kermani, H. & Chen, S. (2005).Handheld, Wireless Computers: Can they Improve Learning and Instruction?. Computers in the Schools. 22, 93-106.

Van Horn, R (May 2007).Technology-The Technology Penalty. Phi Delta Kappan. Vol. 87, No. 9.