Embedded Formative Assessment by Dylan Wiliam

5 Key Strategies & TALK:

1. LT/SC (Chapter 3):  30 second share, group discussion on LT/SC
2. Eliciting Evidence of Learning (Chapter 4):  ABCD Cards, Exit Tickets, Diagnostic Questions
3. Provide Feedback that moves learning forward (Chapter 5):  Find & Connect, * * Wish
1. Feedback is more effectively recieved with they are in control of what they are working on.
4. Ownership of Learning:  Stop/Slow Signals, ? Strips, Traffic Lighting, Self-assessment
5. Accountable for Group Learning:  Carousel, Think/Pair/Share, Jigsaw

Embedded Formative Assessment by Dylan Wiliam
*Very informative and helpful book - highly recommend reading it!

Notes:

Chapter 2 - A Case for Formative Assessment:

• The best teachers fail all the time because they have such high aspirations for what their students can achieve (generally much higher than the students themselves have) (29).
• Formative assessment could be called cognitively guided instruction!
• When teachers set rules about how they would review the data and the actions that were to follow before they assessed their students, the gains in achievement were twice as great as those cases in which the follow-up action was left to the judgement of the individual's teacher once the data had been collected.  Interestingly, when teachers produced graphs of the progress of individual students as a guide and stimulus to action, the effect was almost 3 times as great as when this was not done (35).
• We concluded that the research suggested that attention to the use of assessment to inform instruction, particularly at the classroom level, in many cases effectively doubled the speed of student learning. ... Student with which the teachers used formative assessment techniques made almost twice as much progress over the year (37).
• Formative  Assessment:  the process used by teachers and students to recognise and respond to student learning in order to enhance that learning, during the learning (37).
• Formative and summative make much more sense as descriptors of the function that assessment data serve, rather than of the assessments themselves (39).  (Change name to assessment for learning.)
• Assessment functions formatively to the extent that evidence about student achievement is elicited, interpreted, and used by teachers, learners, or their peers to make decisions about the next steps in instruction that are likely to be better, or better founded, than the decisions they would have mde in the absense of that evidence (43).
• Questions:  Where are learners in their learning?  Where are they going?  How will we get there?  (Look at chart on page 46.)

Chapter 3:  Clarifying, Sharing, and Understanding Learning Targets and Success Criteria:

• Ensuring that all students know what quality work looks like has a profound impact on achievement gaps (55).
• Learning Targets and Success Criteria are ..."useless to those who don't understand what is meant (58)
• The advantage of developing the learning (targets) with the students is that doing so creates a mechanism whereby students can discuss and come to own the LT/SC, making it more likely that they will be able to apply the LT/SC in the context of their own work (59).
• LT/SC must be generalizable - transferable...they are best when "students are able to tranfer their learning to novel contexts."  Ex:  instead of "write instruction on how to change a bicycle tire" use "To be able to write clear instructions"  Context of learning:  Changing a bicycle tire.  (More examples on pg 61.)
• During learning, it is useful to build a degree of generality into scoring rubrics so as to promote transfer.  (Task-specific rubrics are generally more appropriate for summative assessment.) (62)
• Process criteria are particularly important in helping student become owners of their own learning (64).
• How?  (Opportunities to talk about quality work appear to be unlimited (67).)

1. Look at samples of other students' work and engage in a discussion about the strengths and weaknesses of each.  Co-construct a scoring rubric.
2. Create a "what not to write/do" list...
3. Return essay with copies of 3 best essays attached.  Read the essays and compare/contrast with own essay.  Revise and edit and resubmit. 
4. Student generated questions for tests - hard and sheds light on their thinking. (68)
5. WALT / WILF / TIB - We are learning to... / What I'm looking for is... / This is because... (69)

Chapter 4 - Eliciting Evidence of Learners' Achievement

• Teachers must acknowledge that what their students learn is not necessarily what they intended, and this is inevitable because of the unpredictability of teaching.  Thus,  it is essential that teachers explore students' thinking before assuming that students have understood something.  However, generating questions that provide these powerful insights into students' thinking is far from straightforward (75).  Goal of questions:  you cannot get the right answer for the wrong reasons.
• Sharing high-quality questions may be the most significant thing we can do to improve the quality of student learning (104)
• Grading vs FA - ...does the teacher find out whether students have understood something when they are still in the class, when there is time to do something about it, or does the teacher only discover this once he looks at the students' notebooks?  Viewed from this perspective, grading can be seen as the punishment given to teachers for failing to find out that they did not achieve the intended learning when the students were in front of them (78)!
• How much students learn depends more on the quality than the quantity of talk (79).
• Less than 10% of the questions that were asked by teachers in these classrooms actually caused new learning.
• 2 reasons to ask questions:  to cause thinking and to reveal thinking (provide information for teachers about what to do next) (79).  
• When teachers allow students to choose whether to participate or not - for example, by allowing them to raise their hands to show they have an answer - they are actually making the achievement gap worse, because those who are participating are getting smarter, while those avoiding engagement are forgoing the opportunities to increase their ability (81).  
• It is better to assume that students do not know something when they do than it is to assume they do know something when they don't (95)
• How? (82-94)

1. Pose - Pause ("1-2-3-4, gotta wait a little more/ 5-6-7-8, don't start now you gotta wait")- Pounce - Bounce
2. Popsicle sticks
3. I don't know = If you did know, what would you say...  OR I'll come back to you after 2 more people share, listen carefully so that you'll know then.
4. Wait time 2-3 seconds...
5. Think - pair - share
6. Listen interpretively  (instead of evaluatively - for the right or wrong answer) - What such teachers seek to learn from the students' response is not, "did they get it?" but rather, "What can I learn about the students' thinking by attending carefully to what they say?"
7. Question Shells:  Instead of "Is a ...?" ask "Why is a ... a ..."  Ex:  "Why is a square a trapezoid?" or instead of "What is a prime number?" ask "Why is 17 prime and 15 not?"
8. Hot seat questioning
9. All student responses (should take 1-2 minutes) - white boards/page protectors, thumbs, fingers,  (Instead of do you understand...ask is this correct? ), ABCD cards/Corners - hold up the card to indicate what you think is right / go to corner and discuss (or get retaught), Exit tickets
10. Hinge Questions (100) - "...design each lesson with at least one "hinge" in the instructional sequence.  The hinge is a point at which the teacher checks whether the class is ready to move on through the use of a diagnostic question.  how the lesson proceeds depends on the level of understanding shown by the students, so the direction of the lesson hinges at this point (101)."

Chapter 5 - Providing Feedback that Moves Learning Forward

• The students receiving the constructive feedback learned twice as fast as the control-group students...(108)  Comment only was better than scores only AND scores and comments.
• Teacher praise is far more effective if it is infrequent, credible, contingent, specific, and genuine.  It is also essential that praise is related to factors within an individual's control (111).
• The greater the mindfulness, the more the learning (111) - Connected to "struggle" in studio work.
• Students given the scaffolded response (only given the minimum amount of support to get them unstuck and to make progress) learned more and retained their learning longer than those given full solutions (111).
• Feedback - oral over written - that was given in class with time to use it improved their work more so than other types of feedback (113).
• Possible Responses to feedback (115):  exert more/less effort, increase/decrease aspiration (goal), abandon goal, and/or reject feedback.
• Dimensions of Attraibutions of Success/Failure (pg 117)
• Feedback functions formatively only if the inforamtion fed back to the learner is used by the learner in improving performance (120).  Think of coaching... feedback moves learning forward.  Effective feedback provides a recipe for future action... 4 key elements:  1.  setting the desired state/ 2.  establishes the current state / 3. compares the current with desired state / 4.  means of bringing the current state in line with the desired state.
• Never grade students while they are still learning.  As soon as students get a grade, the learning stops.  ~Alfie Kohn
• Grades should be used as infrequently as possible.
• GREAT GRADING IDEA - 125
• Feedback should cause thinking (instead of emotional response).  Feedback should be more work for the recipient than the donor (129)  Feedback should relate to the LT/SC that have been shared with the students.
• How?

1. Grading scheme for improvement (128)
2.  Minus, equal, plus - work is decreasing, staying the same, or getting better + comments
3. Three Questions  - read work and respond with three questions. Students respond to the questions...
4. 5 wrong answers - find them.  OR Dot in the margin = something wrong...

Chapter 6 - Activating Students as Instructional Resources for One Another

• Working in a group provides:  motivation, social cohesion (students care about the group), personalization (students learn more from peers), and cognitive elaboration (those who provide help in group settings are forced to think through the ideas more clearly) (134).
• The effect of peer tutoring can be almost as strong as one-on-one instruction from a teacher (134)
• There must be group goals AND individual accountability.
• How?

1. Secret student (norms etc)
2. Trailing Edge - each group is given the grade of the lowest-scoring member of the group.  "If the score is taken as an indication of how well the group worked as a group, then the awared score seems more reasonable.")
3. C3B4Me
4. Peer evaluation of HW
5. HW help board  OR corners
6. 2 stars and a wish - if on sticky notes, teacher should use doc camera to show effective/useful sticky notes and not.
7. Group questions - must have one.
8. Error classification - classify the errors that were made and find someone who didn't make them!
9. What did we learn today?  (Connect to LT/SC) - groups can produce a list and then share one.
10. I - You - We reflection:  Each student records something about his/her own contributions, something about other individual's contributions, and an evaluation of the quality of the work of the group as a whole.
11. Group based test - like KV's homework structure
12. If you've learned it, help someone who hasn't... "The students who learn most in group work are those who give help and those who receive it, provided the help is in the form of elaborated explanations rather than just answers (143).

Chapter 7 - Activating Students as Owners of Their Own Learning

• All students can improve how they  manage the learning process and become owners of their own learning.
• We intuitively grasp that teachers do not create learning; only learners create learning (145).
• Students can develop sufficient insights into their own learning to improve it (146).
• Using self-assessment in these 25 classrooms had almost doubled the rate at which students were learning (147).
• There is another way to think about motivation - not as a cause but as a consequence of achievement (149).
• "Flow" ...arise through a match between one's capability and the challenge of the task.  When the level of challenge is low and the level of capability is high = boredom.  when the level of challenge is high and the level of capability is low = anxiety.  When both are low, the result is apathy.  However, when both capability and challenge are high = "flow" (150).  C:  "Just right" books
• Assessment can imporve instruction, but it can also impact the learner's willingness, desire, and capacity to learn (151).
• Goals;  1.  share LT/SC so that they are able to monitor their own progress toward them / 2.  Promote the belief that ability is incremental rather than fixed / 3.  Make it more difficult for students to compare themselves with others in terms of achievement / 4.  Provide feedback that contains a recipe for future action / 5.  Use every opportunity to transfer executive control of the learning form the teacher to the students to support their development as autonomous learners (152).
• How?

1. Compile list of what they have learned.
2. Traffic Lights - red, yellow, green cups  (Green indicates confidence that the intended learning has been achieved.  Yellow indicates either ambivalence about the extent to which the intended learning has been achieved or that the objectives have been partially met.  Red indicates that the student believes that he or she has not learned what was intended.  Students rate themselves and then teacher says, "Reds over here with me; greens help the yellows; yellows make sure that greens understand this as well as they think they do."  (Also, mark pages of review packet with red, yellow, green)
3. Learning Logs:  Today I learned... I was surprised by... The most useful think I will take from this lesson is... I was interested in... What I liked most about this lesson was... One thing I'm not sure about is... The main thing I want to find out more about is....After this session, I feel... I might have gotten more from this lesson is...

Epilogue - Why Formative Assessment?

• Indications suggest that it is more, and better, general education for more of the population that will guarantee America's future prosperity (159).
• Marianne Williamson (1992) writes, "Our deepest fear is not that we are inadequate.  Our deepest fear is that we are powerful beyond measure."  We now know that the teacher is the most powerful influence on how much a student learns and that teachers can continue to make significant improvements in their practice throughout their entire careers (162)."

Thursday, I watched a webinar on Formative Assessment given by Dylan William (author of Inside the Black Box and Minute by Minute...)

Below are the big ideas:

Webinar notes - Webinar featuring Dylan William: 

 

Big Ideas:

• What children learn is unpredictable….
• Assessment is the bridge between teaching and learning….
• Use evidence about achievement to make decisions about our instructions.
• Metaphor:  Pilots navigate (origin, destination, plan the route, monitor progress make adjustments to the course as conditions dictate) VS Teachers instruct/assess 
• Changing classroom practice is more than providing a tips/techniques for teachers.  What kinds of support do teachers need to implement these types of techniques:  Slow down, provide choice, and create accountability.

What is FA?  Research says...

•  long cycle – 4-6 weeks, monitor student progress, “weighing the pig doesn’t fatten it” – are we headed in the right direction?  Grades – improves thinking about learning, but not learning itself.
• shorter the cycle, the bigger the impact!  The closer the measurement is to the learning, the more impactful it will be.  We use the cues, but not systematically.

Most important parts of the FA:

• Engagement (no place to hide) and 
• responsiveness (using the responses to make adjustments or not)….

Unpack FA:  three questions:  Where are we going?  Where are we now?  How do we get there?

1.  Where are we going?  
   1. Clarify, Share and Communicate  (Learning Target)  This is essential BUT consider these cautions:
      1. Not always possible to be that clear….reacting to poetry!  A whole horizon of goals that are OK. 
      2. Telling them where they are going can ruin the journey/problem.  
      3. Uninspired teaching/teacher if start every class period with a learning objective
   2. LT:  all students benefited, but the lowest students benefited the most from LT and reflective feedback.  Sharing learning intentions (codes of success/ what good work in a subject looks like) closes the achievement gap.
      1. When they find mistakes in other people’s work, they are far less likely to make those same mistakes in their own work.
      2. Provide students the chance to design their own tasks/tests.  More effective form of test preparation
2. Where are we now?
   1. Engineering effective questions, discussions and tasks to find out where students are right now.  Sometimes a statement will lead to more thoughtful discussions than a question.
   2. The RIGHT questions are hard to create on the spot - we need to spend time planning these.
      1. When we ask a question and we get the answers we are looking for, we are tempted to assume it is for the right reasons.  What assumptions are we making?
      2. You cannot give effective feedback unless you ask the right questions in the first place.  It’s hard to come up with these questions…that’s what we should spend time planning!
      3. No Hands Up – more important – (if hands – 2 classrooms:  1) volunteers who are getting smarter and 2) avoiders that are missing out on getting smarter!  Creates bigger and bigger gap.) 
         1.  Random picking,
         2. “IDK” – “Great I’ll come back to you.” Get 3 answers and then say, “Which of those did you like more and why?”
         3. Every 20 – 30 minutes – get response from every student.  (clickers, dry erase boards, etc…)
3. What will we need to do to get there?
   1. Providing feedback that will move learning forward is essential.  Goal:  not feedback about what happened in the past, but feedback about how to improve in the future.  Students should be able to use it in future learning/efforts.
      1. Tells learners what they need to do and HOW to do that.  Causes thinking  - as soon as feedback causes an emotional reaction, learning has stopped.
      2. 4 Responses to feedback:  A) change behavior, B) change goal (increase or decrease level of complexity), C) abandon goal ("math is stupid"), D) reject feedback
      3. Do grades result in the response we want?
         1. If there is a grade, students don't read comments.  We have to stop doing the things that take too much time and don't produce the results we want.  What is that?  Grading!  We grade too much and it doesn't impact learning in a positive way.
         2. How do we get students to read comments? 
            1. Write comments and give them to groups.  Have groups match the comment with the paper/project.  Explain why.
            2. Vague comments:  5 problems are wrong - find them
            3. Feedback should be more work for the recipient than teachers! Make sure your feedback causes them to do something!!  This means there has to be time allotted to respond to feedback.
            4. Give the same amount of feedback to all students.  How to improve and how to get even better.  "There is no there!"
            5. Goal:  less feedback, high quality feedback, and time to respond to the feedback.
   2. Create students as learners and resources for each other.
      1. When students are involved in their own learning and the learning of others, they have better retention both in the long and short term.
      2. Activating students as owners of their learning (independence and autonomy, increased freedom of choice and personal responsibility):
         1. Doubles the speed in student learning
         2. Idea:  Learning portfolio (shows your journey, effort leads to progress, smart is not something you are, it is something you become.  Looking back reinforces this idea.) – not performance portfolio (latest and best work)-
         3. What did you find + easy / - hard, challenging / Interesting or weird
      3. Training students to pose questions after a piece of learning.  Encourage students to ask questions, “In your group, find out if anyone has a question that you cannot answer in your group.”  Or “Put a question on a slip of paper and then sort questions and answer them.”  “We went over complex material, you must have at least one question.”
         1. **Making students ask questions as they are learning has a lot of research backing it.****
      4. Resources for each other:
         1. Idea:  always have a "pre-flight" check list - buddy is responsible and held accountable for signing off on checklists....

·         Keeping Learning on Track – NWEA resource.