What
lessons did you learn from this study? How can you apply them to your own
experiences as a teacher and/or writer? What were the study’s strengths and weaknesses (See blog post assignment questions in Appendix.)
OR
Design a Follow-up or Spin-off Study that related to the issue of global and local feedback.
Reading Montgomery and Baker (2007), I realized how important self-assessment is in education. Many times we assumed that teachers know what we’re doing, and we are trying our best. I know it sounds horrible, because, like this study showed, teachers do believe they are doing their best. Sometimes teachers are underpaid and overloaded with work, so it’s not like they are slacking on purpose. Sometimes overcrowded classrooms do not allow time for more detailed planning, let alone self-assessment. This study was an eye-opener for many reasons, but mostly for the insights presented in the Discussion:
ReplyDelete“(…) students perceived receiving more feedback than their teachers perceived giving.” (p.93);
I thought it was interesting to see how, providing feedback that the students understood, was enough for them to consider it enough. It was about the kind and quality of the feedback, not the amount.
“(…) students are content with the amount of feedback on global issues (…)” (p.94);
Considering it was an English class, I am not sure why the researchers were so surprised. Maybe for them the amount of global issues wasn’t enough, but the students’ expectations were not taken into consideration, only what they thought of the feedback already given.
“(…) teachers did not provide the same amount of feedback to each student”
I agree with the authors that this topic should be further researched, I’m just not sure how that could be done (maybe making papers anonymous?). I wonder if the students who received a higher number of feedback comments were perceived differently by the instructors, or if it was simply a matter of the moment when the grading took place. I have been told that it is advisable to try to grade all papers in one sitting, so I am in “the same mood,” and the students get the same “treatment,” (The justification for this is that being in a happy mood may make the teacher see a paper differently than if the teacher is cranky). Should the teachers’ emotions while grading be considered? Or is it plainly a question of preference towards some students?
“(…) in general teachers provided a substantial amount of local feedback and relatively little global feedback throughout the drafts (…)”
I wish the authors had explored differently the data when it came to this finding. They hypothesize students asking for specific help with local issues, which might be what students expect in a L2 writing class, so why not include on their surveys questions about expectations as well? I don’t remember reading specifics about the assignments either (do they specify what kinds of writings students had to complete for their portfolio?) I believe that feedback has to be directly connected to the purpose of the evaluation. This study made me reflect on my feedback practices. I wonder what my former students thought about them, and if they found them helpful.
(disclaimer: sorry, way too long, but no time to edit my wordiness this week. My apologies!)
ReplyDeleteThis study gave me a lot to think about. They had a couple of findings that felt quite firm and important to me, drew a couple of interpretations that seemed particularly worth considering in greater depth, and opened up a lot of fascinating questions for future research. There were also some conclusions they drew that seemed problematic, however, and I also think their calculator might be broken (but I’m not a maths wiz, so maybe it’s my head that needs fixing).
The biggest finding by far is that teachers are far less aware of their practices than they believe. I feel like this has been widely documented in other arenas (I’m working with a dissertation writer right now who has seen math teachers say they are student centered and then proceed to design heavily teacher-centered syllabi), but it sounds like the field for investigating why this happens in composition is wide open for investigation. The article brought up a number of reasons why this might be occurring. One that I found particularly interesting was the idea that teachers might be subconsciously responding to messages they are getting (or believe they are getting) from students. For example, maybe the student who got 210 points of feedback was sending the message that he/she really wanted that level of correction and the teacher was responding to that. I think that in-depth looks at student-teacher interaction around feedback is an imperative next step in this line of research. I also really liked their point that there could be a very different perception of teacher responsibility depending on whether the class is an English language class or a content-based class.
The main conclusions I had issues with were that student reporting on frequency of feedback could be confidently correlated with either student satisfaction or student understanding of/ability to use the feedback. “too little” is not synonymous with “a little” or “none”, and “too much is not an option at all. Also, it’s possible that students get a lot of feedback and have no idea what to do with it. The only support they name for these conclusions are 1) the finding in Leki 2006 that students prefer lots of comments on local issues and 2) their own finding that students were not describing global feedback as “a little” or “None” in spite of the quantitative assessment that there was in fact very little. This seems like far too flimsy evidence for the claim they’re making here. Overall, I think that the issue of feedback is super nuanced, that it relies on so many more factors than just what teachers are trained to do or talk about in meetings. Part of the issue that I’d want to look at is how much of the conventional wisdom around feedback is actually closer to what is called in the writing center world “lore”. Sometimes, lore doesn’t fit with experience on the ground, and tutors may buck the convention with varying degrees of awareness that they are doing so. That could be the case here as well.
I started thinking about factors that influence how I give online feedback. Just a couple of things I consider: time to turnaround (what will the student be able to implement in the time they have before the deadline?); my confidence that I will be able to clearly communicate my suggestion or that the student will be able to utilize it (since this is a one-shot deal, generally, with no opportunity for either the student or myself to check for understanding and/or clarify); and my perception of what students may want (based on what I’ve learned from face-to-face interactions with students and what people write in the online feedback request they send us) or what the teachers might be expecting (which I feel I have less of a grasp of), just to name a couple of issues.
I have a lot to say about this study, but not very much time to stay it! I’m looking forward to talking about this article because this is territory I want to engage in my dissertation.
ReplyDeleteI understand that many L2 students’ studying in the U.S. expect corrective feedback both because it is what they are used to and because there is an economic value attached to an error, but I worry about language framing the L2 experience in deficit when a body of research shows bi/multilingualism to be a cognitive benefit. I worry about emphasis on “error free T units” and language like “L2 writing teachers and their students must deal with writing that contains many error types” (p. 231). I also worry about a writing task that requires a person to continue resubmitting until their work is error-free (p. 234). Authors report that this process “required multiple drafts” (p. 234), but it sounds less like drafting and more like line editing/ error correction.
I’m especially hesitant to accept the researchers’ attempt to understand why declines in accuracy might have occurred in the control group. After all, students who compose more complex sentences or ideas risk higher numbers of errors. Writing accurately may be associated with writing “safely” or limiting word choice or sentence structure to what the writer already knows. Linguistic accuracy is only one of many components of academic writing.
Is “traditional process approach” a thing? Who are they referring to? I disagree that an experimental design is necessary to learn more about corrective feedback. Finally, the passive voice is killing me.
This comment has been removed by the author.
ReplyDeleteThe strengths of this study are that the authors examined both the teachers’ and students’ perception on their feedback and the correlation between the perception and actual feedback that the teachers provide and students take. I also like the idea of investigating how teachers provide global and local feedback to students. While I read this part, I also think that each consultant might be different perception and beliefs regarding global (content and organization) and local (grammar and mechanics) feedback. Perhaps, the consultants believe that the amount of proving global and local feedback can be different from students’ writing proficiency. I also think not only the external factors, such as setting (e.g. classroom vs outside of the classroom) and the medium (online vs. face to face) but also the internal factors (perception and beliefs) can become a factor to produce different results.
DeleteThe previous research in this study has shown how the time (when) and the way (how) of providing each global and local feedback, which can differ throughout writing process (initial, during, or final stage of writing or either first or last drafts). Montgomery and Baker (2007) emphasized more on whether the teachers’ perception of giving feedback are correlated to students’ receiving feedback and also how teacher’s perception correlated to the real feedback they provide. I think this study is well designed in quantizing (using self-assessment) the qualitative value (perceptions) and also provides rich information by comparing the perception and real outcomes in writing feedback, which gives me a great deal of information to frame how the writing feedback study should be like.
Because this study was based on the quantitative research, I felt that the authors tried to support the arguments about the reasons for the results based on the previous studies. As a result, the authors made assumptions why students addressed local errors is crucial to the writing process. I think the interview would provide better inference why teachers equally provide feedback for each student. I also was wonder how teachers’ actual feedback was counted, especially in counting global feedback. I also wonder if the number of providing feedback is crucial when students’ writing piece have major problems in global issues. The way of providing the feedback would affect the quality of writing. In this sense, I don’t know how it is meaningful to compare the number of feedback. One comment that a teacher provided in terms of global issues cannot be counted as one error. I also think each student has different strength and weaknesses in writing so teacher ability to identify the students’ need is much important.
ReplyDeleteI must say that this article is a must-read article in the field of written feedback. I have benefitted a lot from reading it. The study has a very delicate research design, many novel discoveries, and a well-written publication text. I was also impressed by their efforts to make their study more valid by asking for help from statistical advisors.
In terms of the research findings, just by looking at table 4, I can not help myself feeling surprised about those discoveries. Many questions came to my mind while analyzing the data in that table, such as why teachers gave the most amount of feedback to the first drafts of those high pass students, what happened to those pass and low pass students when they revised their first draft, why their second draft received even more feedback, how to explain the most number of feedback exists in the high pass compositions, etc? Also, in terms of the most important findings in this study (from the most amount of feedback to the least amount of feedback: grammar, mechanics, ideas/content, vocabulary, organization), I just kept wondering the reason behind it. Perhaps grammar is the easiest feedback teachers can provide, or grammar is the most frequently covered construct in class??? I agree with the author that teachers should rethink their roles, and how much time and efforts they should give for both product and process. Because students are satisfied with the little feedback they receive on global issues, it should be seen as an alarm for teachers in regard to their pedagogical emphases. Additionally, I was impressed by the further research question that authors proposed at the end: whether students who received global-level feedback could be more successful than students who received local-level feedback. The findings can be used to compare with the current debates of when to offer what type of written feedback to students.
Montgomery and Baker’s (2007) article was interesting because the type of feedback is always something I consider when teaching ESL/EFL and when working with individuals in the writing resource/center. In the writing center I often ask exactly what they want me to look at but I do not necessarily think that I do that in my classroom. This article made me reflect about my own feedback practices and if I had an “accurate” representation of the amount I gave. While I may not have a distinct tool to take into the classroom, I think I may incorporate some reflective practices when I give feedback.
ReplyDeleteAs for the strengths, I found the design of the study interesting in that it tried to gather quantitative data (frequency count) with perhaps a slightly qualitative focus (Likert scale perceptions). I felt like these different data approaches added some depth to the study, but I must admit I also wondered why they chose this combination rather than others.
I think another strength for this study was the possibility for future research that may tell more about the habits of teachers, the considerations they make when giving feedback, and if this varied with the task, the draft number, or the different point in the year. One specific case I would be interested in exploring would be the example of the one teacher who gave 210 comment to one student and zero to another, even though they received the same pass grade. I would be very interested in following her interactions with students and understanding her justifications for the large difference in feedback comments.
While the study did have a few good points, I felt like there were some large concerns with the design and information provided about the study. As for the design, I did not really feel like the study measured what it was intended to measure and I found some of the connections rather irrelevant. Also, I did not feel like the study was explained enough to fully understand. I would have liked more information about the participants, the classes, and the writing tasks.
Another possible weakness could be the Likert scale estimations made in the questionnaire because they may not always be accurate portrayals of what occurred and they can be difficult to quantify (e.g. how much is “some” or “a lot”). I understand why they used the system they did (i.e. participants may feel intimidated) but I am not really sure this is completely justified when trying to answer the question of correction quantity or value.
This comment has been removed by the author.
ReplyDeleteFrom the study, I have leant about teachers self assessment on written feedback. As a teacher I have to assess myself on the feedback I provide to students. The feedback should help students improve their performance in writing. This semester I teach an elementary Swahili class. I gave the students an assignment on watching a Swahili series on line and writing a paragraph about it. Most of the students worked hard to get good performance but, still had many errors. I thought for a while and came up with the idea, that they have not been able to master grammar. I have decided to evaluate my assessment on the grammar feedback I provide to students, so that it can be helpful to them.
ReplyDeleteStudents stated that their teachers were giving none or only a little feedback on global issues. It may suggest that that the teachers are providing feedback in a way that students prefer or understand. In my own experience as a teacher I usually give students feedback in a way that they will understand. For example: I once gave students an exercise of writing sentences using possessives, unfortunately some students wrote correct sentences but did not use possessives. I wrote a comment on the paper telling them that, the sentences are good but possessives are not included. This information might be little but students understand.