Wednesday, February 15, 2017
For Thursday, Feb. 15: What are the key results of the Dobao study? What are their implications? What are the study's strengths and weaknesses? To help you evaluate this and other research-based articles, please see the section called "For blog posts on research-based articles" on p. 12 of the syllabus.
Subscribe to:
Post Comments (Atom)
A key finding of Dobao’s (2012) study was that collaboration had a significant influence on accuracy, which she claimed could be due to the number of students and more linguistic resources. Also as I expected, individual writers were able to write more (fluency) but that their accuracy was quite a bit lower than pairs or groups. Another result was that complexity was relatively similar across all three groups, but I suppose this is not surprising as they were all of similar proficiencies. One of the key results I found interesting was that fluency and complexity were not really any different between the groups and the pairs. I was expecting that pairs would have a higher fluency because there were fewer people and perhaps they could have written significantly more. Another result I found interesting was the differences in LRE production between groups and pairs. In each case, there was one group/pair that produced significantly fewer LREs than the average, which might have quite a few implications regarding within group differences. I think this would be interesting to explore further because there might be some other difference that were missed and it might change some claims about collaboration.
ReplyDeleteSimilar to the other study, I think the results might help teachers reconceptualize the use of pair and group work in their classes. To be honest, in my classes pair/group work in writing was mostly done for feedback and I had not considered using it to actually write the document; which I am now eager to test out. I think another important implication of this study is that learning may occur in group collaborative writing in that students are more focused on their language which allowed them to utilize their different knowledge and build on what each can bring.
As with the other study, I found that Dobao was able to take a complex design with many variables and present it in a way that is easy to understand, walking through the data for each result pointing out where certain things occur. But also similar to the other study I think one of the weaknesses was not enough attention to individual variables that may assist with people working collaboratively. However, Dobao’s study did make an effort towards this in that the groups were self-generated meaning that it is possible they were friends and likely to collaborate and agree.
Dobao’s result were very interesting, even if not surprising. When it comes to accuracy, fluency, and complexity (Research question 1) there weren’t many surprises. Text written by groups were more accurate but had fewer numbers of words than those written by pairs or individuals. Individuals had less accurate texts, which I believe is what I was expecting, and what we briefly discussed in class.
ReplyDeleteRegarding the second research question, which dealt with frequency and nature of LREs, Dobao concluded that the number of participants didn’t significantly influenced the focus of the LREs but their frequency and outcome (p.50). I guess this results make sense, given that the focus was similar for all groups: they knew they had to put their attention on the verb forms. Also, because groups had more participants than pairs, it is understandable that groups had more LREs than pairs.
I found Dobao’s description on p. 52 fascinating, on the way novice learners help each other to become “collective experts,” “(…) [they] pool their incomplete L2 knowledge to reach a solution to the language problem encountered and co-construct an utterance that none of them seem to be able to produce on their own.” I also think this is an interesting complementing idea to Storch and Aldosari’s study, the combination of students’ proficiencies presented. Their study gave us little hope for pairing LL students unless it was for the particular purpose of developing fluency.
One question that came to mind while I read the study was if the students were able to use their notes during the activity, or even if the teacher’s notes were still on the blackboard. I don’t think Dobao mentions it, and I understood that students were meant to see this as another activity part of their class, not an evaluation. I wonder if for people working individually, it would make a very big difference, and I imagine it would significantly affect the results of the correctly resolved LREs in groups. Like Storch and Aldosari, Dobao didn’t interview participants before or after the activity. She doesn’t mention asking the teachers for their input either. I wonder if teachers found this activity valuable and continue applying it in the classroom after this experience. I also wonder if the students who participated in this study had a better performance on their formal evaluations on past tenses.
Questions for Brian:
When studying the benefits of using Collaborative writing in the FL classroom, have you found a particular learning situation in which collaborative writing is not advised to be used?
Do you think FL teachers who use collaborative writing should evaluate one single linguistic aspect on the task, or several? (in the case of Spanish, for example, only verb forms, or only gender agreement between noun and adjectives..)
I also wonder how much/ if the writing proficiency and personality types of students should be taken into account.
Dobado’s (2012) key findings are that groups produced a higher quantity of LREs and demonstrated greater accuracy than pairs and individuals. The major implication is that we should be doing more group work in classrooms! Easier said than done. Facilitating group work continues to be one of the most challenging parts of teaching for myself and for the pre-service teachers in my Language and Learning class. An additional implication of this research is that teachers should be better prepared to challenge traditional teaching models and design and implement collaborative tasks.
ReplyDeleteDobado builds on established literature and methods comparing individual work to pair work, and she carves out her own space by extending her research to group work. She divides the literature into two sections so that one topic moves logically into the next as she builds a case for her study:
1) Peer collaboration and L2 learning (socio-cultural theory in general → sociocultural theory in L2 contexts → collaboration/ “Languaging” → LREs → Factors affecting LREs)
2) Collaborative Writing Tasks (task design in L2 → collaborative tasks → benefits of collaborative writing→ key studies)
Her gap statements would be more compelling if she went beyond filling a hole in the literature to linking her research to the real life problems involved in classroom practice. The summary of the literature review attempts to connect writing in FL contexts to learning to write in all contexts, but that statement (p. 43, second full paragraph) doesn’t logically fit into the case she built.
Brian, I wonder if you have identified struggles/ challenges involved with designing and implementing collaborative tasks?
This comment has been removed by the author.
ReplyDeleteCompared to the Storch and Aldosari study, the strength of Dobao (2012) is the organization (design) of the study, in which research questions are clearly described. In addition, both studies referred to previous studies regarding LREs, but Dobao clearly demonstrated the gap between other studies (e.g. mechanical part) and filing the gap by examining specific measures of accuracy, fluency, and complexity of the texts written by the groups, pairs, and individual learners.
DeleteFurthermore, Dobao also considered inter-rater reliability by recruiting two other raters, which resulted in a high rate of reliability. To produce valid and reliable research findings, Dobao conducted comparative analyses between as well as within the variables (e.g. between groups and pairs, groups and individuals, and pairs and individuals as well as each of them) and also adjusted the p-values to 0.01, which reduced the possibilities of making type I errors and at the same time, values less than 0.05 were also considered. The study consists of quantitative and qualitative strands, is organized sequentially, and provides a deeper understanding of the difference between the group and pair interactions.
In terms of fluency, I’m not sure why both studies only took account of the numbers of words. I think the notion of fluency is supported by comprehension. A person’s ability to say many words doesn’t fully represent fluency in writing. The content of the writing should be persuasive and also capture the intended meaning in order to accurately respond to the prompt. In addition, in the table on page 50, the number of participants in a small group (how big is the group?) was hard to identify, but in the description, a group refers to a small group. I wonder whether a group of 3 is enough to represent a group and whether it is meaningful to compare groups to pairs (2 people). In relation to Storch and Aldosari, the efficacy of LREs can be differed by the proficiency of each participant in a group in addition to the number of participants.
Honestly, I like this article very much because it studies one of the neglected areas of collaborative writing: group work. The results that “learners working in groups of four were sometimes able to reach a level of performance that seemed to be not only beyond their individual level of competence, but also beyond that they might have been able to achieve had they been working in pairs” are very meaningful (p. 53). It reflects how knowledge telling or transforming (borrowed words, but have different meanings here) in group work is influenced by the social capital. In a same group, there are people who benefit more and people who benefit less. It is very hard for teachers to balance this unequal knowledge gain. It is only possible for teachers to provide each student with specific recommendations of how to learn from peers and contribute thoughts in a group.
ReplyDeleteOne weakness I noticed is the lack of study of the interaction patterns in pairs and small groups. The author hastily reached a similar conclusion that learners learn the least amount of knowledge when they are in a passive attitude in a dominant-passive group. Unfortunately, there was qualitative or quantitative analysis to specifically explain this finding. It will be awesome for the author to propose several types of interaction occurred in group works.
Another thing I can think of is that I wonder how the research of collaborative writing develops in Iran. In the last sentence of this article, the author pointed out that the task type also influenced the amount of LREs and the type of social interactions. Given that participants are Americans, I am thinking that whether this dictogloss writing is applicable to Americans, not to other groups of speakers. I once worked with an Iranian on a task similar to the dictogloss, I was astonished to find how he struggled with coming up with imaginary ideas. In current academia, many research findings are drawn based on a western context. Researchers from other “minor” countries have a hard time to announce their study results, especially when they write the manuscripts in their native language. I want to know how collaborative writing is viewed in those countries where collaboration or collaborative on a certain type of task (e.g. dictogloss/imaginary/abstract writing) is unusual. Instead of just focusing on the western countries, I want to keep an eye on how collaborative writing (or other areas of research) is developed in a global level.
This comment has been removed by the author.
ReplyDeleteThis comment has been removed by the author.
ReplyDeleteThe main finding is that collaboration, regardless of # of people, resulted in greater accuracy than did individual efforts, due to the fact that group members could pool their knowledge and reach solutions that none of them would have been able to produce on their own (Donato's collaborative scaffolding again). The authors also found that, in terms of fluency, individually-written texts were longer than those written by pairs or groups. One final key finding is that # of participants didn't seem to impact the specific focus of LREs (syntactic, lexical, mechanical), but 4-person groups focused more on language issues overall than pairs and had statistically significant differences in terms of frequency and correctly-resolved outcomes.
ReplyDeleteThe implication of this, from the perspective of the authors is that L2 classroom assignments should include pair and group work activities. I think another important implication is that careful attention must be paid to the problem task in terms of the learning teachers hope it will promote. Mechanical LREs were low, but the task was designed to create discussion around grammatical and lexical issues. The use of a single scribe for the whole group may also have contributed to diminished group attention of aspects of the task related to recording the answers. This study goes into new territory by moving the discussion beyond a comparison of individuals and pairs to include larger groups. One important area for further research will be to see how degree and type of participation in the task impact learning (e.g. do more passive group participants reap similar gains to their more actively-engaged peers?). Like Rocky, I was surprised that # of people wasn't a factor in terms of complexity. I would probably have hypothesized that group writing would be more complex than individual writing, driven by the same mechanism that supported group accuracy.
I think a major strength of this study was the mixed methods sequential explanatory design. Not only did the qualitative strand shed light on the quantitative findings, it also supported them by providing some degree of methodological triangulation on the data. Another strength was the fact that they had a large enough N to justify tests of statistical inference, though I think they're getting cutting it pretty close to be able to analyze pairs versus groups of four. And, they may have selected the courses randomly, there's nothing random about the sampling method. participants self-selected, and as Rocky pointed out, there's no way of knowing what the effect of their pairings was on the outcomes. What if all of the 4-person groups were friends or people who had worked together before while all of the 2-person groups were not? Anyhow, large Ns and true experimental design are hard to come by in qualitative research, so maybe that's quant bias talking). I do wonder whether they could have done a bit more number crunching, however. Averages are vulnerable to outliers, and as they point out, there was quite a range in terms of the data collected within each group. They might have even expanded their qualitative analysis to include questions that might have gotten at those differences, but I suppose that leaves more room for future research, possibly even on this same data set.
So, I've already made a mess in this blog post, but I was still thinking, and there's one amendment to my answer that seems important. I went back to read more carefully, and they did use the Mann-Whitney, which doesn't assume normal distribution, making my question about sample size a lot less important, I think (this from my very limited stats knowledge, so perhaps that's wrong, too!) And I also appreciate Kye's point that they adjusted their p-values to protect against type I errors. Both were good choices, I think.
DeleteDobao’s study was very interesting when I was reading it. She has written about many things one of her result is the text written by the groups obtained better accuracy scores than those written by the pairs on all the measures. I think this is obvious since the larger number of participants has a better chance of coming up with good correct answers and ideas compared to two participants.
ReplyDeleteShe also says that language related episodes are rather frequent in both small group and pair interaction. Although the groups spend slightly more time on task the analysis on LRE’s per minute confirmed that LRE’s where more frequent in small group interaction than in pair interaction.
The weakness of this study is, the students who work in groups and pairs take more time to accomplish their work compared to the students who work individually. I once grouped my students into fours and they took a long time to accomplish work compared to when they work individually.
The strength of this study is helpful in the classroom, because the learners writing in pairs produced linguistically accurate text. As a teacher I have understood how to pair students, put them in groups and how they can work individually. Since am teaching Swahili I will pair my students so that they may give one another equal chance to speak and practice the language.
I think Dobao’s did a great job by researching on the written text produced by pairs, groups and individual’s leaner’s to analyze their accuracy fluency and complexity in writing. Because most researchers based in individuals and pairs.