As much as we'd like to pretend otherwise, research with humans is messy. We're not combining chemicals or observing microscopic organisms in a highly controlled lab environment. We can select our participants carefully, but we can't prevent the individual variation that may be introducing confounding variables as uncontrollable as how much sleep the subjects got the night before, what they did earlier that day or have planned for later, or personal issues that might result in performance quite unlike it would be normally.
In education research, for instance studying language learning and/or instruction, we can work around this complexity or work within it. If we work around it, we tend to avoid actual classrooms, creating artificial learning environments with highly controlled content presentation (e.g. learning words from a pre-recorded video or a grammatical concept from a computer program). Unfortunately, that trades control for realism, or ecological validity. Can we really say "X works better than Y" when the learning context was so far removed from how X would be put into practice in real life? This kind of set-up also leaves little room for extended posttesting (good luck getting everyone to come back in for a 6-month follow-up), so gains might be overstated. Or we can eschew the before and after of the learning process altogether in favor of cross-section studies examining highly restricted linguistic elements (e.g. grammaticality judgments, self-paced reading). In fact, this is similar to what I did with my dissertation research, examining verbal fluency across L2 proficiency levels. But while we can get a glimpse of how learners at different levels behave--which is valuable for understanding bilingual processing--there's no good way to tie that knowledge to a particular classroom practice. The longer a student has studied a language, the more eclectic a range of teaching styles they've been exposed to, so to answer a question like "What classroom activity should we add/eliminate?" results in pure speculation. If we're dedicated to applied classroom research, we're forced to work within the limitations. One of those limitations is the sheer number of potential treatments/interventions. We can compare method A to the control, but that just tells us about the efficacy of A compared to doing nothing. What about how A compares to B or C or D or N? Educators want to know not just what technique is better than nothing, they want to know what is the most effective. To set up a study that would explore all the potential variables is basically impossible. To have a sufficiently large sample requires lots of participants, either randomly assigned to different groups (but they still might be different in some way) or presented with all the conditions in counterbalanced order (and if there are too many conditions and too many stimuli, participants might quit or zone out). And ideally all the participants should interact with the same instructor, otherwise it's not just participant individual variation but instructor variation that might pose a problem. Of course, if only the one particular trained instructor is capable of enacting the interventions properly, we're headed towards problems with ecological validity again. The scale of the undertaking required to produce a genuinely generalizable result is mind-boggling, and unlikely in the still relatively guarded, individualistic realm of liberal arts research. Basically, there is no perfect option, and there's always a sacrifice for every gain. We decide which trade-off is acceptable for the questions we're asking and continue on, fueled by fierce curiosity and optimism. Or maybe that's just me.
0 Comments
Here is the reference information for the articles I cite in the various posts. It will be updated as needed.
Bloom, K.C., & Shuell, T.J. (1981). Effects of massed and distributed practice on the learning and retention of second-language vocabulary. The Journal of Educational Research, 74(4), 245-248. Hartshorne, J.K., Tenenbaum, J.B., & Pinker, S. (2018). A critical period for second language acquisition: Evidence from 2/3 million English speakers. Cognition, 177, 263-277. Iacozza, S., Costa, A., and Duñabeitia, J.A. (2017). What do your eyes reveal about your foreign language? Reading emotional sentences in a native and foreign language. PLoS ONE, 12(10), 1-10. Kroll, J.F. & Stewart, E. (1994). Category interference in translation and picture naming: Evidence for asymmetric connections between bilingual memory representations. Journal of Memory and Language, 33, 149-174. Liu, F., Sulpizio, S., Kornpetpanee, S., & Remo, J. (2017). It takes biking to learn: Physical activity improves learning a second language. PLOS One, 12(5), e0177624. Valdés, G. Multilingualism. Linguistic Society of America. http://www.linguisticsociety.org/ resource/multilingualism |
AuthorThis is a place where I record thoughts on second language research and pedagogical theory Archives
June 2019
Categories |