二语具身语言加工中视觉信息和文本信息互动研究

本文是一篇语言学论文，本研究选取英语专业研究生和义务教学阶段初中生分别 30 名为研究对象，采用反应时测试工具 E-prime 2.0 进行实验，并借助 SPSS 20.0 软件对实验所得数据进行统计和结果分析，结合模块在线语言发展和使用框架(MOGUL)的指导来考察具身语言加工过程中，语内模块的文本信息加工时出现的“动作-句子兼容效应” 在语内、语外模块之间的启动和传递情况，探究语外模块的视觉信息在加工时类似效应“动作-图片兼容效应” 的启动情况。

Chapter One Introduction

1.1 Research Background
Embodied cognition has always been a hot topic in second language comprehension in recent years. Languag comprehension studies mainly focus on how languages convey meanings. Traditional Amodal Theory states that conceptual meanings of languages are stored in human’s memories in an abstract, amodal and arbitrary way, that are symbols (Fodor, 2000; Burgess & Lurd, 1997; Pinker, 1994; Chomsky, 1980). However, the later researches showed that the Amodal Theory could not solve the grounding problem (Harnad, 1990). So the Embodied Cognition Theory came into being which suggests that human’s comprehensions of languages are not merely a processing of a number of abstract symbols, but an embodied comprehending procedure involving body actions (Finker-Kiefer, 2001; Stanfield & Zwaan, 2001; Glenberg & Robertson, 2000, 1999; Barsalou, 1999; Glenberg, 1997). The Action-sentence Compatibility Effect (hereinafter refers to the ACE) is a considerably new sub-theory based on the Embodied Cognition Theory. It was pointed out by Glenberg and Kaschak in their study “Grounding language in action” (Glenberg & Kaschak, 2002). The ACE suggests that a participant will response faster to a sentence which contains action in direction if he makes a same direction action in reality. That is to say, if a participant makes a same psychological characterization of the action contained in a given sentence, he will have a faster comprehending of the given sentence. Later on, many researchers did a great number of further studies under Glenberg and Kaschak’s proposal of the ACE. They changed the research methods, the types of research materials and etc. to make further findings of the ACE. But is the phenomenon of the ACE only exists in the processing of textual information? Whether does the ACE exist in the processing of non-textual information which can be concluded as information from extra-lingual module? The present study argues that the ACE may expend to other fields rather than particularly exists in textual information from intra-lingual module.
........................

1.2 Research Scope and Significance of the Thesis
Public research attentions and focuses have been put on the Embodied Cognition Theory for years, during which the ACE gains plenty of concerns. Up to now, researchers who get interested in the ACE mainly keep their eyes on the intra-lingual language materials when they do relevant studies, that is textual information. A lot of studies have provided empirical evidences to the ACE which suggests that readers tend to have quicker responses to action-contained sentences when their actual action directions are consistent with those in given sentences. All results of those studiesindicate that language comprehensions and real actions have interactive influences to each other. However the research objects of those studies are mainly different types of English sentences, which seem to be simplex in the intra-lingual module. The simplicity draw some controversial voices to the ACE. Thus, the present study has an intention to improve the persity of the experimental objects when relative researches are done about the ACE and furthermore to verify a wider applicability of the ACE.
Besides, the present study also promotes the development of the Modular On-line Growth and Use of Language. MOGUL is a theoretical framework which inspects the development of inpidual as well as holistic language from the view of language processing. Therefore, the MOGUL is actually not a concrete model or a theory itself. As it is so, the MOGUL has a considerable large flexibility and can absorb many kinds of theories in a certain extent. The present study tries to take advantage of the flexibility of the MOGUL and to combine it with the ACE to inspect the relationship between visual structure and conceptual structure. What’s more, the present study also tends to verify that the ACE can transfer to the extra-lingual module from intra-lingual module and starts a similar effect because of the on-line interactive processing of the MOGUL, and so as to expand the coverage and the explanation ability of the MOGUL.
..........................

Chapter Two Literature Review

2.1 Studies on Embodied Cognition Theory
Among the researches of language comprehension, the establishment and development of the conception of Embodied Cognition Theory is a comparatively new and important procedure in the contemporary western psychology research field. Embodied Cognition Theory states that the essential core of cognition is strongly connected with the participated body. The feelings and senses of body make an important impact on the conceptual processing of language. Embodied Cognition Theory emphasizes that the procedure of conceptual characterization of language is not exactly like what the traditional cognition theories have said. Traditional cognition theories believe that the conceptual characterization is just a procedure of processing abstract and arbitrary symbols which can form languages. Those symbols are stored in our minds. And when conceptual characterization occurs, those symbols are selected and processed under some certain kinds of grammar structures and rules to reach the aim of language comprehension (Collins & Loftus, 1975). For example, if the word “man” is to be described, a series of symbols can be used. “Man [Human + Male + Adult]”. According to the traditional cognition theories, the whole procedure of conceptual characterization has no connection with the body’s sensory system. Thus the eventual comprehensions of one same concept, no matter who the participant is,are going to be the same. However, this conclusion soon appealed a lot of disagreements. Take the direction words for example, if the word “north” is to be described, “Opposite to the south” is used. While at the same time, “Opposite to the north” is used to describe the word “south”. From this example, the loop statement problem and the grounding problem are obviously to be seen (Harnad, 1990). These two words use each other to describe themselves, but none of their distinct features are shown in both descriptions. In this kind of situation, the conceptual characterization is just a meaningless loop among abstract symbols which doesn’t make any sense. On the contrary, the Embodied Cognition Theory states that the procedure of conceptual characterization is actually based on the body experience in real environments. Our brains contain our minds and are in our bodies, while our bodies are in some certain kinds of real environments. Any changes in the environments will have an influence on our bodies, thus will also have an impact on our minds, which actually is the impact on the procedure of conceptual characterization.
.........................

2.2 Studies on the Action-sentence Compatibility Effect
The ACE was put up with by Arthur M. Glenberg and Michael P. Kaschak in their study “Grounding language in action” (Glenberg & Kaschak, 2002). The main point of the ACE is that a reader will have a quicker reaction and a faster comprehension of a given sentence which contains a certain action with direction ifthe reader acts a same direction action. The results of their study finely indicated that language was not merely a serious of abstract symbols and language comprehension was not merely a relatively stable extraction and processing of those abstract symbols. As a matter of fact, language comprehension has a close relationship with the participated body and action. Sentences can express actions through verbs, no matter concrete verbs or abstract verbs. Both concrete verbs and abstract verbs can indicate certain motor behaviors which contain certain directions. Take the following sentences as examples.
(1)a Close the door.
(1)b Open the door.
(2)a You sent a message.
(2)b You got a message.
The words “Close” and “Open” in example (1)a and (1)b are all concrete verbs. And there is only one character participate in both described situations. When a reader tries to understand the meanings of those verbs, he will first make a conceptual characterization and then try to stimulate the motor pattern in his own mind. Once the reader is immersed in the described situation, he will produce understandings of the verbs that “Close” refers to an action with an away-from-body direction and “Open” refers to an action with a towards-body direction. The words “sent” and “got” in example (2)a and (2)b are all abstract verbs. In those two sentences there are two participated characters. One is the dominant “You” and the other is the recessive “message” receiver. And once the reader is immersed in the described situations, he will also produce understandings of the verbs that “sent” refers to an action with an away-from-body direction and “got” refers to an action with a towards-body direction. This kind of comprehending pattern was specifically explained in the experiments of Glemberg and Kaschak’s study (Glemberg & Kaschak, 2002).
..........................
Chapter Three Theoretical Backgrounds .............................. 14
3.1 Connections between Different Language Modules ......................... 14
3.2 Interaction and Restrain of Multimentional Worlds ......................... 16
3.3 Summary ............................. 17
Chapter Four Empirical Study ................... 18
4.1 Research Questions ............................... 18
4.2 Overview of the Experiments ..................... 18
4.3 Method ...................... 19
Chapter Five Discussion ....................... 37
5.1 The Interaction of Visual-textual Information .................. 38
5.2 BOI in Processing of Visual Information ....................... 40
5.3 Hierarchical Restrain and Strategy Selection in Information Processing ....... 41

Chapter Five Discussion

5.1 The Interaction of Visual-textual Information
Since 2002, many scholars have studied the ACE on the basis of the study of Glenberg and Kadschak (2002). They further expended the original study materials. But most of the in-depth researches of the ACE are based on the textual materials within the intra-lingual module. There are few researches involving the extra-lingual module, and even less involving the intermediate zone between the intra-lingual module and the extra-lingual module. Therefore the present study did an empirical study to see what the situation is in this intermediate zone and whether the ACE has a similar reflection in the extra-lingual module. And the APCE the present study came up with is verified in our experiments, which expanded the research scope of the ACE. Throughout the whole study, the present study found that there is an interaction between the textual information and the visual information.
Visual information belongs to extra-lingual module while textual information belongs to intra-lingual module. The interaction between visual information and textual information occurs on the on-line information transmitting through interfaces among different language systems. Visual information comes into intra-lingual module through the interface between visual system and syntactic system. Textual information comes directly into syntactic system. These two kinds of information are processed and combined together and then transmitted through interfaces to the conceptual system to form final comprehension (see figure 5.1).

...................

Chapter Six Conclusion

6.1 Major Findings
Throughout the empirical experiments, the present study founded an interaction between intra-lingual module and extra-lingual module of language, based on the connections among different language systems. This interaction offers opportunities for the ACE to transfer into the intermediate zone between the syntax system and the visual system. And the transfer of the ACE touches off a similar effect in the visual system which is named the APCE in the present study. The APCE suggests that a reader will have a faster comprehension of the given visual information if he acts a same direction motion as the pictured one. The function of the APCE is very like that of the ACE in the processing of textual information. Therefore the finding of the APCE supports and explores the development of embodied language comprehension.
reference(omitted)