认知心理学语言课件

1认知心理学认知心理学Cognitive PsychologyCognitive Psychology语言Language张亚旭1认知心理学Cognitive Psychology语2为什么研究语言认知.2为什么研究语言认知.3Rich Information in WordsMintGrapheme&phonememm,inin,t tWord length effecteateat treattreatRegularity effectmintmint mint pint pint paintgrapheme-phoneme correspondence,GPC字素表征音素的字母或字母组合3Rich Information in WordsMint4Rich Information in WordsTreatBody or sub-syllabic unitseateatConsistency effecttr treateat,sw,sweateat,gr,greateatclean clean treattreat4Rich Information in WordsTrea5Rich Information in WordsTreatmentMorphemetreat,menttreat,mentLexical decisionspellcungspellcungflurbpairflurbpairthrimnadethrimnade真假真假假真假真假假假假sized sized rakedraked(frequency effect)(frequency effect)5Rich Information in WordsTrea6Morphology 形态学nMorphemes can be added to derive one word from another派生nsad-sadnessnsad-sadlynor to change the inflection of a word(e.g.,number or tense)屈折变化nwalk-he walksnwalk Yesterday he walked6Morphology 形态学Morphemes can b7Rich Information in Words怡怡Semantic&phonetic radicalsRegularity effect抬抬 怡怡 (哪些额外变量需要控制哪些额外变量需要控制?)?)Consistency effect煌煌 护士护士护士护士 vs.面包面包面包面包-护士护士护士护士n n词汇判断（词汇判断（lexical decisionlexical decision）14心理词典的三个层次15汉字阅读中的亚词汇加工n青(semantic)/猜(complex)/波(control)-紫Zhou&Marslen-Wilson,1999,JEP:LMC15汉字阅读中的亚词汇加工青(semantic)/猜(com16动词论元结构Verb Argument Structurenindicates the type of syntactic constituents with which a verb co-occurs,the grammatical roles and thematic roles/题元角色(e.g.,Agent/Agent/施事施事,Recipient/Recipient/受事受事,and Theme/Theme/主题主题)of those constituentsnSusan gave the boy a banana.nPeter repaired a car yesterday.nJohn departed last week.16动词论元结构Verb Argument Struc17Rich Information in SentencesnSyntactic informationnSyntactic category(noun,verb,etc.)nMorphosyntactic informationnFunctional wordsnWord ordern.nSemantic informationnWord meaning17Rich Information in Sentence18Rich Information beyond Sentences她是去年生的孩子AB:她是去年生的孩子老张下午没去开会，他去做手术了AB:老张下午没去开会，他去做手术了Pragmatic information18Rich Information beyond Sent19nPragmatics is a subfield of linguistics which studies the ways in which context contributes to meaningnPragmatics encompasses speech act theory,conversational implicature,talk in interaction and other approaches to language behavior in philosophy,sociology,and linguistics.nIt studies how the transmission of meaning depends not only on the linguistic knowledge(e.g.grammar,lexicon,etc.)of the speaker and listener,but also on the context of the utterance,knowledge about the status of those involved,the inferred intent of the speaker,and so on.nIn this respect,pragmatics explains how language users are able to overcome apparent ambiguity,since meaning relies on the manner,place,time etc.of an utterance.nThe ability to understand another speakers intended meaning is called pragmatic competence.Pragmatic awareness is regarded as one of the most challenging aspects of language learning,and,though it can be taught,often comes only through experience19Pragmatics is a subfield of 20Rich Information beyond SentencesA:我不想和你过了B:他是谁20Rich Information beyond Sent字词/句子/篇章听觉视觉语言理解听阅读语言产生说写字词/句子/篇章听觉视觉22孤立状态下的视觉词汇识别直接通达假设语义语音词形灯/rose22孤立状态下的视觉词汇识别直接通达假设语义语音词形灯/r23孤立状态下的视觉词汇识别间接通达假设语义语音词形灯/rose23孤立状态下的视觉词汇识别间接通达假设语义语音词形灯/r24nDirect-access hypothesisndeep dyslexian ndapdap,ishish,larlarnDual-route hypothesis24Direct-access hypothesis25Semantic Activation in Reading ChineseZhou,X.,&Marslen-Wilson,W.(1999).Phonology,orthography,and lexical semantic activation in reading Chinese.Journal of Memory and Language,41,579-606Zhou,X.,&Marslen-Wilson,W.(2000).The relative time course of semantic and phonological activation in reading Chinese.Journal of Experimental Psychology:Learning,Memory,and Cognition,26,1245126525Semantic Activation in Readi26Lexical Ambiguity Resolution and Word Recognition in Sentence Contextn在去医院看望女朋友之前，小宋先去店里买了杜鹃n郑老伯埋怨屋里一点声音也没有，老伴准备第二天去市场买杜鹃n中心问题n语境作用的时间点n nWhenWhen/howhow/whetherwhether26Lexical Ambiguity Resolution27nMultiple access modelnSelective access modelnReordered access modelnIntegration modeln n公式公式公式公式 公示公示公示公示 攻势攻势攻势攻势 工事工事工事工事 公事公事公事公事 宫室宫室宫室宫室Zhang,Y.,Wu,N.,&Yip,M.(2006).Lexical ambiguity resolution in Chinese sentence reading.In P.Li,L H.Tan,E.Bates,&O.J.L.Tzeng(Eds.),Handbook of Handbook of East Asian PsycholinguisticsEast Asian Psycholinguistics(Vol.1:Chinese,pp.268-278).Cambridge,UK:Cambridge University Press27Multiple access model认知心理学语言课件29Modularity of CognitionnModular theorynFodor,1983;Forster,1979nThe autonomy of the lexical processorlexical processornInteractive theorynMarslen-Wilson&Tyler,1980;Marslen-Wilson&Welsh,1978;McClelland&Elman,1986nThe output of higher subsystems can influence computation of lower subsystemsFodor,J.A.(1983).Fodor,J.A.(1983).The modularity of mind:an essay on The modularity of mind:an essay on faculty psychologyfaculty psychology.Cambridge,Mass.:MIT Press.Cambridge,Mass.:MIT Press29Modularity of CognitionModul30Syntactic parsing and sentence comprehension30Syntactic parsing and senten31The defendant examined.31The defendant examined.32Garden-Path SentencesThe defendant examinedexamined by the lawyer turned out to be unreliable.The defendant that was examinedexamined by the lawyer turned out to be unreliable.The evidence examinedexamined by the lawyer turned out to be unreliable.The evidence that was examinedexamined by the lawyer turned out to be unreliable.32Garden-Path SentencesThe de33Minimal Attachment Principle最小附加原则The defendant examined.Syntax-first modelGarden-path model S SNP VPNP VP NP RRS:sentenceNP:noun phraseVP:verb phraseRR:reduced relative33Minimal Attachment Principle34Minimal Attachment Principle最小附加原则The evidence examined.Syntax-first modelGarden-path model S SNP VPNP VP NP RRS:sentenceNP:noun phrase名词短语VP:verb phrase动词短语RR:reduced relative关系34Minimal Attachment Principle35PP Attachment Ambiguity介词短语附加歧义nVPnThe man painted the doors with new brushes before the festival.nNPnThe man painted the doors with large cracks before the festival.VP VPverb NP PPverb NP NP PP35PP Attachment Ambiguity介词短语36Put the apple on the towel in the box.Tanenhaus,M.K.et al.(1995).Integration of visual and linguistic information in spoken language comprehension.Science,268,1632-163436Put the apple on the towel i认知心理学语言课件38Late Closure PrincipleAfter Mary and John hugged the woman left.38Late Closure PrincipleAfter 39Modularity of Syntactic ProcessingnSerial,syntax-first modelsnFrazier&Fodor,1978;Frazier&Rayner,1982;Friederici,2002nConstraint-based modelsnMacDonald et al.,1994,Psychological ReviewnConcurrent modelnBoland,199739Modularity of Syntactic Proc40Three-Phase Neurocognitive ModelAngela D.Friederici,2002nPhase 1(100-300 ms after the onset of the critical word)nThe initial syntactic structure is built on the basis of the syntactic categorysyntactic category information of a wordnPhase 2(300-500 ms)nLexical-semantic and morphosyntactic processes occur with the goal of thematic role thematic role assignment(assignment(题元角色指派题元角色指派题元角色指派题元角色指派)nPhase 3(500-1000 ms)nThe different types of information are integratedFriederici,A.D.(2002).Towards a neural basis of auditory sentence processing.Trend in Cognitive Sciences,6Trend in Cognitive Sciences,6,7884.“With some 450 citations to date(ISI Web of Science),the 2002 article alone has been cited With some 450 citations to date(ISI Web of Science),the 2002 article alone has been cited once every week on average since it went to press.once every week on average since it went to press.”(Steinhauer&Drury,2012,B&L)(Steinhauer&Drury,2012,B&L)40Three-Phase Neurocognitive M41Concurrent ModelJulie E.Boland,1997nAlthough syntactic constraints,when available,prevent adoption of a plausible,but syntactically ill-formed,interpretationnsemantic analysis is permitted to get ahead of syntactic analysis,assigning thematic roles and developing a provisional interpretation before a syntactic commitment has been madeThe evidence examinedThe evidence examinedBoland,J.E.(1997).The relationship between syntactic and semantic processes in sentence comprehension.Language and Cognitive Processes,12Language and Cognitive Processes,12,423484.41Concurrent ModelJulie E.Bo42Ferreira&Clifton,1986,JML;Trueswell et al.,1994,JML;Clifton et al.,2003,JMLnThe defendant examinedexamined by the lawyer turned out to be unreliable.nThe defendant that was examinedexamined by the lawyer turned out to be unreliable.nThe evidence examinedexamined by the lawyer turned out to be unreliable.nThe evidence that was examinedexamined by the lawyer turned out to be unreliable.42Ferreira&Clifton,1986,JM43咬死猎人的狗咬死猎人的鸡咬死狐狸的狗卖掉猎人的狗VPN1deN243咬死 猎人的狗44n怠慢客人的孩子 偏正n嘱咐患者的家属 动宾n撞倒肖明的车子 偏正或动宾44怠慢客人的孩子 偏正45Limited Syntactic Parallelism in Chinese Ambiguity Resolutionn粉刷|公寓|的|房间|之后，|小王|还|打扫|了|客厅|。n粉刷|脏乱|的|房间|之后，|小王|还|打扫|了|客厅|。n训练|士兵|的|将军|之后，|总司令|發表|了|簡短|演說|。n训练|年轻|的|将军|之后，|總司令|發表|了|簡短|演說|。Hsieh,Y.,Boland,J.E.,Zhang,Y.,&Yan,M.(2009).Limited syntactic parallelism in Chinese ambiguity resolution.Language and Cognitive Processes,24(7/8):1227-1264.45Limited Syntactic Parallelis46Exp.2 First-fixation durations粉刷 公寓 的 房间 之后，小王 还打扫了客厅训练 士兵 的 将军 之后，总司令 发表了简短的演说46Exp.2 First-fixation durat47Exp.2 Gaze durations47Exp.2 Gaze durations48Exp.2 Regression-path durations48Exp.2 Regression-path dura49Exp.2 Probability of first-pass regressions49Exp.2 Probability of first50Exp.2 Total times50Exp.2 Total times51你们这里有没有30的卡卖100的？The mouse was eaten by the cheese.Sentence interpretation may not always be determined by the syntax(e.g.Bever 1970;Townsend and Bever 2001;Ferreira 2003;Kolk et al.2003;Kim and Osterhout 2005;van Herten et al.2005,2006;Ferreira and Patson 2007)51你们这里有没有30的卡卖100的？52A central question in the study of sentence comprehensionnwhether syntactic processes are temporally and/or functionally prior to semantic processesnsuch that semantic interpretation of a sentence must be established on top of the syntactic structure of that sentence52A central question in the st53Syntax first,both temporally and functionally?nYesnserial,syntax-first models(e.g.,Frazier&Fodor,1978;Friederici,2002)nNonconstraint-based modelsnconcurrent modelnunification model(Hagoort,2003,2005)nnon-syntactocentric,dynamic model(Kuperberg,2007)53Syntax first,both temporall54n“During normal on-line processing,semantic integration of words does not take place for words which are not syntactically licensed(have wrong word category)”n(Friederici&Weissenbor,2007,Brain Research,p.54)nFunctional primacy of syntactic(word category)over semantic processesFriederici,A.D.,&Weissenborn,Y.(2007).Mapping sentence form onto meaning:The syntax-semantic interface.Brain Research,1146Brain Research,1146,5058.54“During normal on-line proce55ERP Approach55ERP Approach56ERP Effects of Semantic Processes56ERP Effects of Semantic Proc57575858nHagoort,P.Hald,L.,Bastiaansen,M.,&Petersson,K-M.(2004).Integration of word meaning and world knowledge in language comprehension.Science,304,438-441Hagoort,P.Hald,L.,BastiaanAs agreed upon,Jane was to wake her sister and her brother at five oclock in the morning.But the sister had already washed herself,and the brother had even got dressed.Jane told the brother that he was exceptionally quick/slow.As agreed upon,Jane was to wa61nKutas,M.,&Federmeier,K.D.(2011).Thirty Thirty years and counting:Finding meaning in the years and counting:Finding meaning in the N400 component of the event-related brain N400 component of the event-related brain potential(ERP).potential(ERP).Annual Review of Psychology,62,621-647nKutas,M.,&Hillyard,S.A.(1980).Reading senseless sentence:brain potentials reflect semantic incongruity.Science,207,203-20561Kutas,M.,&Federmeier,K.62ERP Effects of Syntactic ProcessesnSyntactic category violationsn(E)LAN+P60062ERP Effects of Syntactic Pro63Hahne&Friederici,2002Das Eis wurde im gegessen.应该去掉im(The ice cream was in-the eaten.)语法错误63Hahne&Friederici,2002Das64Hahne&Friederici,2002Das Eis wurde im gegessen.(The ice cream was in-the eaten.)64Hahne&Friederici,2002Das65Distinct Brain Responses to Syntactic and Semantic Processes65Distinct Brain Responses to 66(L)AN to Syntactic Category ViolationsnHagoort,Wassenaar,&Brown,2003nCorrectnDe houthakker ontweek de ijdele schroefschroef op dinsdag.n(The lumberjack dodged the vain propeller on Tuesday.)nSyntactic Category ViolationnDe houthakker ontweek de ijdele schroefschroeft t op dinsdag.n(The lumberjack dodged the vain propelled on Tuesday.)66(L)AN to Syntactic Category 676768ELAN or LAN?nModality:Auditory or visualnPresentation ratenVisual contrastnSyntactic category uniqueness point(prefix or suffix)68ELAN or LAN?Modality:Audito69Morphosyntactic Violationsnagreement violations of person,number,gender,or tensenLAN+P600（有的实验发现）ne.g.,Osterhout&Mobley,1995;Silva-Pereyra&Carreiras,2007nP600（其他实验发现）ne.g.,Hagoort,2003;Nevins et al.,200769Morphosyntactic Violationsag70Newman et al.,2007,NeuroImagenViolations of regular past tense inflection evoke a LAN左侧前部负活动n*Yesterday I frown at Billy.规则变化规则变化nBut violations of irregular past tense inflection do not没有LANn*Yesterday I speak to him frankly.不规则变不规则变化化nLAN reflects rule-governed compositional processes at the syntactic level,rather than the syntactic tense violation itselfLAN表示规则变化动词的70Newman et al.,2007,NeuroIm71nLAN can also be evoked by over-regularizations of irregular verbs(Morris&Holcomb,2005)nbringed 71LAN can also be evoked by ov72nLAN has also been explained as reflecting domain-general,rule-governed processing of symbols,including non-linguistic sequencing(Hoen&Dominey,2000,NeuroReport)72LAN has also been explained 73Kim&Osterhout,2005*The meal was devouring饭正在吞咽红色词引发P600,no N400N400反映语义，P600反映语法Kim,A.,&Osterhout,L.(2005).The independence of combinatory semantic processing:Evidence from event-related potentials.Journal of Memory and Language,52,205-225.73Kim&Osterhout,2005*The me认知心理学语言课件75When Semantics Meets Failed Syntaxn跨语言差异75When Semantics Meets Failed 76Functional Primacy in German:a brief reviewnDas Brot wurde gegessen.Correct(The bread was eateneaten)nDer Vulkan wurde gegessen.Semantic(The volcano was eateneaten)语义异常N400nDas Eis wurde im gegessen.Syntactic(The ice cream was in-the eateneaten)句法错误E LAN P600nDas Trschlo wurde im gegessen.Double(The door lock was in-the eateneaten)双重错误无N400Hahne&Friederici,2002,Cognitive Brain Research76Functional Primacy in German77N400 responses to semantic violationsThe volcano was eaten.Hahne&Friederici,2002,Cognitive Brain Research77N400 responses to semantic v78(E)LAN-P600 to syntactic violationsThe ice-cream was in-the eaten.Hahne&Friederici,2002,Cognitive Brain Research78(E)LAN-P600 to syntactic vio79(E)LAN-P600,but no N400 for double violationsThe door lock was in-the eaten.Hahne&Friederici,2002,Cognitive Brain Research79(E)LAN-P600,but no N400 for80nThis finding of no N400 suggests thatnthe failed local phrase structure construction blocks lexical-semantic integrationnsupporting the functional primacy of syntactic category over semantic processesHahne&Friederici,2002,Cognitive Brain Research80This finding of no N400 sugg81No N400 effects were found for double violations,regardless ofnModalityn nAuditoryAuditory(Friederici et al.,2004;Hahne&Friederici,2002,Experiment 1;Hahne&Jescheniak,2001;Isel et al.,2007)n nvisualvisual(Friederici et al.,1999)nSome task demandsn noverall correctness judgmentoverall correctness judgment(e.g.,Hahne&Friederici,2002,Experiment 1;Isel et al.,2007)n nprobe verificationprobe verification(e.g.,Friederici et al.,1999)nunless semantic integration was forced by instructions(in German:Hahne&Friederici,2002,Experiment 2;in English:Thierry et al.,2008)nSyntactic category uniqueness pointn nPrefixPrefix(e.g.,Hahne&Friederici,2002,Experiment 1)n nsuffixsuffix(Friederici et al.,2004)81No N400 effects were found f82Das Buch wurde trotz verpflanzt von(The book was despite replanted by.)nFriederici et al.,2004,NeuroReportnSuffix82Das Buch wurde trotz verpfla83Le fauteuil qui est dans la dort(The chair which is in the is sleeping)nIsel et al.,2007nFrench83Le fauteuil qui est dans la 84Friederici&Weissenborn,2007,Brain Research,p.54n“During normal on-line processing,semantic integration of words does not take place for words which are not syntactically licensed(have wrong word category)”84Friederici&Weissenborn,2085Het vrouwtje veegde de vloer met een oude bedelde gemaakt van twijgen(The woman swept the floor with an old begged made of twigs)nvan den Brink&Hagoort,2004nDutch85Het vrouwtje veegde de vloer86nThierry et al.(2008,NeuroImage)observed an N400 effect for the combined violationn nShakespearean functional shiftShakespearean functional shiftnHowever,their participants were asked to judge sentences for meaningfulness86Thierry et al.(2008,NeuroI认知心理学语言课件88Are there cross-linguistic differences in syntactic primacy?nPossibly yes,and could hinge upon morphology形态学形态学nIn German and some other Indo-European languages,the syntactic category of a word can be marked withngrammatical morphology(e.g.,inflectional or derivational affixes)northographic conventions(such as capitalizing all German nouns)88Are there cross-linguistic d89nIn contrast,Chinese is a non-Indo-European language with very limited inflections(e.g.,plural inflection of human nouns;perfective and progressive markers)nThere is no grammatical morphology that either marks syntactic category or marks syntactic features such as person,number(except plural inflection of human nouns),gender,case,and tensenGrammatical relations in Chinese are not generally cued morphologically;rather they are cued by word order,function words(e.g.,ba PREP),and lexical/semantic content89In contrast,Chinese is a no90nThese linguistic properties of Chinese have lead to the conjecture that sentence comprehension in Chinese relies more on semantic analyses compared to Indo-European languages(e.g.,Xu,1997;see Li,1998;Ye et al.,2006)90These linguistic properties 设计师制作新衣，把布料裁了。Correct 伐木工开采森林，把松树裁了。语义Semantic Violation设计师制作新衣，把裁了。句法 Syntactic Violation伐木工开采森林，把裁了。双重 Combined ViolationYe,Luo,Friederici,&Zhou,2006,Brain Research设计师制作新衣，把布料裁了。Correct 伐木工开清洁工/把/大厦的/窗户/全部/擦/了/一遍。Correct 清洁工/把/大厦的/窗户/全部/赢/了/一遍。Semantic Violation清洁工/把/大厦的/窗户/全部/糖/了/一遍。Combined Violation语法错误，可能不会引发语义矛盾的N400Yu&Zhang,2008,NeuroReport清洁工/把/大厦的/窗户/全部/擦/了/一遍。Correct女孩/把/新到的/长裙/立刻/购买/了/下来。Correct 女孩/把/新到的/长裙/立刻/欺诈/了/下来。Semantic女孩/把/新到的/长裙/立刻/钞票/了/下来。Weak女孩/把/新到的/长裙/立刻/铁锹/了/下来。StrongZhang,Y.,Yu,J.,&Boland,J.E.(2010).Semantics doesnt need a processing license from syntax in reading Chinese.Journal of Experimental Psychology:Learning,Memory,and Cognition,36,765-781.女孩/把/新到的/长裙/立刻/购买/了/下来。Correc94Violation Reference PointsDas Trschlo wurde im gegessen.(The door lock was in-the eaten)94Violation Reference PointsDa95TasknVerification sentence judgment in Weak-Strong experimentnOverall acceptability judgment or other tasks in German and French studies95TaskVerification sentence ju女孩/买/了/裙子/和/手套。Correct女孩/买/了/很/裙子/和/手套。Syntactic句法女孩/吃/了/裙子/和/手套。Semantic语义女孩/吃/了/很/裙子/和/手套。CombinedZhang,Y.,Yu,J.,&Boland,J.E.(2010).Semantics doesnt need a processing license from syntax in reading Chinese.Journal of Experimental Psychology:Learning,Memory,and Cognition,36,765-781.女孩/买/了/裙子/和/手套。Correct女孩/买/97很男人男人很淑女淑女很德国德国无极之后，有人说，陈凯歌不再霸王霸王别姬别姬。97很男人98Discussion&ConclusionnSemantic integration proceeds even when syntactic category processing fails during Chinese sentence reading,resulting in N400 effects for combined violations,even under conditions in which semantic integration is not forcednThere is no language-universal functional primacy of syntactic category over semantic processesnIn order to account for all the cross-linguistic data,a processing model must have a means for semantic integration in the face of severe grammatical violations98Discussion&ConclusionSeman9999Friederici,A.D.(2011).The brain basis of language processing:From structure to function.Physiological Reviews,91Physiological Reviews,91,13571392.Friederici,A.D.(2011).The 101n“A review of the literature on the timing of syntactic information and sema