


1.1 Symbolic Representation of Language 1.1.1 Symbolic Structures of Language



Category | English | Chinese |
Character | A letter (e.g., 'a', 'b', 'c') has no meaning by itself. | A character (e.g., '水', '火', '学') often has meaning by itself. |
Morpheme | The smallest meaningful unit (e.g., 'un-', 'run', '-ing'). | A character or a combination of characters that carries meaning (e.g., '学' in '学习', '化' in '现代化'). |
Word | A combination of morphemes that forms a complete meaning (e.g., 'running', 'unhappy'). | Usually a compound word combining multiple morphemes (e.g., '电脑' from '电' + '脑'). |

Category | Definition | English Examples | Chinese Examples |
Word | A combination of morphemes forming a meaningful unit. | 'computer', 'running' | '电脑' , '跑步' |
Chunk | A group of words forming a meaningful phrase but not a full sentence. | 'the big house', 'eating lunch' | '漂亮的房子', '吃午饭' |
Sentence | A complete grammatical structure expressing a full thought. | 'I am reading a book.' | '南京市长江大桥' |

Category | Definition | Note |
Sentence | A complete grammatical structure expressing a full thought. | |
Paragraph | A collection of related sentences forming a coherent unit. | Sentences in it are related and not independent. |
Document | A structured collection of paragraphs forming a complete text. | Paragraphs in it are related and not independent. |
Corpus | A large collection of documents used for linguistic or computational analysis. | The documents are generally considered independent. |


1.1 Symbolic Representation of Language 1.1.2 History of Symbolic Language Representation







Theory | View on Surface Expression | View on Underlying Meaning | Key Mechanism of Mapping | Example |
Structuralism (Saussure, 1916) | Words and grammar are part of a structured system of signs. | Meaning comes from differences between signs, not direct reference to reality. | Signifier (word form) ↔ Signified (concept) | The word “dog” means “dog” not because of an inherent property, but because it differs from “cat”. |
Generative Grammar (Chomsky, 1957) | Sentences have surface structure (what we see) and deep structure (abstract rules). | Deep structure captures core meaning, independent of syntax. | Transformational rules convert deep structure → surface structure. | “John saw Mary” vs. “Mary was seen by John” have different syntax but same deep meaning. |
Dependency Grammar (Tesnière, 1959) | Words are connected by dependency relations (head-modifier structure). | Meaning is derived from syntactic relations rather than linear order. | Dependency trees capture who does what to whom. | “The cat chased the mouse” → “chased” is the head, “cat” is subject, “mouse” is object. |
Abstract Meaning Representation (AMR, 2010s) | Surface syntax varies, but core meaning is a graph. | Meaning is a directed graph of concepts and relations. | Sentences are mapped into AMR graphs. | “She wants to buy a car” → (w / want-01 :arg0 (s / she) :arg1 (b / buy-01 :arg1 (c / car))) |
Open Information Annotation (OIA, 2020s) | Language can be expressed as Predicate/Function/Arguemnt structure; | Meaning is relationships among entities and facts. | Sentences are mapped into OIA Graphs; | “She buy a car” → (buy) –arg1->(She) -arg2->(a car) |

1.1 Symbolic Representation of Language 1.1.3 Case Studies













































Type | Description | Example |
Causal Relations (因果关系) | One sentence (cause) leads to another (effect). | “It rained heavily. The streets got flooded.” (Cause → Effect) |
- Result (结果关系) | The second sentence is a direct consequence of the first. | “She studied hard. She passed the exam.” |
- Reason/Explanation (原因关系) | The second sentence explains the first. | “He left early because he was feeling sick.” |
Temporal Relations (时间关系) | One event happens before, after, or at the same time as another. | “She woke up. Then she made coffee.” |
- Sequential (顺序关系) | One event follows another. | “He finished work. Then he went home.” |
- Simultaneous (同时关系) | Two events happen at the same time. | “She was cooking while he was reading.” |
Contrastive Relations (对比关系) | The two segments express opposite ideas. | “She likes tea. He prefers coffee.” |
- Contrast (对照关系) | Highlights a clear opposition. | “She loves winter, but I hate it.” |
- Concession (让步关系) | Acknowledges one idea but presents another contradicting it. | “It was raining, but they still went out.” |
Additive Relations (添加关系) | The second segment adds more information. | “She speaks English. She also knows French.” |
- Elaboration (详细说明关系) | The second sentence expands on the first. | “He bought a car. It’s a red Tesla.” |
- Parallel (并列关系) | The two segments describe similar ideas. | “She enjoys hiking. He likes running.” |
Attribution Relations (引述关系) | One sentence attributes an action or statement to someone. | “The professor said that the exam would be postponed.” |
- Statement (陈述关系) | Reporting an assertion. | “Experts claim that AI will change the world.” |
- Question (提问关系) | Reporting a question. | “She asked if I could help her.” |
Condition & Hypothetical Relations (条件关系) | Expresses a condition that affects an outcome. | “If you study hard, you will pass the exam.” |
- Conditional (条件关系) | Explicit “if-then” condition. | “If it rains, we will cancel the trip.” |
- Hypothetical (假设关系) | A condition that is unlikely or speculative. | “If I were rich, I would travel the world.” |
Purpose & Means Relations (目的与手段关系) | Expresses a goal and the method to achieve it. | “He left early to catch the train.” |
- Purpose (目的关系) | The second sentence explains the goal of the first. | “She trains daily to become a professional athlete.” |
- Means (手段关系) | The second sentence describes how something is achieved. | “He solved the problem by using an equation.” |


1.1 Symbolic Representation of Language 1.1.4 Obtaining Symbolic Representations













Feature | Penn Treebank (PTB) | Universal Dependencies (UD) | OntoNotes |
POS Tagging | ✅ Yes | ✅ Yes | ✅ Yes |
Constituency Trees | ✅ Yes | ❌ No | ✅ Yes |
Dependency Parsing | ❌ No | ✅ Yes | ❌ No |
Named Entities (NER) | ❌ No | ❌ No | ✅ Yes |
Coreference | ❌ No | ❌ No | ✅ Yes |
Semantic Role Labeling | ❌ No | ❌ No | ✅ Yes |
Word Sense Disambiguation (WSD) | ❌ No | ❌ No | ✅ Yes |
Multilingual | ❌ No | ✅ Yes | ✅ Yes (Eng, Chi, Ara) |



























1.1 Symbolic Representation of Language 1.1.5 Suggested Readings







1.2 Symbolic Representation of Knowledge 1.2.1 History of Symbolic Knowledge Representation







Category | Relation Type | Description | Example |
Hierarchical Relations (Taxonomy) | is-a (Hypernymy/Hyponymy) | X is a subtype of Y (generalization/specialization) | Dog is-a Mammal |
part-of (Meronymy/Holonymy) | X is a part of Y | Wheel part-of Car | |
has-a (Possession) | Y possesses X | Bird has-a Beak | |
Lexical Relations | Synonymy | X and Y have similar meanings | Big ↔ Large |
Antonymy | X and Y have opposite meanings | Hot ↔ Cold | |
Homonymy | X and Y have the same form but different meanings | Bank (financial) vs. Bank (riverbank) | |
Polysemy | X has multiple related meanings | Mouth (of a river, of a person) | |
Associative Relations | Causality (Cause-Effect) | X causes Y | Fire causes Smoke |
Entailment | If X is true, then Y must be true | Kill entails Die | |
Related-to | General associative relation | Doctor related-to Hospital | |
Thematic Roles (Case Roles) | Agent | The doer of an action | John (agent) eats an apple |
Patient | The receiver of an action | John eats an Apple (patient) | |
Instrument | The means used for an action | He wrote with a Pen (instrument) | |
Experiencer | The entity experiencing a state | She felt Happy (experiencer) | |
Event-Based Relations | Precondition | X must occur before Y | Study is required for Passing |
Temporal (Before/After) | X occurs before/after Y | Breakfast before Work | |
Subevent | X is a part of Y as an event | Buying includes Paying | |
Linguistic Relations | Coreference | X and Y refer to the same entity | John said he is happy. (he = John) |
Anaphora | X refers back to Y in discourse | Mary saw a dog. It was barking. | |
Ellipsis | Omitted elements understood from context | I like coffee, and she does too. (ellipsis of “likes coffee”) |

Role | Example |
Buyer | Alice |
Seller | Store |
Goods | A book |
Money | $10 |

Theory | View on Surface Expression | View on Underlying Meaning | Key Mechanism of Mapping | Example |
Montague Semantics (Montague, 1970s) | Language is a formal system like logic. | Meaning is truth conditions in a formal logical system. | Lambda calculus and first-order logic assign formal meaning to expressions. | “Every student passed” = ∀x (Student(x) → Passed(x)) in logic. |
Frame Semantics (Fillmore, 1970s) | Words activate semantic frames (structured scenarios). | Meaning is a network of roles (agent, patient, action). | Words are linked to real-world experiences. | “Buy” involves a buyer, seller, goods, and money. |
Knowledge-Based Representations (WordNet, ConceptNet, 1990s–present) | Words are nodes in a network of meanings. | Meaning is defined by relations to other concepts. | Hierarchical & associative links between concepts. | “Dog” → “is-a” → “Mammal”. |

1.2 Symbolic Representation of Knowledge 1.2.2 Knowledge-Based Systems







1.2 Symbolic Representation of Knowledge 1.2.3 Successes of Knowledge-Based Systems























1.2 Symbolic Representation of Knowledge 1.2.4 Limitations of Knowledge-Based Systems













1.2 Symbolic Representation of Knowledge 1.2.5 Introduction to Knowledge Graphs
































1.2 Symbolic Representation of Knowledge 1.2.6 Knowledge Graph Construction




Dimension | Schema-based | Schema-free |
Prior knowledge | Strong (know what you need) | Weak (discover what the text contains) |
Quality profile | Precise, consistent, alignable | Broad, noisy, needs post-processing |
Evolution | Schema changes are costly | Easy to absorb new relations |
Typical pipeline | Define schema → extract → ingest | Open extraction → induce/normalize → ingest |

1.2 Symbolic Representation of Knowledge 1.2.7 Schema-Based Knowledge Extraction






1.2 Symbolic Representation of Knowledge 1.2.8 Named Entity Recognition












1.2 Symbolic Representation of Knowledge 1.2.9 Relation Extraction










1.2 Symbolic Representation of Knowledge 1.2.10 Schema-Free Knowledge Extraction



1.2 Symbolic Representation of Knowledge 1.2.11 Open Information Extraction



Feature | Information Extraction (IE) | Open Information Extraction (OpenIE) |
Predefined Schemas | Requires predefined schemas. | Does not require predefined schemas. |
Relation Expression | Predefined Symbols | Natural phrase from original text |
Scope | Typically domain-specific (e.g., medical, legal, financial). | Domain-independent; works on any text. |
Input Type | Structured or semi-structured text, sometimes annotated data. | Raw unstructured text (e.g., news articles, blogs). |
Extraction | Focuses on extracting specific entities or relations using rules. | Extracts general subject-relation-object triples without prior knowledge. |
Supervision | Often supervised (requiring labeled data for training). | Primarily unsupervised, does not need labeled training data. |
Flexibility | Less flexible; works best with a fixed domain and predefined structure. | Highly flexible; can be applied to a variety of domains. |


















1.2 Symbolic Representation of Knowledge 1.2.12 Taxonomy and Ontology Learning













1.2 Symbolic Representation of Knowledge 1.2.13 Entity Linking










1.2 Symbolic Representation of Knowledge 1.2.14 Knowledge from Other Sources











1.2 Symbolic Representation of Knowledge 1.2.15 Knowledge Graph Refinement




1.2 Symbolic Representation of Knowledge 1.2.16 Knowledge Graph Construction: Case Study




Entity Type | Count |
Persons | ~1,792,308 |
Places | ~748,372 |
Creative Works | ~610,589 |
Organizations | ~345,523 |
Species | ~1,933,436 |
Diseases | ~10,591 |





1.2 Symbolic Representation of Knowledge 1.2.17 Suggested Readings







1.3 Statistical Representation of Language and Knowledge 1.3.1 Why Statistical Representation






1.3 Statistical Representation of Language and Knowledge 1.3.2 Vector Semantics for Words









word1 | word2 | similarity |
vanish | disappear | 9.8 |
behave | obey | 7.3 |
belief | impression | 5.95 |
muscle | bone | 3.65 |
modest | flexible | 0.98 |
hole | agreement | 0.3 |

















1.3 Statistical Representation of Language and Knowledge 1.3.3 Sparse Embeddings











1.3 Statistical Representation of Language and Knowledge 1.3.4 Vector Semantics for Documents



































































1.3 Statistical Representation of Language and Knowledge 1.3.5 Neural Embeddings




1.3 Statistical Representation of Language and Knowledge 1.3.6 Word2Vec




















1.3 Statistical Representation of Language and Knowledge 1.3.7 GloVe





















1.3 Statistical Representation of Language and Knowledge 1.3.8 Embedding Properties

















1.3 Statistical Representation of Language and Knowledge 1.3.9 Knowledge Graph Embedding

































1.3 Statistical Representation of Language and Knowledge 1.3.10 Suggested Readings



1.3 Statistical Representation of Language and Knowledge 1.3.11 Assignment




