Return to Video

21-02 Language Models

  • 0:00 - 0:03
    We'll start by talking about language models.
  • 0:03 - 0:07
    Historically, there have been two types of models that have been popular
  • 0:07 - 0:10
    for natural language understanding within AI.
  • 0:10 - 0:16
    One of the types of models has to do with sequences of letters or words?
  • 0:16 - 0:20
    These types of models tend to be probabilistic
  • 0:20 - 0:24
    in that we're talking about the probability of a sequence,
  • 0:24 - 0:30
    word based in that mostly what we're dealing with is the surface words themselves,
  • 0:30 - 0:33
    and sometimes letters.
  • 0:33 - 0:37
    But we're dealing with the actual data of what we see,
  • 0:37 - 0:39
    Rather than some underlying extractions,
  • 0:39 - 0:44
    and these models are primarily learned from data.
  • 0:44 - 0:50
    Now, in contrast to that is another type of model that you might have seen before,
  • 0:50 - 0:54
    where we're primarily dealing with trees and with abstract structures.
  • 0:54 - 1:01
    So we say we can have a sentence, which is composed of a noun phrase and a verb phrase,
  • 1:01 - 1:07
    and a noun phrase might be a person's name, and that might be "Sam."
  • 1:07 - 1:14
    And the verb phrase might be a verb and we might say "Sam slept"--
  • 1:14 - 1:16
    a very simple sentence.
  • 1:16 - 1:20
    Now, these types of models have different properties.
  • 1:20 - 1:25
    For one, they tend to be logical rather than probabilistic--
  • 1:25 - 1:32
    that is whereas on this side, we're talking about the probability of a sequence of words,
  • 1:32 - 1:40
    on this side we're talking about a set of sentences and that set defines the language,
  • 1:40 - 1:44
    and a sentence is either in the language or not.
  • 1:44 - 1:50
    It's a Boolean logical distinction rather than on this side a probabilistic distinction.
  • 1:50 - 1:57
    These models are based on abstraction such as trees and categories--
  • 1:57 - 2:02
    categories like noun phrase and verb phrase and tree structures like this
  • 2:02 - 2:08
    that don't actually occur in the surface form, so the words that we can observe.
  • 2:08 - 2:12
    An agent can observe the words "Sam" and "slept,"
  • 2:12 - 2:19
    but an agent can't directly observe the fact that slept is a verb or that it's part of this tree structure.
  • 2:19 - 2:25
    Traditionally, these types of approaches have been primarily hand-coded.
  • 2:25 - 2:29
    That is, rather than learning this type of structure from data,
  • 2:29 - 2:35
    we learn it by going out and having linguists and other experts write down these rules.
  • 2:35 - 2:39
    Now, these distinctions are not hard to cut.
  • 2:39 - 2:45
    You could have trees and have a probabilistic model of them.
  • 2:45 - 2:48
    You could learn trees.
  • 2:48 -
    We can go back and forth, but traditionally the two camps have divided up in this way.
タイトル:
21-02 Language Models
Video Language:
English
Team:
Udacity
プロジェクト:
CS271 - Intro to Artificial Intelligence
Duration:
02:54
Amara Bot added a translation

English subtitles

改訂