English subtitles

← 21-02 Language Models

Get Embed Code
3 Languages

Showing Revision 1 created 11/28/2012 by Amara Bot.

  1. We'll start by talking about language models.
  2. Historically, there have been two types of models that have been popular
  3. for natural language understanding within AI.
  4. One of the types of models has to do with sequences of letters or words?
  5. These types of models tend to be probabilistic
  6. in that we're talking about the probability of a sequence,
  7. word based in that mostly what we're dealing with is the surface words themselves,
  8. and sometimes letters.
  9. But we're dealing with the actual data of what we see,
  10. Rather than some underlying extractions,
  11. and these models are primarily learned from data.
  12. Now, in contrast to that is another type of model that you might have seen before,
  13. where we're primarily dealing with trees and with abstract structures.
  14. So we say we can have a sentence, which is composed of a noun phrase and a verb phrase,
  15. and a noun phrase might be a person's name, and that might be "Sam."
  16. And the verb phrase might be a verb and we might say "Sam slept"--
  17. a very simple sentence.
  18. Now, these types of models have different properties.
  19. For one, they tend to be logical rather than probabilistic--
  20. that is whereas on this side, we're talking about the probability of a sequence of words,
  21. on this side we're talking about a set of sentences and that set defines the language,
  22. and a sentence is either in the language or not.
  23. It's a Boolean logical distinction rather than on this side a probabilistic distinction.
  24. These models are based on abstraction such as trees and categories--
  25. categories like noun phrase and verb phrase and tree structures like this
  26. that don't actually occur in the surface form, so the words that we can observe.
  27. An agent can observe the words "Sam" and "slept,"
  28. but an agent can't directly observe the fact that slept is a verb or that it's part of this tree structure.
  29. Traditionally, these types of approaches have been primarily hand-coded.
  30. That is, rather than learning this type of structure from data,
  31. we learn it by going out and having linguists and other experts write down these rules.
  32. Now, these distinctions are not hard to cut.
  33. You could have trees and have a probabilistic model of them.
  34. You could learn trees.
  35. We can go back and forth, but traditionally the two camps have divided up in this way.