Return to Video

Dependencies Encode Relational Structure

  • 0:00 - 0:04
    In this segment I'm going to show you that dependency syntax
  • 0:04 - 0:09
    is a very natural representation for relation extraction applications.
  • 0:11 - 0:17
    One domain in which a lot of work has been done on relation extraction is in the biomedical text domain.
  • 0:17 - 0:19
    So here for example, we have the sentence
  • 0:19 - 0:26
    “The results demonstrated that KaiC interacts rhythmically with SasA, KaiA, and KaiB.”
  • 0:26 - 0:31
    And what we’d like to get out of that is a protein interaction event.
  • 0:31 - 0:35
    So here’s the “interacts” that indicates the relation,
  • 0:35 - 0:37
    and these are the proteins involved.
  • 0:37 - 0:40
    And there are a bunch of other proteins involved as well.
  • 0:41 - 0:48
    Well, the point we get out of here is that if we can have this kind of dependency syntax,
  • 0:48 - 0:55
    then it's very easy starting from here to follow along the arguments of the subject and the preposition “with”
  • 0:55 - 0:59
    and to easily see the relation that we’d like to get out.
  • 0:59 - 1:02
    And if we're just a little bit cleverer,
  • 1:02 - 1:06
    we can then also follow along the conjunction relations
  • 1:06 - 1:13
    and see that KaiC is also interacting with these other two proteins.
  • 1:14 - 1:17
    And that's something that a lot of people have worked on.
  • 1:17 - 1:24
    In particular, one representation that’s being widely used for relation extraction applications in biomedicine
  • 1:24 - 1:28
    is the Stanford dependencies representation.
  • 1:28 - 1:34
    So the basic form of this representation is as a projective dependency tree.
  • 1:34 - 1:41
    And it was designed that way so it could be easily generated by postprocessing of phrase structure trees.
  • 1:41 - 1:44
    So if you have a notion of headedness in the phrase structure tree,
  • 1:44 - 1:50
    the Stanford dependency software provides a set of matching pattern rules
  • 1:50 - 1:55
    that will then type the dependency relations and give you out a Stanford dependency tree.
  • 1:55 - 2:02
    But Stanford dependencies can also be, and now increasingly are generated directly
  • 2:02 - 2:07
    by dependency parsers such as the MaltParser that we looked at recently.
  • 2:07 - 2:11
    Okay, so this is roughly what the representation looks like.
  • 2:11 - 2:13
    So it's just as we saw before,
  • 2:13 - 2:18
    with the words connected by type dependency arcs.
  • 2:20 - 2:24
    But something that has been explored in the Stanford dependencies framework
  • 2:24 - 2:28
    is, starting from that basic dependencies representation,
  • 2:28 - 2:34
    let’s make some changes to it to facilitate relation extraction applications.
  • 2:34 - 2:38
    And the idea here is to emphasize the relationships
  • 2:38 - 2:43
    between content words that are useful for relation extraction applications.
  • 2:43 - 2:45
    Let me give a couple of examples.
  • 2:45 - 2:52
    So, one example is that commonly you’ll have a content word like “based”
  • 2:52 - 2:57
    and where the company here is based—Los Angeles—
  • 2:57 - 3:01
    and it’s separated by this preposition “in”, a function word.
  • 3:01 - 3:07
    And you can think of these function words as really functioning like case markers in a lot of other languages.
  • 3:07 - 3:11
    So it’d seem more useful if we directly connected “based” and “LA”,
  • 3:11 - 3:15
    and we introduced the relationship of “prep_in”.
  • 3:16 - 3:21
    And so that’s what we do, and we simplify the structure.
  • 3:21 - 3:23
    But there are some other places, too,
  • 3:23 - 3:30
    in which we can do a better job at representing the semantics with some modifications of the graph structure.
  • 3:30 - 3:35
    And so a particular place of that is these coordination relationships.
  • 3:35 - 3:40
    So we very directly got here that “Bell makes products”.
  • 3:40 - 3:44
    But we’d also like to get out that Bell distributes products,
  • 3:44 - 3:52
    and one way we could do that is by recognizing this “and” relationship
  • 3:52 - 4:02
    and saying “Okay, well that means that ‘Bell’ should also be the subject of ‘distributing’
  • 4:03 - 4:07
    and what they distribute is ‘products.’”
  • 4:09 - 4:11
    And similarly down here,
  • 4:11 - 4:21
    we can recognize that they’re computer products as well as electronic products.
  • 4:22 - 4:25
    So we can make those changes to the graph,
  • 4:25 - 4:28
    and get a reduced graph representation.
  • 4:29 - 4:33
    Now, once you do this, there are some things that are not as simple.
  • 4:33 - 4:39
    In particular, if you look at this structure, it’s no longer a dependency tree
  • 4:39 - 4:43
    because we have multiple arcs pointing at this node,
  • 4:43 - 4:46
    and multiple arcs pointing at this node.
  • 4:47 - 4:49
    But on the other hand,
  • 4:49 - 4:55
    the relations that we’d like to extract are represented much more directly.
  • 4:55 - 4:58
    And let me just show you one graph that gives an indication of this.
  • 4:59 - 5:06
    So, this was a graph that was originally put together by Jari Björne et al,
  • 5:06 - 5:12
    who were the team that won the BioNLP 2009 shared tasks in relation extraction
  • 5:12 - 5:17
    using, as the representational substrate, Stanford dependencies.
  • 5:17 - 5:21
    And what they wanted to illustrate with this graph
  • 5:21 - 5:25
    is how much more effective dependency structures were
  • 5:25 - 5:31
    at linking up the words that you wanted to extract in a relation,
  • 5:31 - 5:35
    than simply looking for words in the linear context.
  • 5:35 - 5:40
    So, here what we have is that this is the distance
  • 5:41 - 5:46
    which can be measured either by just counting words to the left or right,
  • 5:46 - 5:50
    or by counting the number of dependency arcs that you have to follow.
  • 5:50 - 5:53
    And this is the percent of time that it occurred.
  • 5:53 - 5:56
    And so what you see is, if you just look at linear distance,
  • 5:56 - 6:03
    there are lots of times that there are arguments and relations that you want to connect out
  • 6:03 - 6:06
    that are four, five, six, seven, eight words away.
  • 6:06 - 6:12
    In fact, there’s even a pretty large residue here of well over ten percent
  • 6:12 - 6:17
    where the linear distance away in words is greater than ten words.
  • 6:17 - 6:21
    If on the other hand though, you are trying to identify,
  • 6:21 - 6:26
    relate the arguments of relations by looking at the dependency distance,
  • 6:26 - 6:30
    then what you’d discover is that the vast majority of the arguments
  • 6:30 - 6:35
    are very close-by neighbors in terms of dependency distance.
  • 6:35 - 6:42
    So, about 47 percent of them are direct dependencies and another 30 percent of distance too.
  • 6:42 - 6:49
    So take those together and that’s greater than three quarters of the dependencies that you want to find.
  • 6:49 - 6:52
    And then this number trails away quickly.
  • 6:52 - 6:59
    So there are virtually no arguments of relations that aren’t fairly close together in dependency distance
  • 6:59 - 7:03
    and it’s precisely because of this reason that you can get
  • 7:03 - 7:10
    a lot of mileage in doing relation extraction by having a representation-like dependency syntax.
  • 7:11 - 7:16
    Okay, I hope that’s given you some idea of why knowing about syntax is useful,
  • 7:16 -
    when you want to do various semantic tasks in natural language processing.
Title:
Dependencies Encode Relational Structure
Video Language:
English

English subtitles

Revisions Compare revisions