Return to Video

Conclusion - Intro to Hadoop and MapReduce

  • 0:01 - 0:03
    So we've talked about a few design patterns
  • 0:03 - 0:08
    with MapReduce. We've talked about filtering patterns, summarizing
  • 0:08 - 0:12
    patterns, and structural patterns, but what good are
  • 0:12 - 0:14
    they? Well, these patterns, can be thought of as
  • 0:14 - 0:17
    sort of lenses. To try and view your
  • 0:17 - 0:20
    problems through. If you have a big data problem
  • 0:21 - 0:23
    and you're trying to do some analysis, that
  • 0:23 - 0:26
    at its heart is really about filtering, about selecting
  • 0:26 - 0:32
    from a large data set and getting some smaller sample of that data set. There's
  • 0:32 - 0:34
    a good chance that you can use this
  • 0:34 - 0:38
    filtering pattern. Likewise, if the fundamental nature of
  • 0:38 - 0:41
    your, of your problem is summarizing or
  • 0:41 - 0:45
    going from a structured to hierarchical sort of
  • 0:45 - 0:47
    database. In fact, these aren't all the patterns
  • 0:47 - 0:51
    you might want to use. There's also organizational patterns.
  • 0:51 - 0:54
    There's patterns that deal with input and output and
  • 0:54 - 0:57
    there's others. But getting yourself to ask this question, can,
  • 0:57 - 1:00
    you just have to say. Is this problem I'm dealing
  • 1:00 - 1:03
    with filtering, or summarizing, or structural, or something else? That
  • 1:03 - 1:07
    isn't simple. In fact, just identifying when map reduce
  • 1:07 - 1:10
    is the appropriate tool for a problem is a large
  • 1:10 - 1:12
    part of what it means to be a MapReduce expert.
  • 1:12 - 1:16
    And getting this expertise takes practice. This practice can come
  • 1:16 - 1:19
    naturally in the job environment. Or you could
  • 1:19 - 1:21
    choose to make it happen deliberately by seeking out
  • 1:21 - 1:23
    problems and trying to solve them. But, if
  • 1:23 - 1:27
    you enjoy learning about and thinking about these patterns,
  • 1:27 - 1:29
    you should check that book, I mentioned earlier,
  • 1:29 - 1:33
    Mapreduce Design Patterns and you can learn more there.
  • 1:33 - 1:35
    But now, it's time to try implementing some
  • 1:35 - 1:38
    of these patterns in, in sort of novel fitting.
タイトル:
Conclusion - Intro to Hadoop and MapReduce
概説:

07-13 Conclusion

more » « less
Video Language:
English
Team:
Udacity
プロジェクト:
ud617 - Intro to Hadoop and Mapreduce
Duration:
01:39

English subtitles

改訂 Compare revisions