So we've talked about a few design patterns
with MapReduce. We've talked about filtering patterns, summarizing
patterns, and structural patterns, but what good are
they? Well, these patterns, can be thought of as
sort of lenses. To try and view your
problems through. If you have a big data problem
and you're trying to do some analysis, that
at its heart is really about filtering, about selecting
from a large data set and getting some smaller sample of that data set. There's
a good chance that you can use this
filtering pattern. Likewise, if the fundamental nature of
your, of your problem is summarizing or
going from a structured to hierarchical sort of
database. In fact, these aren't all the patterns
you might want to use. There's also organizational patterns.
There's patterns that deal with input and output and
there's others. But getting yourself to ask this question, can,
you just have to say. Is this problem I'm dealing
with filtering, or summarizing, or structural, or something else? That
isn't simple. In fact, just identifying when map reduce
is the appropriate tool for a problem is a large
part of what it means to be a MapReduce expert.
And getting this expertise takes practice. This practice can come
naturally in the job environment. Or you could
choose to make it happen deliberately by seeking out
problems and trying to solve them. But, if
you enjoy learning about and thinking about these patterns,
you should check that book, I mentioned earlier,
Mapreduce Design Patterns and you can learn more there.
But now, it's time to try implementing some
of these patterns in, in sort of novel fitting.