Data streams (ex: logs) are becoming larger, while analytics in (semi-) real time is also becoming more important today. We want to collect huge data sets from many sources, and analyze these data in various way to gain valuable business insights. For these purposes, software on jvm (Hadoop, Flume, Storm, ...) works well, but we (Rubyists!) need more Ruby-like, scriptable, ease-to-deploy and extensible software. That is Fluentd.

I'll introduce Fluentd, a famous log collector in Japan, and its plugin systems. Fluentd is a very simple and well-designed software that have various plugins to collect/convert/aggregate/write stream data, which are already used in many production environments. And I will also talk about Fluentd's development plans, newly implemented features and case studies in LINE.