In this week, Fluentd v0.10.43 has been released. Since this version, Fluentd introduced log_level parameter in Input / Output plugin. It enables you can set different log level separated from global log level, e.g. -v, -q command line option.

This article shows “How to support log_level option in your plugin.”

log_level option use cases

Disable in_tail warning

in_tail prints “pattern no match” warning when receives invalid log. It is useful information for almost users, but some users want to ignore this log for other important plugin warning.

In this case, you can set “log_level error” in in_tail configuration to disable “pattern no match”.

1
2
3
4
5

<source>
  type tail
  ...
  log_level error
</source>

Debugging

Without log_level, we get many verbose logs using -vv command line option for one plugin. With log_level, you can set verbose configuration in only one plugin.

It is useful for debugging a plugin on acutual environment.

1
2
3
4
5

<match foo.**>
  type unstable_plugin
  ...
  log_level trace
</match>

Support log_level option in your plugin

This section is for plugin developers.

First of all, Fluentd provides $log object as heretofore. So all plugin should work without changing on Fluentd v0.10.43 or later.

To suppot log_level is very easy. Replace $log with log. Following example is fluent-plugin-td’s diff:

1
2

-        $log.debug "checking whether table '#{key}' exists on Treasure Data"
+        log.debug "checking whether table '#{key}' exists on Treasure Data"

Support older Fluentd versions

After replaced $log with log, your plugin only works on Fluentd v0.10.43 or later. If you want to support older Fluentd versions, you can use following code in your plugin.

1
2
3
4
5
6
7
8
9

module Fluent
  module FooPluginOutput < Output
    # Define `log` method for v0.10.42 or earlier
    unless method_defined?(:log)
      define_method(:log) { $log }
    end
    ...
  end
end

This code defines log method using $log when log method is not defined, so log.error is same as $log.error on older Fluentd.

fluent-plugin-td uses same approach.

Conculusion

log_level feature is very useful for Fluentd users. So if you have a time, please apply above changes and release new version plugin ;)

I will release new version of several plugins soon, S3, TD, Mongo and etc.