Catch-all-config opens too many files when services not present.

[bradobro]

We've consistently experienced google-fluentd opening around 40,000 files under Debian Jessie when started with all the catch-all-config *.conf files installed and few of the associated services installed.

Removing all but the few *.conf we need (syslog and a tcp listener) reduces this number to around 3,000 on a quiet machine.

We haven't isolated whether particular *.conf's are the culprit, but suspect that attempting--and re-attempting--to tail non-existent log files might be part of the problem.

The best fix would be for fluentd to detect this situation and stop leaking file handles.

Until then, it seems a good idea to provide the catch-all configs as examples, not automatically installed. They could be put in an examples directory or named *.conf.example.

[mivdnber]

We're noticing this issue too on Ubuntu 16.04 LTS. google-fluentd seems to be openingthread.rb up to 150,000 times with the catch-all config. By keeping only the following config files, we reduced that number to 172:

• cassandra.conf
• redis.conf
• elasticsearch.conf
• jenkins.conf
• syslog.conf
• tomcat.conf

This is all with latest google-fluentd at the time of writing (1.5.27-1)

[jkohen]

@igorpeshansky can you take a look, as you had some thoughts about the catch-all config? I'm surprised it causes the agent to open thread.rb so many times, since it's clearly not a log file.