Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

date filter fails to parse "/" when using locale #112

Open
mostolog opened this issue Oct 16, 2017 · 1 comment
Open

date filter fails to parse "/" when using locale #112

mostolog opened this issue Oct 16, 2017 · 1 comment

Comments

@mostolog
Copy link

As described below, date filter doesn't seem to properly implement http://www.joda.org/joda-time/key_format.html when using locale.

  • Version: 6.0.6rc-1
  • Operating System: Ubuntu 16.04 (docker-ce 17.09)
  • Config File:
        ...
	mutate {
		add_field => {
			"date" => "16/Oct/2017:16:27:26 +0200"
			}
		}
	date {
		timezone => "Europe/Madrid"
		match => ["date", "dd'/'MMM'/'yyyy':'HH':'mm':'ss' 'Z"]
		locale => "es"
		tag_on_failure => "error_date"
	}
        ...
  • Steps to Reproduce:
    The above configuration should parse field date, however it doesn't and tags "error_date"
    According to a few Internet searches, it could be happening due to using locale+"/", although with the usage of delimiters ( ' ) it shouldn't be an issue.

elastic/logstash#8491

@mostolog
Copy link
Author

Further info:

logstash -e "input { stdin { type => stdin } } filter { mutate { add_field => { \"date\" => \"16/Oct/2017:16:27:26 +0200\" } } date { timezone => \"Europe/Madrid\" match => [\"date\", \"dd'/'MMM'/'yyyy':'HH':'mm':'ss' 'Z\"] locale => \"es\" tag_on_failure => \"error_date\" } } output { stdout { codec => rubydebug } }"

shows:

Sending Logstash's logs to /logstash/logs which is now configured via log4j2.properties
[2017-10-16T14:37:49,972][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"fb_apache", :directory=>"/logstash/modules/fb_apache/configuration"}
[2017-10-16T14:37:49,979][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"netflow", :directory=>"/logstash/modules/netflow/configuration"}
[2017-10-16T14:37:50,000][INFO ][logstash.setting.writabledirectory] Creating directory {:setting=>"path.queue", :path=>"/logstash/data/queue"}
[2017-10-16T14:37:50,002][INFO ][logstash.setting.writabledirectory] Creating directory {:setting=>"path.dead_letter_queue", :path=>"/logstash/data/dead_letter_queue"}
[2017-10-16T14:37:50,059][INFO ][logstash.agent           ] No persistent UUID file found. Generating new UUID {:uuid=>"07f88ca8-907a-4d9e-a1cb-7fb1c199966d", :path=>"/logstash/data/uuid"}
[2017-10-16T14:37:53,046][INFO ][logstash.pipeline        ] Starting pipeline {"id"=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>5, "pipeline.max_inflight"=>500}
[2017-10-16T14:37:53,292][INFO ][logstash.pipeline        ] Pipeline main started
The stdin plugin is now waiting for input:
[2017-10-16T14:37:53,479][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}

Writing "a" on input results in:

{
          "date" => "16/Oct/2017:16:27:26 +0200",
    "@timestamp" => 2017-10-16T14:38:06.448Z,
      "@version" => "1",
          "host" => "7d0847e692f9",
       "message" => "a",
          "type" => "stdin",
          "tags" => [
        [0] "error_date"
    ]
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant