Exception occurs if dictionary file has CRLF

[tak7iji]

• Version: Logstash 6.5.1(logstash-filter-translate-3.2.3)

• Operating System: Linux

• Config File (if you have sensitive info, please remove it):
  config file like this:

  input {
    generator {
      lines => [ "100", "200", "300" ]
      count => 1
    }
  }
  filter {
    translate {
      field => "[message]"
      destination => "[dest]"
      dictionary_path => "/root/logstash/config/dic.csv"
    }
  }

  dictionary file:

  100,foo
  200,bar
  300,boo
  

• Steps to Reproduce:
  An exception is raised if there is a CRLF in the dictionary file.
  After downgrading logstash-filter-translate from 3.2.3 to 3.1.0 or replacing CRLF to LF, everything works as expected.

• Logs:
  [2019-02-12T16:37:31,073][ERROR][logstash.pipeline ] Error registering plugin {:pipeline_id=>"main", :plugin=>"#LogStash::FilterDelegator:0x466e9904", :error=>"Translate: Unquoted fields do not allow \r or \n (line 1). when loading dictionary file at /etc/logstash/dic.csv", :thread=>"#<Thread:0x3543a170 run>"}

• Details:
  https://github.com/logstash-plugins/logstash-filter-translate/blob/master/lib/logstash/filters/dictionary/csv_file.rb#L20
  @csv.shift returns this error if line has \r\n as eol.

• Workarounds:

1. replace dictionary file's eol from \r\n to \n
   This workarond is no impact to current code.

2. strip \r\n
   https://github.com/logstash-plugins/logstash-filter-translate/blob/master/lib/logstash/filters/dictionary/csv_file.rb#L19
   @io.string = line
   ⇒ @io.string = line.strip

[dpresbit]

In a related note, in case anyone comes across this:

Spent quite a bit of time today scratching my head with a translate filter / dictionary_path issue. Resolution was to ensure that my keys had a colon with a space immediately after it, before the value was entered.
Contents of YAML File - Dictionary_Path
“1.1.1.1”: Net1
“2.1.1.1”: Net2