Cannot reference a field inside string

[ozturkberkay]

Hi,

I'm trying to reference a configuration field from a string inside message_properties. Instead of the field value, what I get is the literal representation of the string; in this case: %{myfield}.

• Version: Logstash 7.4.2
• Operating System: Windows Server 2016 Standard (1607)
• Config File:

input {
    tcp {
        ...
    }
}

filter {
    if [somefield] == "something" {
        ...
        split {
	    field => "somefield2"
	}
	csv {
	    columns => [ "c1", "c2", "c3", "c4", "c5", "c6", "c7", "c8", "c9", "c10", "c11" ]
	    separator => ","
	}
        if [c11] =~ someregex {
            ...
            mutate {
                rename => { "somefield3" => "myfield" }
            }
           ...
        } else {
            drop {}
        }

    }

}

output {
    if "_jsonparsefailure" in [tags] {
        ...
    } else {
        ...
        rabbitmq {
            ...
            message_properties => {
              "content_type" => "application/json"
              "priority" => 1
              "app_id" => "%{myfield}"
            }
            ...
        }
    }
}

• Sample Data:

{"somefield": "something", "somefield2":"field1,field2,field3,field4,field5,field6,field7,field8,field9,field10,field11", "somefield3": "something"}

• Steps to Reproduce: Try to reference a field from the app_id field.
• Discussion: https://discuss.elastic.co/t/cannot-reference-a-field-inside-string/212276/4

[ozturkberkay]

This is the workaround I used on Windows:

1. Download 7.0.2 release source.
2. Apply output: add support for sprintf templating in message_properties #9 patch manually. Do not overwrite the files directly.
3. Make sure to add the Logstash path to LOGSTASH_PATH environment variable.
4. jRuby -S bundle install
5. jRuby -S gem build logstash-integration-rabbitmq.gemspec
6. cd %LOGSTASH_PATH%
7. bin/logstash-plugin install --no-verify your_gem_file
8. Now I can die in peace.