Gzip compression not working

[kunalmohan]

Logstash information:

Please include the following information:

1. Logstash version -

logstash 8.8.1

2. Logstash installation source - homebrew
3. How is Logstash being run- via commandline
4. How was the Logstash Plugin installed - no separate installation command

JVM (e.g. java -version):

If the affected version of Logstash is 7.9 (or earlier), or if it is NOT using the bundled JDK or using the 'no-jdk' version in 7.10 (or higher), please provide the following information:

1. JVM version (java -version) - openjdk version "20.0.1" 2023-04-18
2. JVM installation source (e.g. from the Operating System's package manager, from source, etc). - homebrew
3. Value of the JAVA_HOME environment variable if set.

OS version (uname -a if on a Unix-like system): Darwin Mac.local 22.4.0 Darwin Kernel Version 22.4.0

Description of the problem including expected versus actual behavior:
When specifying encoding => "gzip" in the config file, the file uploaded to S3 has .gz extension but is just a plain text file. I'm using "json_lines" codec.

[ShaiBarak]

I'm experiencing this as well, also when try to read with spark job i get an error : "java.io.IOException: not a gzip file"
also using codec => "json_lines"

[thom-vend]

Could not reproduce with the config bellow and logstash 8.9.0.

output {
  s3 {
    id => "s3archive"
    region => "${S3_REGION}"
    bucket => "${S3_BUCKET}"
    canned_acl => "private"
    prefix => "${S3_PREFIX}/%{+YYYY}/%{+MM}/%{+dd}/redacted/"
    size_file => 6291456
    time_file => 5
    encoding => "gzip"
    codec => "json_lines"
    temporary_directory => "${TMP_PATH_PREFIX}/${HOSTNAME}"
    validate_credentials_on_root_bucket => "${S3_VALIDATE_CREDS}"
  }
}

Using a docker image from docker.elastic.co/logstash/logstash-oss

json vs json_lines valid gzipped files

file *
codec_json_lines_.txt.gz:    gzip compressed data, original size modulo 2^32 113329455
codec_json_.txt.gz:              gzip compressed data, original size modulo 2^32 111993519

Both decompressed are ~ 110MB files
codec json_lines files got ~52124 lines
codec json got 0 new line character.