Revert "Migrate to zlib-ng, part 3: Remove zlib and zlib-intel source code an…" (#105284)

This reverts commit 4d2d3d3.

Author: carlossanlop

THIRD-PARTY-NOTICES.TXT

@@ -66,6 +66,38 @@ shall not be used in advertising or otherwise to promote the sale,
 use or other dealings in these Data Files or Software without prior
 written authorization of the copyright holder.
 
+License notice for Zlib
+-----------------------
+
+https://github.com/madler/zlib
+https://zlib.net/zlib_license.html
+
+/* zlib.h -- interface of the 'zlib' general purpose compression library
+  version 1.3.1, January 22nd, 2024
+
+  Copyright (C) 1995-2022 Jean-loup Gailly and Mark Adler
+
+  This software is provided 'as-is', without any express or implied
+  warranty.  In no event will the authors be held liable for any damages
+  arising from the use of this software.
+
+  Permission is granted to anyone to use this software for any purpose,
+  including commercial applications, and to alter it and redistribute it
+  freely, subject to the following restrictions:
+
+  1. The origin of this software must not be misrepresented; you must not
+     claim that you wrote the original software. If you use this software
+     in a product, an acknowledgment in the product documentation would be
+     appreciated but is not required.
+  2. Altered source versions must be plainly marked as such, and must not be
+     misrepresented as being the original software.
+  3. This notice may not be removed or altered from any source distribution.
+
+  Jean-loup Gailly        Mark Adler
+  [email protected]          [email protected]
+
+*/
+
 License notice for zlib-ng
 -----------------------
 

docs/workflow/requirements/linux-requirements.md

@@ -47,14 +47,15 @@ Install the following packages for the toolchain:
 * liblttng-ust-dev
 * libssl-dev
 * libkrb5-dev
+* zlib1g-dev
 * ninja-build (optional, enables building native code with ninja instead of make)
 
 **NOTE**: If you have an Ubuntu version older than 22.04 LTS, or Debian version older than 12, don't install `cmake` using `apt` directly. Follow the note written down below.
 
 ```bash
 sudo apt install -y cmake llvm lld clang build-essential \
-python-is-python3 curl git lldb libicu-dev liblttng-ust-dev \
-libssl-dev libkrb5-dev ninja-build
+  python-is-python3 curl git lldb libicu-dev liblttng-ust-dev \
+  libssl-dev libkrb5-dev zlib1g-dev ninja-build
 ```
 
 **NOTE**: As of now, Ubuntu's `apt` only has until CMake version 3.16.3 if you're using Ubuntu 20.04 LTS (less in older Ubuntu versions), and version 3.18.4 in Debian 11 (less in older Debian versions). This is lower than the required 3.20, which in turn makes it incompatible with the repo. For this case, we can use the `snap` package manager or the _Kitware APT feed_ to get a new enough version of CMake.
@@ -97,12 +98,13 @@ Install the following packages for the toolchain:
 * libicu-devel
 * openssl-devel
 * krb5-devel
+* zlib-devel
 * lttng-ust-devel
 * ninja-build (optional, enables building native code with ninja instead of make)
 
 ```bash
 sudo dnf install -y cmake llvm lld lldb clang python curl git libicu-devel openssl-devel \
-  krb5-devel lttng-ust-devel ninja-build
+  krb5-devel zlib-devel lttng-ust-devel ninja-build
 ```
 
 ### Gentoo

eng/install-native-dependencies.sh

@@ -24,13 +24,13 @@ case "$os" in
             apt update
 
             apt install -y build-essential gettext locales cmake llvm clang lldb liblldb-dev libunwind8-dev libicu-dev liblttng-ust-dev \
-                libssl-dev libkrb5-dev
+                libssl-dev libkrb5-dev zlib1g-dev
 
             localedef -i en_US -c -f UTF-8 -A /usr/share/locale/locale.alias en_US.UTF-8
         elif [ "$ID" = "fedora" ]; then
-            dnf install -y cmake llvm lld lldb clang python curl libicu-devel openssl-devel krb5-devel lttng-ust-devel
+            dnf install -y cmake llvm lld lldb clang python curl libicu-devel openssl-devel krb5-devel zlib-devel lttng-ust-devel
         elif [ "$ID" = "alpine" ]; then
-            apk add build-base cmake bash curl clang llvm-dev krb5-dev lttng-ust-dev icu-dev openssl-dev
+            apk add build-base cmake bash curl clang llvm-dev krb5-dev lttng-ust-dev icu-dev zlib-dev openssl-dev
         else
             echo "Unsupported distro. distro: $ID"
             exit 1

src/installer/pkg/THIRD-PARTY-NOTICES.TXT

@@ -65,6 +65,38 @@ shall not be used in advertising or otherwise to promote the sale,
 use or other dealings in these Data Files or Software without prior
 written authorization of the copyright holder.
 
+License notice for Zlib
+-----------------------
+
+https://github.com/madler/zlib
+https://zlib.net/zlib_license.html
+
+/* zlib.h -- interface of the 'zlib' general purpose compression library
+  version 1.2.13, October 13th, 2022
+
+  Copyright (C) 1995-2022 Jean-loup Gailly and Mark Adler
+
+  This software is provided 'as-is', without any express or implied
+  warranty.  In no event will the authors be held liable for any damages
+  arising from the use of this software.
+
+  Permission is granted to anyone to use this software for any purpose,
+  including commercial applications, and to alter it and redistribute it
+  freely, subject to the following restrictions:
+
+  1. The origin of this software must not be misrepresented; you must not
+     claim that you wrote the original software. If you use this software
+     in a product, an acknowledgment in the product documentation would be
+     appreciated but is not required.
+  2. Altered source versions must be plainly marked as such, and must not be
+     misrepresented as being the original software.
+  3. This notice may not be removed or altered from any source distribution.
+
+  Jean-loup Gailly        Mark Adler
+  [email protected]          [email protected]
+
+*/
+
 License notice for Mono
 -------------------------------
 

src/native/external/cgmanifest.json

@@ -41,6 +41,26 @@
       },
       "DevelopmentDependency": false
     },
+    {
+      "Component": {
+        "Type": "git",
+        "Git": {
+          "RepositoryUrl": "https://github.com/madler/zlib",
+          "CommitHash": "51b7f2abdade71cd9bb0e7a373ef2610ec6f9daf"
+        }
+      },
+      "DevelopmentDependency": false
+    },
+    {
+      "Component": {
+        "Type": "git",
+        "Git": {
+          "RepositoryUrl": "https://github.com/intel/zlib",
+          "CommitHash": "6160a8f20c3626aec2f8b0fda5bf2e65bfe31781"
+        }
+      },
+      "DevelopmentDependency": false
+    },
     {
       "Component": {
         "Type": "git",

src/native/external/patches/zlib-intel/0001-Make-zlib-intel-compile-clean-against-C4244.patch

@@ -0,0 +1,75 @@
+From edabaf799fd071a328e0adb743a98628df6649f0 Mon Sep 17 00:00:00 2001
+From: Levi Broderick <[email protected]>
+Date: Mon, 28 Aug 2023 15:26:38 -0700
+Subject: [PATCH] Make zlib-intel compile clean against C4244 clang equivalent
+ is "implicit-int-conversion" warning
+
+The change to deflate.c is legal because 'len' has an upper bound of
+MAX_STORED, which means it fits cleanly into a 16-bit integer. So
+writing out 2x 8-bit values will not result in data loss.
+
+The change to trees.c is legal because within this loop, 'count' is
+intended to have an upper bound of 138, with the target assignment
+only executing if 'count' is bounded by 4. Neither the 'count' local
+in isolation nor the addition that's part of the target line is
+expected to result in integer overflow. But even if it did, that's a
+matter for a different warning code and doesn't impact the correctness
+of the narrowing cast being considered here.
+
+The change to slide_sse.c is legal because 'w_size' is limited to
+1 << 15 (see deflateInit2_ in deflate.c), so this fits cleanly into
+a 16-bit value.
+---
+ src/native/external/zlib-intel/deflate.c   | 8 ++++----
+ src/native/external/zlib-intel/slide_sse.c | 2 +-
+ src/native/external/zlib-intel/trees.c     | 2 +-
+ 3 files changed, 6 insertions(+), 6 deletions(-)
+
+diff --git a/src/native/external/zlib-intel/deflate.c b/src/native/external/zlib-intel/deflate.c
+index bd5e95774a6..108b1a187af 100644
+--- a/src/native/external/zlib-intel/deflate.c
++++ b/src/native/external/zlib-intel/deflate.c
+@@ -1553,10 +1553,10 @@ local block_state deflate_stored(s, flush)
+         _tr_stored_block(s, (char *)0, 0L, last);
+ 
+         /* Replace the lengths in the dummy stored block with len. */
+-        s->pending_buf[s->pending - 4] = len;
+-        s->pending_buf[s->pending - 3] = len >> 8;
+-        s->pending_buf[s->pending - 2] = ~len;
+-        s->pending_buf[s->pending - 1] = ~len >> 8;
++        s->pending_buf[s->pending - 4] = (Bytef)len;
++        s->pending_buf[s->pending - 3] = (Bytef)(len >> 8);
++        s->pending_buf[s->pending - 2] = (Bytef)~len;
++        s->pending_buf[s->pending - 1] = (Bytef)(~len >> 8);
+ 
+         /* Write the stored block header bytes. */
+         flush_pending(s->strm);
+diff --git a/src/native/external/zlib-intel/slide_sse.c b/src/native/external/zlib-intel/slide_sse.c
+index 342fd562dd1..eb74202c5a0 100644
+--- a/src/native/external/zlib-intel/slide_sse.c
++++ b/src/native/external/zlib-intel/slide_sse.c
+@@ -18,7 +18,7 @@ void slide_hash_sse(deflate_state *s)
+     unsigned n;
+     Posf *p;
+     uInt wsize = s->w_size;
+-    z_const __m128i xmm_wsize = _mm_set1_epi16(s->w_size);
++    z_const __m128i xmm_wsize = _mm_set1_epi16((short)s->w_size);
+ 
+     n = s->hash_size;
+     p = &s->head[n] - 8;
+diff --git a/src/native/external/zlib-intel/trees.c b/src/native/external/zlib-intel/trees.c
+index 35462a1313a..f78b7d8c63e 100644
+--- a/src/native/external/zlib-intel/trees.c
++++ b/src/native/external/zlib-intel/trees.c
+@@ -717,7 +717,7 @@ local void scan_tree(s, tree, max_code)
+         if (++count < max_count && curlen == nextlen) {
+             continue;
+         } else if (count < min_count) {
+-            s->bl_tree[curlen].Freq += count;
++            s->bl_tree[curlen].Freq += (ush)count;
+         } else if (curlen != 0) {
+             if (curlen != prevlen) s->bl_tree[curlen].Freq++;
+             s->bl_tree[REP_3_6].Freq++;
+-- 
+2.42.0.windows.1
+

src/native/external/patches/zlib/0001-Make-zlib-compile-clean-against-C4244.patch

@@ -0,0 +1,57 @@
+From 86d96652ddd60f61dc7b0c94b601f6d156d34632 Mon Sep 17 00:00:00 2001
+From: Levi Broderick <[email protected]>
+Date: Mon, 28 Aug 2023 15:26:38 -0700
+Subject: [PATCH] Make zlib compile clean against C4244 clang equivalent is
+ "implicit-int-conversion" warning
+
+The change to deflate.c is legal because 'len' has an upper bound of
+MAX_STORED, which means it fits cleanly into a 16-bit integer. So
+writing out 2x 8-bit values will not result in data loss.
+
+The change to trees.c is legal because within this loop, 'count' is
+intended to have an upper bound of 138, with the target assignment
+only executing if 'count' is bounded by 4. Neither the 'count' local
+in isolation nor the addition that's part of the target line is
+expected to result in integer overflow. But even if it did, that's a
+matter for a different warning code and doesn't impact the correctness
+of the narrowing cast being considered here.
+---
+ src/native/external/zlib/deflate.c | 8 ++++----
+ src/native/external/zlib/trees.c   | 2 +-
+ 2 files changed, 5 insertions(+), 5 deletions(-)
+
+diff --git a/src/native/external/zlib/deflate.c b/src/native/external/zlib/deflate.c
+index d2e1106ef5d..b7636639754 100644
+--- a/src/native/external/zlib/deflate.c
++++ b/src/native/external/zlib/deflate.c
+@@ -1738,10 +1738,10 @@ local block_state deflate_stored(s, flush)
+         _tr_stored_block(s, (char *)0, 0L, last);
+ 
+         /* Replace the lengths in the dummy stored block with len. */
+-        s->pending_buf[s->pending - 4] = len;
+-        s->pending_buf[s->pending - 3] = len >> 8;
+-        s->pending_buf[s->pending - 2] = ~len;
+-        s->pending_buf[s->pending - 1] = ~len >> 8;
++        s->pending_buf[s->pending - 4] = (Bytef)len;
++        s->pending_buf[s->pending - 3] = (Bytef)(len >> 8);
++        s->pending_buf[s->pending - 2] = (Bytef)~len;
++        s->pending_buf[s->pending - 1] = (Bytef)(~len >> 8);
+ 
+         /* Write the stored block header bytes. */
+         flush_pending(s->strm);
+diff --git a/src/native/external/zlib/trees.c b/src/native/external/zlib/trees.c
+index 5f305c47221..8a3eec559e5 100644
+--- a/src/native/external/zlib/trees.c
++++ b/src/native/external/zlib/trees.c
+@@ -721,7 +721,7 @@ local void scan_tree(s, tree, max_code)
+         if (++count < max_count && curlen == nextlen) {
+             continue;
+         } else if (count < min_count) {
+-            s->bl_tree[curlen].Freq += count;
++            s->bl_tree[curlen].Freq += (ush)count;
+         } else if (curlen != 0) {
+             if (curlen != prevlen) s->bl_tree[curlen].Freq++;
+             s->bl_tree[REP_3_6].Freq++;
+-- 
+2.42.0.windows.1
+

src/native/external/zlib-intel-version.txt

@@ -0,0 +1,16 @@
+v1.2.13_jtk
+(6160a8f20c3626aec2f8b0fda5bf2e65bfe31781)
+
+https://github.com/intel/zlib/releases/tag/v1.2.13_jtk
+
+Note: This library does not use a standard release lifecycle.
+      We're pointing to the latest available tagged version as of this writing.
+
+We do not pull in every file from the intel/zlib repo. Instead, some years ago,
+somebody determined what they believed was the minimum set that needed to be
+copied into our repo in order to build. Since then, we've just updated only
+those files in-place, ignoring other files in the intel/zlib repo. If new files
+are introduced which are necessary for building the product, feel free to bring
+those down as well.
+
+We have also applied the custom patches under the patches/zlib-intel folder.

src/native/external/zlib-intel/README.txt

@@ -0,0 +1,11 @@
+Zlib-Intel is an optimized version of Zlib developed by Intel to take advantage of modern CPU features. 
+
+Brief Listing of Changes:
+New deflate_medium() strategy for compression levels 4-6
+Improved hashing using the SSE4.2 CRC32 instruction
+SSE Saturated Subtractions for hash table shifting
+Faster CRC32 calculations using SSE4.2 PCLMULQDQ instruction
+Reduced loop unrolling
+
+Detailed explanations of these optimizations available in the official whitepaper: http://www.intel.com/content/dam/www/public/us/en/documents/white-papers/zlib-compression-whitepaper-copy.pdf
+