[pull] master from curl:master #597
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![pull[bot]](https://avatars.githubusercontent.com/in/12910?s=60&v=4){kind=small}
Enable TLS Early Data for wolfSSL: - merge WOLFSSL_CTX and WOLFSSL setup from ngtcp2 with the general implemenation in wolfssl.c - enable for QUIC via ngtcp2 - give Curl_vquic_tls_init() a `struct alpn_spec` like used for the TCP case. Adapt gnutls and other users. - enable pytest test cases for early data with wolfSSL and while this messes up wolfssl.c anyway, do - rename all struct/functions with prefix 'wolfssl_' to 'wssl_' to not pollute that name prefix - rename `ctx/handle` to `ssl_ctx/ssl`, as used in openssl case Closes #16167
Now using 'sort' for sorting the names. This has the small side-effect that it sorts slightly different than the previously used sort function (emacs). I think this is a better sort and over all it makes it more convenient to use the script as it removes a manual step. Closes #16448
Give contrithanks.sh the option to send the list of names to stdout so that delta can use it appropriately for counting. Closes #16449
Support multi-target cmake builds via `CURL_DIRSUFFIX` env. For example:
`export CURL_DIRSUFFIX=Debug/`.
Multi-target generators place their output to `src/<subdir>/`,
`lib/<subdir>/`, `tests/server/<subdir>`, `tests/libtest/<subdir>` and
`tests/unit/<subdir>/` by default. Before this patch, `runtests.pl`
couldn't run on such builds because it expected the binaries under the
their `<subdir>`-less directories. This patch allows to set such subdir
and make `runtests.pl` find the binaries. In CI we use multi-target
builds with tests for MSVC. It also helps Xcode-generator builds, though
in CI we don't have such job running tests.
There may be better solutions to configure this, but passing a custom
value to `runtests.pl` including its subprocesses is somewhat tricky.
The reason the configuration value expects the slash at the end is
because MSYS is automagically expanding the env to a (wrong) absolute
path if the slash is in the front.
Also:
- drop the `-DCMAKE_RUNTIME_OUTPUT_DIRECTORY_*=` workaround from CI.
- replace `resolve` references in tests with a new `%RESOLVE` variable.
It didn't use a filename extension before. After this patch it uses
`exe_ext('TOOL')`. I'm not sure if this is the correct choice vs.
`exe_ext('SRV')`.
- fix `-c` option format in manual.
- fix some whitespace.
Note, in CI we still tweak `CMAKE_RUNTIME_OUTPUT_DIRECTORY_*` in jobs
which share steps between `./configure` and cmake. It's easier that way.
Ref: #15000
Cherry-picked from #16394
Closes #16452
curl requires Windows XP since 2023. Drop version detection code using `GetVersionEx()` aimed to support earlier Windows versions. With that call deleted, the embedded manifest in `curl.rc` becomes unnecessary. Delete it too, along with the enabler logic in build systems. This allows to stop forcing `/MANIFEST:NO` for MSVC builds. Dropping it fixes VS2008 shared builds, that require an auto-generated SxS (side-by-side assembly) manifest to find their CRT DLLs. This was the issue that prevented VS2008 `curl.exe` launching on AppVeyor CI: ``` src/curl.exe: error while loading shared libraries: ?: cannot open shared object file: No such file or directory ``` Ref: https://ci.appveyor.com/project/curlorg/curl/builds/51577006/job/eitypvwlb1rxr11d#L261 FWIW the `curl.rc` embedded manifest wasn't ever enabled for VS2008 CI builds either, because CMake did not pass our custom macro via `CMAKE_RC_FLAGS` to `rc.exe`. For reasons I could not figure out. After this patch the curl build no longer inject its own manifest, and lets the default be applied by linkers and toolchains. It fixes VS2008 shared builds. curl continues to detect the real Windows version via `RtlVerifyVersionInfo()` from `ntdll`. Follow-up to 960d601 #12225 Follow-up to 5044909 #7810 Follow-up to ebd2132 #1221 Ref: #15972 Cherry-picked from #16394 Closes #16453
Static CRT crashes MSVCR* MSVC builds (in VS2008, VS2010, VS2012,
VS2013) according to CI and local tests. The reproducible crash happens
in `curl_mfprintf() -> fputc(s, stderr)` when trying to display the
warning message in `curl -V`. `stderr` is non-NULL and resolves to `2`.
This reproducer needs a debug-enabled build, but it's unrelated to debug
features or curl's memory tracker. It happens regardless of unity build,
CPU architecture or `DllMain()` use. Example from VS2013:
```
+ _bld/src/Debug/curl.exe --disable --version
./appveyor.sh: line 124: 203 Segmentation fault "${curl}" --disable --version
```
Ref: https://ci.appveyor.com/project/curlorg/curl/builds/51570451/job/ojpdqrsm1hmpmq6a#L210
Another crash happened in an UCRT build (VS2017) with a couple of
`printf()`s added to curl's `main()` function:
```
Microsoft Visual C++ Runtime Library
Debug Assertion Failed!
Program: C:/projects/curl/bld/src/Debug/curl.exe
File: minkernel/crts/ucrt/src/appcrt/heap/debug_heap.cpp
Line: 996
Expression: _act_first_block == header
```
(it hangs the job in CI due to the GUI popup)
Ref: #16394 (comment)
To avoid actual and potential issues, this patch issues a warning on
the shared-libcurl + static-CRT combination and falls back to the
default, shared CRT. IOW a static CRT build now requires a static curl
exe when using the `CURL_STATIC_CRT=ON` option.
Follow-up to 4fc6ebe #1621
Cherry-picked from #16394 (with more details there)
Closes #16456
- appveyor: restore VS2008 job, after fixing its issues. Enable OpenSSL in it. It takes 1 minute. Follow-up to 9b0467b #16453 Follow-up to edfa537 #16456 - appveyor: make a copy of OpenSSL DLLs to have them picked up as an artifact (disabled by default) to aid local tests. - appveyor: dump CMake configuration logs on failure. - appveyor: tidy up job parameter defaults. - GHA/windows: add pre-fill check option for dl-mingw jobs. - GHA/windows: fix pre-fill check option for MSYS jobs by installing `diffutils`. Follow-up to e7adf3e #15841 - GHA/windows: de-duplicate to `PATH` commands for Cygwin. - GHA/windows: drop `$SYSTEMROOT/System32` from `PATH` for Cygwin configure. It's not needed. Follow-up to 36fd2dd #13599 - list `.pdb` files in curl version step for MSVC. Ref: #16439 Cherry-picked from #16394 Closes #16458
An attempt to use better language Closes #16461
Drop the `getpart` test tool from standard builds. This tool was not used by tests. Also: - make it easier to build it standalone for testing `getpart.c` on the command-line. - reduce local var scopes in source. Closes #16460
This allows you to use the `certs` and `num_certs` writeout variables in the curl tool, and getting information about the server certificates using CURLINFO_CERTINFO. Closes #16459
It implies -O used for each URL. Mention in the --url documentation. Test 488 and 489 verify. Closes #16099
Fixes (GHA/linux, AWS-LC jobs): ``` test 0307 SKIPPED: curl lacks OpenSSL support test 0308 SKIPPED: curl lacks OpenSSL support [...] ``` Ref: https://github.com/curl/curl/actions/runs/13511134270/job/37751473424#step:42:104 Follow-up to 34ef4fa #10320 Closes #16466
This PR updates the CMake build/install docs in `docs/INSTALL-CMAKE.md`, in particular focusing on the use of libcurl from CMake using `find_package` as well as the newly added features/protocols support via using `COMPONENTS` or `OPTIONAL_COMPONENTS` with `find_package`. See #15854 for initial discussion and the corresponding PR #15858 that was merged. Some additional best-practices notes are added, for example: * Encouraging building out-of-source * Using `--config` with `cmake --build` for multi-config CMake generators, not `CMAKE_BUILD_TYPE` We also add a CURL CMake-specific tip on using `CMAKE_INSTALL_PREFIX` during configure time to set the install prefix, not using `--prefix` when running `cmake --install` so `curl-config` output is consistent. Closes #16329
To save configuration time. After this patch, for mingw-w64 and MSVC curl's CMake builds pre-fill almost all type sizes without auto-detection. In most cases this leaves 3 type size auto-detections. Those depend on 64/32-bitness, and `time_t` also depends on CRT and custom options. Old mingw-w64 versions require some extra detections. We recommend v3.0 or newer to avoid them. For Windows CE, this patch pre-fills all type sizes. If this is causing any issue, please report it and disable pre-filling with `-D_CURL_PREFILL=OFF` in the meantime. Cherry-picked from #16394 Closes #16464
Before this patch, building tests/server (or curl with winbuild) was broken in rare builds when many features were explicitly disabled. Fix it by enabling base64 functions unconditionally when building for anything other than libcurl. Closes #16691
…ixes
Before this patch, standard `E*` errno codes were redefined on Windows,
onto matching winsock2 `WSA*` error codes, which have different values.
This broke uses where using the `E*` value in non-socket context, or
other places expecting a POSIX `errno`, e.g. file I/O, threads, IDN or
interfacing with dependencies.
Fix it by introducing a curl-specific `SOCKE*` set of macros that map to
`WSA*` on Windows and standard POSIX codes on other platforms. Then
verify and update the code to use `SOCKE*` or `E*` macro depending on
context.
- Add `SOCKE*` macros that map to either winsock2 or POSIX error codes.
And use them with `SOCKERRNO` or in contexts requiring
platform-dependent socket error codes.
This fixes `E*` uses which were supposed be POSIX values, not `WSA*`
socket errors, on Windows:
- lib/curl_multibyte.c
- lib/curl_threads.c
- lib/idn.c
- lib/vtls/gtls.c
- lib/vtls/rustls.c
- src/tool_cb_wrt.c
- src/tool_dirhie.c
- Ban `E*` codes having a `SOCKE*` mapping, via checksrc.
Authored-by: Daniel Stenberg
- Add exceptions for `E*` codes used in file I/O, or other contexts
requiring POSIX error codes.
Also:
- ftp: fix missing `SOCKEACCES` mapping for Windows.
- add `SOCKENOMEM` for `Curl_getaddrinfo()` via `asyn-thread.c`.
- tests/server/sockfilt: fix to set `SOCKERRNO` in local `select()`
override on Windows.
- lib/inet_ntop: fix to return `WSAEINVAL` on Windows, where `ENOSPC` is
used on other platforms. To simulate Windows' built-in `inet_ntop()`,
as tested on a Win10 machine.
Note:
- WINE returns `STATUS_INVALID_PARAMETER` = `0xC000000D`.
- Microsoft documentation says it returns `WSA_INVALID_PARAMETER`
(= `ERROR_INVALID_PARAMETER`) 87:
https://learn.microsoft.com/windows/win32/api/ws2tcpip/nf-ws2tcpip-inet_ntop#return-value
- lib/inet_ntop: drop redundant `CURL_SETERRNO(ENOSPC)`.
`inet_ntop4()` already sets it before returning `NULL`.
- replace stray `WSAEWOULDBLOCK` with `USE_WINSOCK` macro to detect
winsock2.
- move existing `SOCKE*` mappings from `tests/server` to
`curl_setup_once.h`.
- add missing `EINTR`, `EINVAL` constants for WinCE.
Follow-up to abf80aa #16612
Follow-up to d69425e #16615
Bug: #16553 (comment)
Closes #16621
Include more sources in unity mode to optimize libtest and tests/server builds for non-debug-enabled builds, syncing this pattern with `lib` and `src`. It reduces build steps from 62 to 47 (-14, -24%) with test bundles. Without test bundles, from 680 to 642 (-38, -6%). Follow-up to de0693f #16274 Follow-up to 3efba94 #14765 Cherry-picked from #15000 Closes #16695
Add an extra guard for the function and variable declarations to avoid
redundant redeclaration warnings when including this header multiple
times. This can happen in unity builds when including it again after
`curl_memory.h`.
Fixes:
```
bld/tests/server/CMakeFiles/servers.dir/Unity/unity_0_c.c
In file included from lib/mprintf.c:32,
from bld/tests/server/CMakeFiles/servers.dir/Unity/unity_0_c.c:7:
lib/memdebug.h:52:14: error: redundant redeclaration of ‘curl_dbg_logfile’ [-Werror=redundant-decls]
52 | extern FILE *curl_dbg_logfile;
| ^~~~~~~~~~~~~~~~
In file included from tests/server/resolve.c:50,
from bld/tests/server/server_bundle.c:7,
from bld/tests/server/CMakeFiles/servers.dir/Unity/unity_0_c.c:4:
lib/memdebug.h:52:14: note: previous declaration of ‘curl_dbg_logfile’ with type ‘FILE *’
52 | extern FILE *curl_dbg_logfile;
| ^~~~~~~~~~~~~~~~
[...]
lib/memdebug.h:110:17: error: redundant redeclaration of ‘curl_dbg_fclose’ [-Werror=redundant-decls]
110 | CURL_EXTERN int curl_dbg_fclose(FILE *file, int line, const char *source);
| ^~~~~~~~~~~~~~~
lib/memdebug.h:110:17: note: previous declaration of ‘curl_dbg_fclose’ with type ‘int(FILE *, int, const char *)’
110 | CURL_EXTERN int curl_dbg_fclose(FILE *file, int line, const char *source);
| ^~~~~~~~~~~~~~~
```
Ref: https://github.com/curl/curl/actions/runs/13822010778/job/38669360980#step:39:55
Cherry-picked from #15000
Closes #16696
```
In file included from server_bundle.c:7:
../../../tests/server/resolve.c:110:5: error: unknown type name 'curl_socket_t'; did you mean 'curl_socklen_t'?
curl_socket_t s = socket(PF_INET6, SOCK_DGRAM, 0);
^~~~~~~~~~~~~
curl_socklen_t
../../../include/curl/system.h:392:38: note: 'curl_socklen_t' declared here
typedef CURL_TYPEOF_CURL_SOCKLEN_T curl_socklen_t;
^
In file included from server_bundle.c:7:
../../../tests/server/resolve.c:111:13: error: use of undeclared identifier 'CURL_SOCKET_BAD'
if(s == CURL_SOCKET_BAD)
^
```
Ref: https://github.com/curl/curl/actions/runs/13825438937/job/38679418428?pr=15000#step:14:47
Cherry-picked from #15000
Closes #16700
Linux AM openssl https-only: ``` configure: WARNING: unrecognized options: --disable-rtmp, --disable-scp, --disable-sftp ``` Ref: https://github.com/curl/curl/actions/runs/13823209634/job/38673119106#step:31:34 macOS AM clang !ssl HTTP-only: ``` configure: WARNING: unrecognized options: --disable-rtmp, --disable-scp, --disable-sftp, --without-ntlm-auth ``` Ref: https://github.com/curl/curl/actions/runs/13823209638/job/38673115560#step:7:54 Closes #16701
After restricting OpenSSH-Windows to a single job, and bumping it to the pre-release version, that job started hanging then timing out with reasonable consistency. Since we saw similar hangs before with OpenSSH-Windows stable, in all jobs, drop OpenSSH-Windows from CI, and replace it with MSYS openssh. After this patch, all Windows jobs use MSYS2 or Cygwin openssh. Follow-up to 0ec72c1 #16672 Closes #16704
Since this is only used for object and function pointers now. Closes #16709
CODEx, DATAx, CLEANx, DECLx, REM3, CHKRET are all removed now. Closes #16709
The condition could not happen, as the function is only called from a single place where the caller already made sure it can't happen. This change still removes the flawed logic. Reported-by: Ronald Crane Closes #16710
Allowing 4GB on a 32-bit system is just asking for problems and could in theory cause integer overflow in the dynbuf code. The dynbuf now has an assert to catch code trying to set a max larger than half SIZE_T_MAX. Reported-by: Rinku Das Closes #16716
When committed in 2004, it served as a developer helper tool while `coreutils` was yet missing a `base64` command. Assisted-by: Dan Fandrich Closes #16713
Sync it with cmake to: - exclude it from all builds except Windows and Cygwin. - exclude it from unity builds for Cygwin to avoid the included `windows.h` header interfere with the rest of the code. Also: - fix to trim ending spaces from `CSOURCES` for the `tidy` target. The solution requires a non-POSIX `-E` `sed` option. Supported by BSD and GNU implementations. Follow-up to 37523c9 #16480 Follow-up to 60c3d04 #14815 Follow-up to 7860f57 #12408 Closes #16712
Use a more descriptive global variable name in server code, also to avoid colliding with this name used elsewhere in libcurl. This isn't causing an issue at this time, but makes the code prone to `-Wshadow` warnings in unity mode, if the global variable is compiled first. This specific variable could collide with the `path` argument of the `curlx_win32_stat()` function. Closes #16719
It seems unnecessary and possibly unexpected to build test servers with debug-enabled features and memory tracking whenever the tested curl is built like that (which is a requirement for some tests, so curl is mostly built like that when running tests.) It also makes building servers a little bit faster with cmake for the most common cases. You can apply debug options to `tests/server` with these new options: - `./configure`: `--enable-server-debug`. - cmake: `-DENABLE_SERVER_DEBUG`. Also sync the way we pass these macros in autotools, with CMake builds. Before this patch, autotools passed them via `curl_config.h`. After this patch it passes them on the command-line, like cmake builds do. This patch also make these option no longer passed to examples and `http/client` in cmake builds, where they were no-ops anyway. Ref: #15000 Closes #16705
Before this patch, autotools builds excluded TrackMemory sources (`lib/memdebug.c` and `lib/curl_multibyte.c`) based on the `DEBUGBUILD` setting. This works in most cases because its value is the same as `CURLDEBUG` by default, but the correct condition is `CURLDEBUG`. It should fix `--disable-debug --enable-curldebug --enable-unity` builds. (not tested in CI) It also syncs behavior with cmake builds. Ref: #16705 Closes #16723
Follow-up to 00fc556 Ref: #16716 (comment) > I have some tooling leftover from when I was implementing CRL support > in the webpki crate that downloaded every CRL I could find referenced > in ccadb (without doing any special filtering for defunct CAs/CRLs > mind you) and found CRLs that spanned the range from very small > (<1mb), to medium sized (11 .. 22mb) to very large (100mb). Reported-by: Daniel McCarney Closes #16724
By allocating the method string as part of the struct, the previous fixed size limit (23 bytes) can be avoided. It would previously make "curl -X [long string]" work against http://localhost but fail against https://curl.se with no clear error message. Closes #16729
It saves about 1 minute (10%) per run. Also reduces log length from 3800 to 2800 lines. Keep building docs on native CPU. Closes #16731
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot]
Can you help keep this open source service alive? 💖 Please sponsor : )