Problems reported when using NGINX by Ondrej PPA #113
Comments
|
We have a history of poor cooperation with the Ondrej PPA lead developer. I think the best thing we can do is a blog post where we explain the situation and that it's hard for us to support this. |
|
Do we want to transfer this one to the website repo? |
|
Sounds like a plan |
I’m all ears… |
|
Now that we have @oerdnj attention 😄 (I'm surprised and glad you are here, 👋 btw), we probably should add what we think the problems (FIXME) were in the original description to see how/if things can be fixed, or just need better documentation on our side (or just the next step). |
You know you could have tagged me and asked me before you start saying things about bad cooperation…. But let’s start afresh… The PCRE2 library isn’t actually part of the nginx repository. I know that the nginx packages were (maybe still are) a mess in the past few months - it comes from the fact that Debian upstream maintainer unbundled the plugins and made some decisions that were not compatible with the PPAs. Unfortunately, this was inevitable as Ubuntu 24.04 picked on that, so I had to follow the suit, and it took me a while to figure out all the nifty details. And there’s apparently still something elusive that’s causing people problems on upgrades. I’m not omnipotent and I make mistakes. And I am usually uncooperative only in case when people come and demand I do things for them for free. But I do try to listen to a solid argument. |
|
Of course, I assumed exactly that. This is, IMHO, the opposite of being "unresponsive" 😄 😄 . Let's figure it out what happens... but as you said clearly here, and I totally agree: let's get a solid argument. I do not like not having a way to reproduce the problem. 💪 |
fzipi
changed the title
It isn't. But it's part of your PHP repository. In most cases (perhaps in all cases) the users who reported issues used some PHP application, and we assume all the used components installed from your repository. But it is also possible that the two have nothing to do with each other. But the fact that there is a situation: if users install Nginx from your repository, then libmodsecurity3 produces the same weird behavior: and some other similar reports on mailing list/StackOverflow forums. If users switch to Debian's version everything is solved. The weird behavior is that the I risk in all cases the reporter built the library (libmodsecurity3) from source, therefore the build flow used the system's PCRE library - so this is why we think that the user installed a non-default PCRE library. |
So, what I am actually hearing is that it has nothing to do with NGINX repository, and it affects mod_security only when PCRE2 (I assume this is PCRE2, right?) is updated to the latest version? The PCRE2 maintainer is usually very responsive if there’s a test case. Anyway, this should be fairly easy to test - Debian stable has PCRE2 10.42-1, Debian testing and Ubuntu 24.04 has PCRE2 10.42-4. My repositories has 10.42-3 (and the only difference between -3 and -4 is riscv64 support) Can you observe the problematic behavior on those systems? Perhaps this boils down to - do you have means to reproduce the weird behavior? |
Unfortunately we don't know any details about that. Here is a link that points the reporter's howto, but that's unavailable now. In the next comment I tried the request with same config, but it worked for me - but I was using Ubuntu's Nginx. Btw I'm not sure it was built with PCRE2, because in Ubuntu 22.04 - as I remember - the default PCRE engine version was the 3, not the 2.
Based on the most valuable issue and its comment, I would try to follow this one. Install Nginx from your repository (onto an Ubuntu 22.04 - just for sure), install libmodsecurity3 from source - perhaps you should try with PCRE3 first - then install the connector. After that install CRS, and try to send the mentioned request. If you need any help to configure/build the WAF just let us know. |
That might be the problem - PCRE (libpcre3) is obsolete for couple of years now and it doesn't receive any updates and fixes anymore. I also see this:
And I see issues like this in modsecurity:
So, instead of saying the repository is at fault, I would suggest that ModSecurity uses PCRE2 by default, and for all bugreports where people compile libmodsecurity3 by hand, first suggest to use PCRE2 (libpcre2) instead of PCRE1 (libpcre3). I understand it's very confusing that libpcre3 is PCRE1 (unsupported) and libpcre2 is PCRE2, but that's related to the SONAMEs of the libraries and is a different number than PCRE vs PCRE2. |
|
Hmm, I am looking at coreruleset/coreruleset#3181 (comment) and this says the user compiled nginx from the source. |
I know, but if I'm right the most packages upgraded in Debian 12 - before that every application used PCRE (libpcre3), eg. Nginx. This is why was (and it's still) the default in libmodsecurity3.
yes, there was an issue, but I'm not sure that had any effect.
There is a intention that we will upgrade the source in case of both engines and the default regex engine will be the PCRE2.
I know this, but here I think it's unrelated too. But may be I'm wrong. Just a question: with which regex engine you used to built your Nginx packages? |
Yes, in this case he did. But in most cases users used your repository. |
Good question - I actually don't do anything specific and |
On each systems? I mean Debian 11 and "older" (but still supported) Ubuntus? Just because on Debian 11 the default pcre engine was the old one (PCRE3), and Nginx package depends on libpcre3: $ ldd /usr/sbin/nginx
...
libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007f6080635000)
...This changed in Debian 12: $ ldd /usr/sbin/nginx
...
libpcre2-8.so.0 => /lib/x86_64-linux-gnu/libpcre2-8.so.0 (0x00007fe10637f000)
...(note: replacing the old PCRE had started somewhere here: https://lists.debian.org/debian-devel/2021/11/msg00176.html) So if you always built your packages with pcre2, and users built the library with pcre3, then - I don't know why/how - it could cause the behavior.... |
I actually can't even reproduce the linking problem, but that could be effect of default Debian linker (Debian bookworm) that removes unused libraries as I see linkage to |
Yes,
Yes, that's pretty much confusing. Perhaps the discrepancy in the regex processing causes an error on a different layer? I don't know, I know literally nothing about ModSecurity. |
|
Do you have any recent case? Or perhaps if there's a new one, can you point me to it, and I'll poke the user around? I don't think this is directly related to the PPAs (and Debian repositories), but is rather some side-effect of changes to nginx. And if that's the case, I am guessing it is not going to be isolated problem. BTW if you install |
|
I can also take a look and perhaps I can start providing I do have a feature request for this already: oerdnj/deb.sury.org#1192 |
Unfortunately (or fortunately? 😄) not. There are too much impulses around here, I just looked for GH issue search mechanism and could find the mentioned three. But we were constantly following the SO (StackOveflow) forums, we have a mailing list and a Slack channel - as far as I remember, we got very similar problems on almost every channel.
Sounds not so good.
Nginx-dev exists only since Bookworm: https://packages.debian.org/search?keywords=nginx-dev As I see the previous versions of Nginx (eg. the last non-modular version) depends on libpcre3: I assume the mentioned Ubuntu systems (like 22.04) also followed this build config. But for sure: we should try to reproduce this behavior. Well, perhaps we need to try it on an older system (Debian 11 or Ubuntu 22.04). |
The CRS team believes that strange CRS problems are reported when users use NGINX Ondrej PPA.
Reports:
What we think the problem is: FIXME
We would like to solve this problem (if any) or have it documented so users understand how to follow.
Statements from chat so that we don't lose them:
We've had, I think, 4 separate reports.
The text was updated successfully, but these errors were encountered: