The Wazuh indexer removal does not remove the installation directory after upgrade

[rauldpm]

Describe the bug
The Wazuh indexer installation directory is not being removed from the system after upgrading from 4.8.2 to 4.9.1

Related wazuh/wazuh-qa#5753

To Reproduce
Steps to reproduce the behavior:

1. Deploy a 4.8.2 All-In-One deployment using the Wazuh installation assistant
2. Upgrade the Wazuh indexer to 4.9.0 or 4.9.1
3. Remove the Wazuh indexer

Expected behavior
The /usr/share/wazuh-indexer directory should not exist after being removed

Plugins
Default ones

Host/Environment (please complete the following information):

• OS: CentOS
• Version: 7

Debian systems should be checked

Additional context

• Removal of a Wazuh indexer 4.8.2 package after being upgraded from 4.7.5

...
  Verifying  : wazuh-indexer-4.8.2-1.x86_64                                                                                      1/2 
  Verifying  : wazuh-indexer-4.7.5-1.x86_64                                                                                      2/2 

Updated:
  wazuh-indexer.x86_64 0:4.8.2-1                                                                                                     

Complete!


....

Removed:
  wazuh-indexer.x86_64 0:4.8.2-1                                                                                                     

Complete!
[root@centos7 vagrant]# ls -l /usr/share/wazuh-indexer
ls: cannot access /usr/share/wazuh-indexer: No such file or directory

• Removal of a Wazuh indexer 4.9.1 package after being upgraded from 4.8.2

....

  Verifying  : wazuh-indexer-4.9.1-0.x86_64                                                         1/2 
  Verifying  : wazuh-indexer-4.8.2-1.x86_64                                                         2/2 

Updated:
  wazuh-indexer.x86_64 0:4.9.1-0                                                                        

Complete!


....

Removed:
  wazuh-indexer.x86_64 0:4.9.1-0                                                                                                     

Complete!
[root@centos7 vagrant]# ls -l /usr/share/wazuh-indexer/
total 0
[root@centos7 vagrant]# ls -ld /usr/share/wazuh-indexer
drwxr-x---. 3 wazuh-indexer wazuh-indexer 20 Sep 19 18:55 /usr/share/wazuh-indexer

[AlexRuiz7]

Removal of the 4.8.2 package

root@ubuntu2204:/home/vagrant# ls -la /usr/share/wazuh-indexer/
total 304
drwxr-x---   8 wazuh-indexer wazuh-indexer   4096 Sep  6 13:14 .
drwxr-xr-x 116 root          root            4096 Sep  6 13:22 ..
drwxr-x---   3 wazuh-indexer wazuh-indexer   4096 Sep  6 13:14 bin
drwxr-x---   9 wazuh-indexer wazuh-indexer   4096 Sep  6 13:14 jdk
drwxr-x---   3 wazuh-indexer wazuh-indexer   4096 Sep  6 13:14 lib
-rwxr-x---   1 wazuh-indexer wazuh-indexer  11358 Sep 19  2023 LICENSE.txt
drwxr-x---  21 wazuh-indexer wazuh-indexer   4096 Sep  6 13:13 modules
-rwxr-x---   1 wazuh-indexer wazuh-indexer 261032 Sep 19  2023 NOTICE.txt
drwxr-x---   5 wazuh-indexer wazuh-indexer   4096 Sep  6 13:14 performance-analyzer-rca
drwxr-x---  21 wazuh-indexer wazuh-indexer   4096 Sep  6 13:14 plugins
-r--r-----   1 wazuh-indexer wazuh-indexer      6 Aug 19 17:14 VERSION

root@ubuntu2204:/home/vagrant# apt-get remove --purge wazuh-indexer -y
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following packages will be REMOVED:
  wazuh-indexer*
0 upgraded, 0 newly installed, 1 to remove and 157 not upgraded.
After this operation, 1050 MB disk space will be freed.
(Reading database ... 190752 files and directories currently installed.)
Removing wazuh-indexer (4.8.2-1) ...
Stopping wazuh-indexer service... OK
(Reading database ... 189618 files and directories currently installed.)
Purging configuration files for wazuh-indexer (4.8.2-1) ...
Deleting configuration directory... OK
dpkg: warning: while removing wazuh-indexer, directory '/var/lib/wazuh-indexer' not empty so not removed
dpkg: warning: while removing wazuh-indexer, directory '/var/log/wazuh-indexer' not empty so not removed
root@ubuntu2204:/home/vagrant# ls -la /usr/share/wazuh-indexer/
ls: cannot access '/usr/share/wazuh-indexer/': No such file or directory

Removal of the 4.9.1 package

root@ubuntu2204:/home/vagrant# apt-get remove --purge wazuh-indexer -y
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following packages will be REMOVED:
  wazuh-indexer*
0 upgraded, 0 newly installed, 1 to remove and 157 not upgraded.
After this operation, 1077 MB disk space will be freed.
(Reading database ... 190765 files and directories currently installed.)
Removing wazuh-indexer (4.9.1-0) ...
Running Wazuh Indexer Pre-Removal Script
(Reading database ... 189618 files and directories currently installed.)
Purging configuration files for wazuh-indexer (4.9.1-0) ...
dpkg: warning: while removing wazuh-indexer, directory '/var/log/wazuh-indexer' not empty so not removed
dpkg: warning: while removing wazuh-indexer, directory '/var/lib/wazuh-indexer' not empty so not removed
dpkg: warning: while removing wazuh-indexer, directory '/etc/wazuh-indexer' not empty so not removed
root@ubuntu2204:/home/vagrant# ls -la /usr/share/wazuh-indexer
ls: cannot access '/usr/share/wazuh-indexer': No such file or directory
root@ubuntu2204:/home/vagrant# ls -lR /etc/wazuh-indexer
/etc/wazuh-indexer:
total 12
dr-x------ 2 wazuh-indexer wazuh-indexer 4096 Sep  6 13:14 certs
drwxr-x--- 2 wazuh-indexer wazuh-indexer 4096 Sep  6 13:23 internalusers-backup
-rw-rw---- 1 wazuh-indexer wazuh-indexer  196 Sep  6 13:14 opensearch.keystore

/etc/wazuh-indexer/certs:
total 20
-r-------- 1 wazuh-indexer wazuh-indexer 1704 Sep  6 13:06 admin-key.pem
-r-------- 1 wazuh-indexer wazuh-indexer 1119 Sep  6 13:06 admin.pem
-r-------- 1 wazuh-indexer wazuh-indexer 1204 Sep  6 13:06 root-ca.pem
-r-------- 1 wazuh-indexer wazuh-indexer 1704 Sep  6 13:06 wazuh-indexer-key.pem
-r-------- 1 wazuh-indexer wazuh-indexer 1285 Sep  6 13:06 wazuh-indexer.pem

/etc/wazuh-indexer/internalusers-backup:
total 4
-rw-r----- 1 wazuh-indexer wazuh-indexer 1145 Sep  6 13:23 internal_users_20240906_132300.yml.bkp

During the uninstallation of the wazuh-indexer package, the /usr/share/wazuh-indexer folder has been removed, like in previous versions.

However, I identified a new warning about the /etc/wazuh-indexer folder not being removed. As we can see, the certificates are kept intact. I compared these files with their 4.8.2 version and they look the same.

root@ubuntu2204:/home/vagrant# ls -lR /etc/wazuh-indexer
/etc/wazuh-indexer:
total 64
dr-x------ 2 wazuh-indexer wazuh-indexer  4096 Sep  6 13:14 certs
drwxr-x--- 2 wazuh-indexer wazuh-indexer  4096 Sep  6 13:23 internalusers-backup
-rw-rw---- 1 wazuh-indexer wazuh-indexer  2943 Sep  6 13:14 jvm.options
drwxr-x--- 2 wazuh-indexer wazuh-indexer  4096 Aug 19 17:14 jvm.options.d
-rw-r----- 1 wazuh-indexer wazuh-indexer 14808 Aug 19 17:14 log4j2.properties
-rw-rw---- 1 wazuh-indexer wazuh-indexer   196 Sep  6 13:14 opensearch.keystore
drwxr-x--- 2 wazuh-indexer wazuh-indexer  4096 Sep  6 13:14 opensearch-notifications
drwxr-x--- 2 wazuh-indexer wazuh-indexer  4096 Sep  6 13:14 opensearch-notifications-core
drwxr-x--- 2 wazuh-indexer wazuh-indexer  4096 Sep  6 13:14 opensearch-observability
drwxr-x--- 2 wazuh-indexer wazuh-indexer  4096 Sep  6 13:14 opensearch-performance-analyzer
drwxr-x--- 2 wazuh-indexer wazuh-indexer  4096 Sep  6 13:14 opensearch-reports-scheduler
drwxr-x--- 2 wazuh-indexer wazuh-indexer  4096 Sep  6 13:14 opensearch-security
-rw-rw---- 1 wazuh-indexer wazuh-indexer  2152 Sep  6 13:14 opensearch.yml

/etc/wazuh-indexer/certs:
total 20
-r-------- 1 wazuh-indexer wazuh-indexer 1704 Sep  6 13:06 admin-key.pem
-r-------- 1 wazuh-indexer wazuh-indexer 1119 Sep  6 13:06 admin.pem
-r-------- 1 wazuh-indexer wazuh-indexer 1204 Sep  6 13:06 root-ca.pem
-r-------- 1 wazuh-indexer wazuh-indexer 1704 Sep  6 13:06 wazuh-indexer-key.pem
-r-------- 1 wazuh-indexer wazuh-indexer 1285 Sep  6 13:06 wazuh-indexer.pem

/etc/wazuh-indexer/internalusers-backup:
total 4
-rw-r----- 1 wazuh-indexer wazuh-indexer 1145 Sep  6 13:23 internal_users_20240906_132300.yml.bkp

[AlexRuiz7]

We have performed several tests and we have been able to reproduce the problem only in RPM based systems. The /usr/share/wazuh-indexer is kept due to a hidden cache folder created by the JNA. The removal of this folder is harmless. As the problem is only reproducible on Red Hat based systems and as we state in our uninstallation guide to remove every wazuh-indexer folder recursively as a manual post-removal step, we have concluded this won't be fixed.

Our tests have been performed under the following operating systems, using Vagrant boxes:

Vagrant box	Reproducible
generic/ubuntu2204	No
generic/rhel9	Yes
generic/rhel7	Yes
bento/centos-8	No