Implement CPack for Debian/RPM #2590

AndrewQuijano · 2024-12-22T19:21:59Z

Your checklist for this pull request

I've documented or updated the documentation of every API function and struct this PR changes.
I've added tests that prove my fix is effective or that my feature works (if possible)

Detailed description
I created a CPackConfig.txt that can create both Debian and RPM packages. The Debian package matches what we have now. I can look further about making a proper RPM package.

Debian Screenshots

This is identical to #2579

RPM

Here is the generated spec file
libcapstone-dev.spec.txt

I do want to get help with macOS package creation and testing using CPack

I updated how to do this in the Building.md file

...

Test plan
We can probably use something like check_capstone.sh? Although for now, we can merge it as another way to create the Debian Package? My hope is, from here, work can be done to get a Mac OSX package built. I can help with RPM, the work is done, but any last necessary touches I can take care of it.

...

Closing issues

...

AndrewQuijano · 2024-12-27T02:37:12Z

@Rot127 While I will need help with the Mac stuff, I can try to poke around how capstone made RPM packages before, but your feedback would be helpful to have! I did poke around and find this, it seems the RPM packaging standard is a lot more of a mess than I thought as it seems anyone can package as they see fit.

https://rpmfind.net/linux/rpm2html/search.php?query=libcapstone-devel
https://rpmfind.net/linux/rpm2html/search.php?query=capstone
https://rpmfind.net/linux/rpm2html/search.php?query=capstone-devel

I do want to ask, if the goal is to automate placing Debian packages in APT, this would require creating both a libcapstone package with the shared objects only and another debian with the static library/headers. I could use cmake twice and leverage CPack to make two packages. To avoid the headers being made twice, would I need to delete these lines of code?

https://github.com/capstone-engine/capstone/blob/next/CMakeLists.txt#L799-L801

Rot127 · 2024-12-29T15:40:40Z

Sorry for the late answer. I will review this one after new year. But to answer your questions meanwhile:

I do want to ask, if the goal is to #2537, this would require creating both a libcapstone package with the shared objects only and another debian with the static library/headers.

Yes, this would be nice to have. Though, I can't put any priority on it. And before v6 Beta, it is not necessary. I think for v6 Gold it is though.

try to poke around how capstone made RPM packages before, but your feedback would be helpful to have!

I have no idea. They were made before my time here. You have to contact the package maintainers and/or follow the docs.

AndrewQuijano · 2024-12-31T00:13:57Z

I poked around a bit more about making the RPM the best I can, so a few things I'll likely need help on,
I will ask based on this
https://github.com/capstone-engine/capstone/blob/next/packages/rpm/capstone.spec

It would be nice to have something to check if the package works, is this legit?

ln -s libcapstone.so libcapstone.so.5
make check LD_LIBRARY_PATH="`pwd`"

It would be nice to have an install too, would this look accurate to add?

DESTDIR=%{buildroot} LIBDIRARCH=%{_lib} \
INCDIR="%{_includedir}" make install
find %{buildroot} -name '*.la' -exec rm -f {} ';'
find %{buildroot} -name '*.a' -exec rm -f {} ';'

# install python bindings
pushd bindings/python
%{__python2} setup.py install --skip-build --root %{buildroot}
%if 0%{?with_python3}
%{__python3} setup.py install --skip-build --root %{buildroot}
%endif # with_python3
popd

# install java bindings
install -D -p -m 0644 bindings/java/%{name}.jar  %{buildroot}/%{_javadir}/%{name}.jar

I'm not sure if a build would be need for the package?

DESTDIR="%{buildroot}" 	V=1 CFLAGS="%{optflags}" \
LIBDIRARCH="%{_lib}" INCDIR="%{_includedir}" make %{?_smp_mflags}

# Fix pkgconfig file
sed -i 's;%{buildroot};;' capstone.pc
grep -v archive capstone.pc > capstone.pc.tmp
mv capstone.pc.tmp capstone.pc

# build python bindings
pushd bindings/python
CFLAGS="%{optflags}" %{__python2} setup.py build
%if 0%{?with_python3}
CFLAGS="%{optflags}" %{__python3} setup.py build
%endif # with_python3
popd

# build java bindings
pushd bindings/java
make CFLAGS="%{optflags}" # %{?_smp_mflags} parallel seems broken
popd

Please feel free to check my current generated spec file and poke around what else should be needed to make this as high quality as possible. I'm also realizing there is no contacts to who packages for Fedora and CentOS, etc.
No worries, we can wait until next year, but will also need your help on updating CPack to also generate the Mac packages too :)

Rot127

Nice! Thanks a lot for getting started with it.

Some general notes:

Let's exclude OSX for now, please. It gets too much otherwise and they can be part of another PR.
The packages/rpm/capstone.spec file shouldn't be necessary. I think it can be removed. CPack should generate it.
Once we have tested the package build locally, we can add the process to the release workflow.

Regarding #2590 (comment)

This looks a little hacky. Let's not worry about the bindings for now. Because they need some refactoring before (see: #2291).
And maybe there are way better ways to create such packages. In the case of the Python bindings, they are already uploaded to pypi.

CPackConfig.txt

packages/rpm/uninstall.sh

packages/rpm/remove_blank_lines.sh

CPackConfig.txt

AndrewQuijano · 2025-01-05T20:08:47Z

@Rot127 Just updated the CPack file per your suggestions (Note both Debian and RPM package content looks identical now), three questions remain:

Is there a way to test the package after the installation to confirm it works?
Have you had a minute to check if the RPM package works as intended?
Once this is tested, should I delete all the docker stuff from the deb folder too? Should I also replicate this on v5 branch as I do still really hope to get a v5.0.4 package ready for faster PANDA installation relatively soon

https://github.com/AndrewQuijano/capstone/releases/tag/6.0.2-Alpha3

Rot127 · 2025-01-06T19:25:13Z

Is there a way to test the package after the installation to confirm it works?

I just ran a cstool command the last time to check. Because cstool uses the dynamic library.
For headers we can just check, if we can include them as system header in a C source file.

Have you had a minute to check if the RPM package works as intended?

Not yet. I waited for you to add the workflow job, so I don't have to set up to much. Will do review this one again tomorrow.

Once this is tested, should I delete all the docker stuff from the deb folder too?

Yeah, sure. Less duplicates are always better.

Should I also replicate this on v5 branch as I do still really hope to get a v5.0.4 package ready for faster PANDA installation relatively soon

This would be awesome!

@kabeor Do you know if we could get the release before lunar new year?
If you have enough time, @AndrewQuijano can open the PR now and we can prioritize it. So it is merged quickly.

AndrewQuijano · 2025-01-07T05:16:02Z

@Rot127 What are the cstool commands you used? I think I can add this to RPM spec file? I just removed the duplicate with Docker and created the PR for v5.

I should note, it seems I broke labeler when switching to v5, I'm trying to fix this, but it is turning out to be a bit more complicated than I thought

https://github.com/actions/labeler

https://github.com/capstone-engine/capstone/actions/runs/12645601156/job/35235004550?pr=2590

Rot127 · 2025-01-08T13:28:37Z

I should note, it seems I broke labeler when switching to v5, I'm trying to fix this, but it is turning out to be a bit more complicated than I thought

No worries. Going to fix it.

What are the cstool commands you used?

Just some basic disassembly. It just needs to prove it loads the share library properly and correctly disassembles an instruction.

Rot127

I made the review on this one. Since they the comments are basically the same for the v5 version.

Btw, I am sorry if I sometimes make multiple suggestions on the same line of code. I learn how the CPack packaging works while reviewing your code. So sometimes I see things only on a second look.

Hope this is not discouraging. I very much appreciate your effort!

.github/labeler.yml

.github/workflows/labeler.yml

packages/rpm/postinstall.sh

CPackConfig.txt

LICENSES/LICENSE_PACKAGE.txt

AndrewQuijano · 2025-01-11T20:32:52Z

@Rot127 Just updated the PR, and just curious would you look into how to update labeler to v5? Also, figure I should ask, you don't mind I use your ISSUE_TEMPLATE folder and labeler stuff for any other open-source work right?

Rot127

Looks good!

But the build fails for me: https://github.com/Rot127/capstone/actions/runs/12734529842/job/35492198289

...
 Enabling CAPSTONE_WASM_SUPPORT
Enabling CAPSTONE_BPF_SUPPORT
Enabling CAPSTONE_RISCV_SUPPORT
Enabling CAPSTONE_SH_SUPPORT
Enabling CAPSTONE_TRICORE_SUPPORT
Enabling CAPSTONE_ALPHA_SUPPORT
Enabling CAPSTONE_HPPA_SUPPORT
Enabling CAPSTONE_LOONGARCH_SUPPORT
Enabling CAPSTONE_XTENSA_SUPPORT
CMake Error at /usr/local/share/cmake-3.31/Modules/WriteBasicConfigVersionFile.cmake:43 (message):
  No VERSION specified for WRITE_BASIC_CONFIG_VERSION_FILE()
Call Stack (most recent call first):
  /usr/local/share/cmake-3.31/Modules/CMakePackageConfigHelpers.cmake:402 (write_basic_config_version_file)
  CMakeLists.txt:897 (write_basic_package_version_file)


-- Configuring incomplete, errors occurred!

Rot127 · 2025-01-12T15:47:07Z

.github/workflows/build_release.yml

+        cd build
+        cpack -G DEB
+        cpack -G RPM
+


While testing I thought it would be nice to upload the produced packages also as artifacts.

So we don't need a release each time we want to download them.

- uses: actions/upload-artifact@v4 with: name: ${{ env.ARTIFACT_NAME }} path: ./build/*

@Rot127 Would I add this to the release workflow? It just seems like this would mean the artifact would only be made on release unless I put it on another workflow?

Please add the commit so I can add it to this PR

Rot127 · 2025-01-12T15:50:56Z

CPackConfig.cmake

The build possibly fails due to this name.
CPackConfig.cmake is a reserved name IIRC. Try to rename it to CPackGeneratorSetup.txt or something like this.

The issue is the version v110.0.0, so I will update the regex to remove the 'v' or 'V' in front of the version argument. Note this also works with current debian packaging standard e. g. 6.0.0-Alpha2

let me know if the regex should be stricter to remove/ignore all numbers before the X.Y.Z?

string(REGEX MATCH "^[vV]?([0-9]+\\.[0-9]+\\.[0-9]+)" _ ${PROJECT_VERSION}) set(PROJECT_VERSION_BASE ${CMAKE_MATCH_1})

Rot127 · 2025-01-12T15:53:43Z

Just updated the PR, and just curious would you look into how to update labeler to v5?

It isn't a priority currently for me. Too much other stuff to do unfortunately. If you want to work on it feel free to do so! If you are faster then @kabeor reviewing, we can replace my downgrading PR with yours.

Also, figure I should ask, you don't mind I use your ISSUE_TEMPLATE folder and labeler stuff for any other open-source work right?

Not at all. It is open-source in the end :D

github-actions bot added the Documentation label Dec 22, 2024

AndrewQuijano force-pushed the cpack branch from 62e02ea to 5931893 Compare December 22, 2024 19:34

AndrewQuijano changed the title ~~Implement CPack for Debian/RPM~~ Implement CPack for Debian/RPM and delete Travis Dec 22, 2024

AndrewQuijano mentioned this pull request Dec 22, 2024

[FR]: Create Packages on every release (Debian, RPM, etc.) google/googletest#4686

Open

AndrewQuijano force-pushed the cpack branch 4 times, most recently from 419750b to 12879d4 Compare December 31, 2024 00:08

Rot127 requested changes Jan 3, 2025

View reviewed changes

AndrewQuijano force-pushed the cpack branch from 12879d4 to 0150dfe Compare January 5, 2025 19:58

github-actions bot added the Github-files Github related files label Jan 5, 2025

AndrewQuijano force-pushed the cpack branch from 0150dfe to 1221c3c Compare January 5, 2025 20:04

AndrewQuijano force-pushed the cpack branch 6 times, most recently from d2bb8b5 to c77d6ff Compare January 6, 2025 00:27

AndrewQuijano force-pushed the cpack branch from c77d6ff to 3ab5f1f Compare January 7, 2025 03:49

This was referenced Jan 7, 2025

Duplicate of #2590 for v5 Branch #2596

Open

Update CPack to also create DMG packages #2597

Open

AndrewQuijano force-pushed the cpack branch 4 times, most recently from 0e85387 to e31a1e9 Compare January 7, 2025 05:12

Rot127 requested changes Jan 8, 2025

View reviewed changes

AndrewQuijano changed the title ~~Implement CPack for Debian/RPM and delete Travis~~ Implement CPack for Debian/RPM Jan 11, 2025

AndrewQuijano force-pushed the cpack branch 2 times, most recently from 023a04f to f82b52a Compare January 11, 2025 20:06

AndrewQuijano mentioned this pull request Jan 11, 2025

Update CPack to support FreeBSD Packages #2601

Open

5 tasks

Rot127 requested changes Jan 12, 2025

View reviewed changes

AndrewQuijano force-pushed the cpack branch from f82b52a to 39bcf36 Compare January 12, 2025 23:49

Create debian and rpm package on releases

cc1aaa1

AndrewQuijano force-pushed the cpack branch from 39bcf36 to cc1aaa1 Compare January 13, 2025 00:02

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement CPack for Debian/RPM #2590

Implement CPack for Debian/RPM #2590

AndrewQuijano commented Dec 22, 2024 •

edited

Loading

AndrewQuijano commented Dec 27, 2024 •

edited

Loading

Rot127 commented Dec 29, 2024

AndrewQuijano commented Dec 31, 2024 •

edited

Loading

Rot127 left a comment

AndrewQuijano commented Jan 5, 2025 •

edited

Loading

Rot127 commented Jan 6, 2025 •

edited

Loading

AndrewQuijano commented Jan 7, 2025 •

edited

Loading

Rot127 commented Jan 8, 2025 •

edited

Loading

Rot127 left a comment

AndrewQuijano commented Jan 11, 2025

Rot127 left a comment

Rot127 Jan 12, 2025

AndrewQuijano Jan 12, 2025

Rot127 Jan 12, 2025

AndrewQuijano Jan 12, 2025 •

edited

Loading

AndrewQuijano Jan 13, 2025

Rot127 commented Jan 12, 2025

Implement CPack for Debian/RPM #2590

Are you sure you want to change the base?

Implement CPack for Debian/RPM #2590

Conversation

AndrewQuijano commented Dec 22, 2024 • edited Loading

AndrewQuijano commented Dec 27, 2024 • edited Loading

Rot127 commented Dec 29, 2024

AndrewQuijano commented Dec 31, 2024 • edited Loading

Rot127 left a comment

Choose a reason for hiding this comment

AndrewQuijano commented Jan 5, 2025 • edited Loading

Rot127 commented Jan 6, 2025 • edited Loading

AndrewQuijano commented Jan 7, 2025 • edited Loading

Rot127 commented Jan 8, 2025 • edited Loading

Rot127 left a comment

Choose a reason for hiding this comment

AndrewQuijano commented Jan 11, 2025

Rot127 left a comment

Choose a reason for hiding this comment

Rot127 Jan 12, 2025

Choose a reason for hiding this comment

AndrewQuijano Jan 12, 2025

Choose a reason for hiding this comment

Rot127 Jan 12, 2025

Choose a reason for hiding this comment

AndrewQuijano Jan 12, 2025 • edited Loading

Choose a reason for hiding this comment

AndrewQuijano Jan 13, 2025

Choose a reason for hiding this comment

Rot127 commented Jan 12, 2025

AndrewQuijano commented Dec 22, 2024 •

edited

Loading

AndrewQuijano commented Dec 27, 2024 •

edited

Loading

AndrewQuijano commented Dec 31, 2024 •

edited

Loading

AndrewQuijano commented Jan 5, 2025 •

edited

Loading

Rot127 commented Jan 6, 2025 •

edited

Loading

AndrewQuijano commented Jan 7, 2025 •

edited

Loading

Rot127 commented Jan 8, 2025 •

edited

Loading

AndrewQuijano Jan 12, 2025 •

edited

Loading