Automate Debian Package generation for Ubuntu apt repos

[AndrewQuijano]

Feature

• New architecture module
• Support for processor extension
• Add more instruction details (elaborated below)
• Binding support for: language
• Other (elaborated below)

Describe the feature you'd like
I will work on this on another PR, but this is based on #2398. But to better leverage CI to help automate your team to provide Ubuntu the latest packages, I would like to automate the process of creating newer 'libcapstone-dev' and 'libcapstone5/6' Debian packages.

To help you with the PR to get this work, I suspect the main solution is to get the following:

1- When I used apt to check information, where is that data coming from? Is that from capstone.pc? How did you create the Debian packages before used by the Ubuntu repository for apt?

2- To create two separate Debian packages, how should I do this? From my understanding, this means that libcapstone-dev will have just the header files, and another will include the library files?

Either way, if I can get guidance on this, I can do the work on getting the PR and tests for both Debian packages. Then on the releases, we can just get two Debian packages, which can easily just be handed over to Ubuntu apt managers.

P. S.
@Rot127 Could you please create Debian 5.0.4 release with the Debian package? I'd like to start updating PANDA to use this.

Also, whenever we got the PR merged, you want to update documentation to either check Releases and point to a libcapstone5, etc.
http://www.capstone-engine.org/documentation.html

Additional context
N/A.

[AndrewQuijano]

Keeping track of this to better plan the PR and help with answering my questions

[Rot127]

1- When I used apt to check information, where is that data coming from? Is that from capstone.pc?

No idea. But either from there or from debian/control I guess.

How did you create the Debian packages before used by the Ubuntu repository for apt?

I don't know actually. The package was added to the Ubuntu repositories by someone else years ago. I am not familiar with the process tbh.

2- To create two separate Debian packages, how should I do this? From my understanding, this means that libcapstone-dev will have just the header files, and another will include the library files?

Yeah, this is the common pattern for -dev packages in my experience.

Either way, if I can get guidance on this, I can do the work on getting the PR and tests for both Debian packages.

As said above. I would also need to read myself into it. And I currently don't have the time, sadly.
This looks like a good description of the process.

@Rot127 Could you please create Debian 5.0.4 release with the Debian package? I'd like to start updating PANDA to use this.

You mean a release in the repo here, right?
We do a patch release of v6-Alpha-1 soon (weekend hopefully).

[AndrewQuijano]

Oh, I meant that if I can get a 5.0.4, currently PANDA uses libcapstone5, and we can just get the Debian package from the release. I figure this would be helpful for everyone else using libcapstone5 too.

https://github.com/capstone-engine/capstone/releases/tag/5.0.3

I can work on the packages, I will just create two new folders to have very similar behavior to make the new Debian packages. I figure we can leave it as future work to see how to get the Debian packages into apt repositories, but please keep me posted on that.

[AndrewQuijano]

@Rot127, for version 6 Debian package creation, I haven't tested passing in an argument like 6.0.0-Alpha1, it works with a v6.0.1 or 6.0.1

[Rot127]

I haven't tested passing in an argument like 6.0.0-Alpha1, it works with a v6.0.1 or 6.0.1

I think Debian packages are not allowed to be an alpha. But you can check the name with dpkg-name apparently.

[AndrewQuijano]

Yeah, the script needs a bit of updating on version checking, but it seems you can put 6.0.0-alpha as a Version for Debian Packages.

Please keep me posted on action items needed to go to full Debian repository, but the more I think about it, it might be easier just to scrap having a 'libcapstone' and 'libcapstone-dev', and just use a 'libcapstone-dev'. I can work on it, and I already have the control file, not sure what else is needed, but keep me posted.

[AndrewQuijano]

Copying this to note for myself here

I looked into your questions in detail and found actually more than one problem. I am sorry that I did not review more thoroughly before. I also never worked with deb packages until now, was in a rush due to other things and oversaw quite some problems.
Thank you for asking about these problems, otherwise I would have missed them.

The issues I found are the following:

We didn't added all mandatory fields to the control file.

• Architecture in the control file was incorrectly set to all.
• As you already said, the package should be installed into /usr/.
• The generated package misses the dependency information.
• The package name was not according to the Debian standard.
• Hash sums missing

I tried fixing some of them, but quickly recognized that it gets out of hand quickly. And for Capstone we need a more complete approach.

As it looks like, the original script, you used as a basis, was never meant to generate a deb package for the official Debian/Ubuntu repositories. This would also explain why it was lacking so many things.

Unfortunately, we cannot provide a package which is this much incomplete for the release. In #2543 I removed it from the release CI, but kept your work in packages/deb/. Also you can trigger the package build by dispatching the new Build Debian Package workflow.

If you want to have a deb package in the official Ubuntu/Debian repositories, I would ask you to do the implementation from scratch. This time though by using CPack. And with CPack we can also generate packages for other distros. Also we have a certain guarantee that the built packages are correct.
If you do so, please first submit the package to the Debian packages and wait for approval. Then we can merge it here.

https://www.debian.org/doc/manuals/maint-guide/

https://wiki.debian.org/DebianMentorsFaq

[AndrewQuijano]

@Rot127 My plan to update the Debian Package is:

1- Create the capstone Debian package, install it within PANDA, and check integration tests (which I suspect the integration tests use capstone). If this works, at least we have a package that should work at the bare minimum for real usage.

2- Could you please provide assistance on all the packages needed to install capstone? I'm surprised I don't see anything on the COMPILE.txt file, etc. This information should be documented somewhere in the README or something as well too

[Rot127]

If this works, at least we have a package that should work at the bare minimum for real usage.

If it fulfills the minimum requirements of a Debian package (as they document it), this is fine. But it should be at least as complete as the ones currently in the Debian repos.

Could you please provide assistance on all the packages needed to install capstone?

For main Capstone? Just libc. For testing one requires libyaml (see: #2560 (comment)). Or the equivalent Python package.

[AndrewQuijano]

Is there a minimum libc version? Based on current libcapstone-dev v4.0.1 it says the only dependancy is libc6 (>= 2.14), not sure if the version should be bumped up.

Also, for now, I think it is easier to just have one Debian package instead of two, especially since both of these aren't too big anyways.

[Rot127]

Also, for now, I think it is easier to just have one Debian package instead of two, especially since both of these aren't too big anyways.

You mean a package with cstest? This is definitely not needed.
A package with cstool and the headers would be nice. So a libcapstone-dev.

Is there a minimum libc version?

There was never one defined. I get 2.2.5 as the oldest when doing:

$ objdump -T build/cstool 

build/cstool:     file format elf64-x86-64

DYNAMIC SYMBOL TABLE:
0000000000000000      DF *UND*	0000000000000000 (GLIBC_2.2.5) free
0000000000000000      DF *UND*	0000000000000000 (GLIBC_2.2.5) putchar
...

I think in general it should be the one provided by the oldest not EOL Debian LTS.

[AndrewQuijano]

See #2569, I think I am getting very close to getting a functional Debian package

[AndrewQuijano]

@kabeor @Rot127 please create a capstone 5.0.4 release so PANDA can use the Debian Package