Skip to content

Create python tutorial. #1648

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ members = [
"examples/sprites",
"examples/todolist",
"examples/traits",
"examples/distributing/uniffi-rust-to-python-library",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why the subfolder?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Somebody told me to create a distributing subfolder.


"fixtures/benchmarks",
"fixtures/coverall",
Expand Down
4 changes: 4 additions & 0 deletions docs/manual/src/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,10 @@
- [Building a Swift module](./swift/module.md)
- [Integrating with Xcode](./swift/xcode.md)

# Python

- [setup project](./python/setup.md)

# Internals
- [Design Principles](./internals/design_principles.md)
- [Navigating the Code](./internals/crates.md)
Expand Down
24 changes: 24 additions & 0 deletions docs/manual/src/python/setup.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# Intro

The main idea is to build the bindings with commands.

So you must create a setup.py file. This file will include the command to generate python .py bindings, link .dll and then build to pipy. Requires python version greater than 3.6.

The full example is available at this [address](https://github.com/mozilla/uniffi-rs/tree/main/examples/distributing/uniffi-rust-to-python-library/src/setup.py).

Once you reproducted the template on your project, fell free to run with:

## Windows Powershell

```powershell
python .\src\setup.py bdist_wheel ;
$wheelFile = Get-ChildItem -Path .\dist\ -Recurse -Include * ;
pip install $wheelFile --force-reinstall ;
```

## MacOs and Linux commands:

```bash
python3 ./src/setup.py bdist_wheel --verbose ;
pip3 install ./dist/* --force-reinstall ;
Comment on lines +22 to +23
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need for semicolons

Suggested change
python3 ./src/setup.py bdist_wheel --verbose ;
pip3 install ./dist/* --force-reinstall ;
python3 ./src/setup.py bdist_wheel --verbose
pip3 install ./dist/* --force-reinstall

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh fwiw, I think we should just go ahead and recommend python3 -m build instead of now deprecated ways.

```
23 changes: 23 additions & 0 deletions examples/distributing/uniffi-rust-to-python-library/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
[package]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the directory name could be just "python"

name = "python"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
uniffi = {version = "*", features = [ "cli" ]}
uniffi_macros = "*"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These all seem odd - you probably want to do what the other examples do, and I don't think this example needs a direct dependency on uniffi_macros or on uniffi_bindgen

uniffi_bindgen = "*"

[build-dependencies]
uniffi = { version = "*", features = [ "build", "cli" ] }

[[bin]]
# This can be whatever name makes sense for your project, but the rest of this tutorial assumes uniffi-bindgen.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As a noob, this is pretty confusing because it makes it ambiguous to whether the intention is to reference this uniffi-bindgen or the uniffi binary tool uniffi-bindgen

name = "uniffi-bindgen"
path = "uniffi-bindgen.rs"

[lib]
crate-type = ["cdylib"]
name = "mymath"
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
## Minimal example of uniffi-rs with setup.py

# Building

.\run.bat
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is Windows specific so should probably be removed

Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
fn main() {
uniffi::generate_scaffolding("src/math.udl").unwrap();
}
16 changes: 16 additions & 0 deletions examples/distributing/uniffi-rust-to-python-library/src/lib.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
pub fn add(left: u32, right: u32) -> u32 {
left + right
}

uniffi::include_scaffolding!("math");

#[cfg(test)]
mod tests {
use super::*;

#[test]
fn it_works() {
let result = add(2, 2);
assert_eq!(result, 4);
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
namespace mymath {
u32 add(u32 left, u32 right);
};
235 changes: 235 additions & 0 deletions examples/distributing/uniffi-rust-to-python-library/src/setup.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,235 @@
from setuptools import setup, Distribution, find_packages
from setuptools.command.install import install

from distutils.command.build import build as _build
import os
import re
import shutil
import subprocess
import sys
from pathlib import Path
import wheel.bdist_wheel

sys.dont_write_bytecode = True

if sys.version_info < (3, 6):
print("the example project requires at least Python 3.6", file=sys.stderr)
sys.exit(1)

from pathlib import Path # noqa

# Path to the directory containing this file
PYTHON_ROOT = Path(__file__).parent.absolute()

# Relative path to this directory from cwd.
FROM_TOP = PYTHON_ROOT.relative_to(Path.cwd())

# Path to the root of the git checkout
SRC_ROOT = PYTHON_ROOT.parents[1]

requirements = [
"wheel",
"setuptools",
]

buildvariant = "release"


class BinaryDistribution(Distribution):
def is_pure(self):
return False

def has_ext_modules(self):
return True


def macos_compat(target):
if target.startswith("aarch64-"):
return "11.0"
return "10.7"


# The logic for specifying wheel tags in setuptools/wheel is very complex, hard
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need this complexity for this sample - I think it's fine if the target only runs on the Python version it was built with.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree. This looks like the horrible hack we did over in Glean. And the proper way and IMO what we should recommend is using something like maturin (e.g. see mozilla/glean#2345).
Though for a small example I'm fine with a minimal setup.py still.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I must watch your PR but it is a draft.

Copy link
Author

@gogo2464 gogo2464 Jul 11, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@badboy I watched your incomming PR. It seems also very much better with .pytoml. @mhammond can I keep the current work as legacy or should I immediately rewrite it please?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

at this moment I work with maturin.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think @badboy and I are both saying that for this trivial example, there's no need to use maturin. It might make good sense for your project, but for the purposes of this example, we should aim for it being as small as reasonably possible.

# to override, and is really meant for extensions that are compiled against
# libpython.so, not this case where we have a fairly portable Rust-compiled
# binary that should work across a number of Python versions. Therefore, we
# just skip all of its logic be overriding the `get_tag` method with something
# simple that only handles the cases we need.
class bdist_wheel(wheel.bdist_wheel.bdist_wheel):
def get_tag(self):
cpu, _, __ = target.partition("-")
impl, abi_tag = "cp36", "abi3"
if "-linux" in target:
plat_name = f"linux_{cpu}"
elif "-darwin" in target:
compat = macos_compat(target).replace(".", "_")
if cpu == "aarch64":
cpu = "arm64"
plat_name = f"macosx_{compat}_{cpu}"
elif "-windows" in target:
impl, abi_tag = "py3", "none"
if cpu == "i686":
plat_name = "win32"
elif cpu == "x86_64":
plat_name = "win_amd64"
else:
raise ValueError("Unsupported Windows platform")
else:
# Keep local wheel build on BSD/etc. working
_, __, plat_name = super().get_tag()

return (impl, abi_tag, plat_name)


class InstallPlatlib(install):
def finalize_options(self):
install.finalize_options(self)
if self.distribution.has_ext_modules():
self.install_lib = self.install_platlib


def get_rustc_info():
"""
Get the rustc info from `rustc --version --verbose`, parsed into a
dictionary.
"""
regex = re.compile(r"(?P<key>[^:]+)(: *(?P<value>\S+))")

output = subprocess.check_output(["rustc", "--version", "--verbose"])

data = {}
for line in output.decode("utf-8").splitlines():
match = regex.match(line)
if match:
d = match.groupdict()
data[d["key"]] = d["value"]

return data


target = get_rustc_info()["host"]

extension = ""
file_start = ""
if "-darwin" in target:
shared_object = "libmath.dylib"
extension = ".dylib"
file_start = "lib"
elif "-windows" in target:
shared_object = "mymath.dll"
extension = ".dll"
file_start = ""
else:
# Anything else must be an ELF platform - Linux, *BSD, Solaris/illumos
shared_object = "libmath.so"
extension = ".so"
file_start = "lib"

new_shared_object_name = file_start + "uniffi_mymath" + extension


class build(_build):
def run(self):
try:
# Use `check_output` to suppress output
subprocess.check_output(["cargo"])
except subprocess.CalledProcessError:
print("Install Rust and Cargo through Rustup: https://rustup.rs/.")
sys.exit(1)

env = os.environ.copy()

# For `musl`-based targets (e.g. Alpine Linux), we need to set a flag
# to produce a shared object Python extension.
if "-musl" in target:
env["RUSTFLAGS"] = (
env.get("RUSTFLAGS", "") + " -C target-feature=-crt-static"
)
if target == "i686-pc-windows-gnu":
env["RUSTFLAGS"] = env.get("RUSTFLAGS", "") + " -C panic=abort"

command = [
"cargo",
"build",
#"--package",
#"math",
"--target-dir",
"out",
"--target",
target,
]

if buildvariant != "debug":
command.append(f"--{buildvariant}")

if "-darwin" in target:
env["MACOSX_DEPLOYMENT_TARGET"] = macos_compat(target)

subprocess.check_call(command, env=env)

#os.makedirs(os.path.dirname(SRC_ROOT / "uniffi-rust-to-python-library" / "out"))#, exist_ok=True

#print("root: {0}".format(SRC_ROOT))

#print([name for name in os.listdir(".") if os.path.isdir(name) and "target" in os.path.isdir(name)][0])


print("{0}".format(SRC_ROOT / "out" / target / buildvariant / "deps" / shared_object))


shutil.copyfile(
SRC_ROOT / "uniffi-rust-to-python-library" / "out" / target / buildvariant / "deps" / shared_object, #SRC_ROOT / "uniffi-rust-to-python-library" / "target" / target / buildvariant / "deps" / shared_object,
SRC_ROOT / "uniffi-rust-to-python-library" / "out" / new_shared_object_name,
)

command = [
"cargo",
"run",
"--features=uniffi/cli",
"--bin",
"uniffi-bindgen",
"generate",
"src/math.udl",
"--language",
"python",
"--out-dir",
SRC_ROOT / "uniffi-rust-to-python-library" / "target",
]

subprocess.check_call(command, env=env)

shutil.copyfile(
SRC_ROOT / "uniffi-rust-to-python-library" / "target" / "mymath.py", SRC_ROOT / "uniffi-rust-to-python-library" / "out" / "mymath.py"
)

return _build.run(self)

setup(
author="gogo2464",
author_email="[email protected]",
classifiers=[
"Intended Audience :: Developers",
"Natural Language :: English",
"Programming Language :: Python :: 3"
],
description="Example project in order to complete a uniffi-rs tutorial.",
long_description="example",
install_requires=requirements,
long_description_content_type="text/markdown",
include_package_data=True,
keywords="example",
name="mymath",
version="0.1.0",
packages=[
"mymath"
],
package_dir={
"mymath": "out"
},
setup_requires=requirements,
url="no_url",
zip_safe=False,
package_data={"mymath": [new_shared_object_name]},
distclass=BinaryDistribution,
cmdclass={"install": InstallPlatlib, "bdist_wheel": bdist_wheel, "build": build},
)
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
from mymath import mymath

value = mymath.add(1,2)
assert value == 3, f"add not ok"
print("1 + 2 = {0}. OK!".format(value))
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
fn main() {
uniffi::uniffi_bindgen_main()
}