-
Notifications
You must be signed in to change notification settings - Fork 195
LPs on Halide #1814
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
LPs on Halide #1814
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,46 @@ | ||
--- | ||
title: Halide Essentials. From Basics to Android Integration | ||
minutes_to_complete: 180 | ||
|
||
who_is_this_for: This is an introductory topic for software developers interested in learning how to use Halide for image processing. | ||
|
||
learning_objectives: | ||
- Introduction, Background, and Installation. | ||
- Building a Simple Camera/Image Processing Workflow. | ||
- Demonstrating Operation Fusion. | ||
- Integrating Halide into an Android (Kotlin) Project | ||
|
||
prerequisites: | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Consider adding:
|
||
|
||
|
||
author: Dawid Borycki | ||
|
||
### Tags | ||
skilllevels: Introductory | ||
subjects: Performance and Architecture | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can you add image processing, or this is chosen from a predefined list? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Agree and "computer vision" as well. |
||
armips: | ||
- Cortex-A | ||
- Cortex-X | ||
operatingsystems: | ||
- Android | ||
tools_software_languages: | ||
- Android Studio | ||
- Coding | ||
|
||
further_reading: | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Consider directly linking the Halide tutorials for further reading. |
||
- resource: | ||
title: Halide 19.0.0 | ||
link: https://halide-lang.org/docs/index.html | ||
type: website | ||
- resource: | ||
title: Halide GitHub | ||
link: https://github.com/halide/Halide | ||
type: repository | ||
|
||
|
||
### FIXED, DO NOT MODIFY | ||
# ================================================================================ | ||
weight: 1 # _index.md always has weight of 1 to order correctly | ||
layout: "learningpathall" # All files under learning paths have this same wrapper | ||
learning_path_main_page: "yes" # This should be surfaced when looking for related content. Only set for _index.md of learning path content. | ||
--- |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,8 @@ | ||
--- | ||
# ================================================================================ | ||
# FIXED, DO NOT MODIFY THIS FILE | ||
# ================================================================================ | ||
weight: 21 # Set to always be larger than the content in this path to be at the end of the navigation. | ||
title: "Next Steps" # Always the same, html page title. | ||
layout: "learningpathall" # All files under learning paths have this same wrapper for Hugo processing. | ||
--- |
Large diffs are not rendered by default.
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,142 @@ | ||
--- | ||
# User change | ||
title: "Ahead-of-time and cross-compilation" | ||
|
||
weight: 5 | ||
|
||
layout: "learningpathall" | ||
--- | ||
|
||
## Ahead-of-time and cross-compilation | ||
One of Halide’s standout features is the ability to compile image processing pipelines ahead-of-time (AOT), enabling developers to generate optimized binary code on their host machines rather than compiling directly on target devices. This AOT compilation process allows developers to create highly efficient libraries that run effectively across diverse hardware without incurring the runtime overhead associated with just-in-time (JIT) compilation. | ||
|
||
Halide also supports robust cross-compilation capabilities. Cross-compilation means using the host version of Halide, typically running on a desktop Linux or macOS system—to target different architectures, such as ARM for Android devices. Developers can thus optimize Halide pipelines on their host machine, produce libraries specifically optimized for Android, and integrate them seamlessly into Android applications. The generated pipeline code includes essential optimizations and can embed minimal runtime support, further reducing workload on the target device and ensuring responsiveness and efficiency. | ||
|
||
## Objective | ||
In this section, we leverage the host version of Halide to perform AOT compilation of an image processing pipeline via cross-compilation. The resulting pipeline library is specifically tailored to Android devices (targeting, for instance, arm64-v8a ABI), while the compilation itself occurs entirely on the host system. This approach significantly accelerates development by eliminating the need to build Halide or perform JIT compilation on Android devices. It also guarantees that the resulting binaries are optimized for the intended hardware, streamlining the deployment of high-performance image processing applications on mobile platforms. | ||
|
||
## Prepare Pipeline for Android | ||
The procedure implemented in the following code demonstrates how Halide’s AOT compilation and cross-compilation features can be utilized to create an optimized image processing pipeline for Android. We will run Halide on our host machine (in this example, macOS) to generate a static library containing the pipeline function, which will later be invoked from an Android device. Below is a step-by-step explanation of this process. | ||
|
||
Create a new file named blur-android.cpp with the following contents: | ||
|
||
```cpp | ||
#include "Halide.h" | ||
#include <iostream> | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Same issue regarding the headers as before. |
||
using namespace Halide; | ||
|
||
int main(int argc, char** argv) { | ||
if (argc < 2) { | ||
std::cerr << "Usage: " << argv[0] << " <output_basename> \n"; | ||
return 1; | ||
} | ||
|
||
std::string output_basename = argv[1]; | ||
|
||
// Configure Halide Target for Android | ||
Halide::Target target; | ||
target.os = Halide::Target::OS::Android; | ||
target.arch = Halide::Target::Arch::ARM; | ||
target.bits = 64; | ||
target.set_feature(Target::NoRuntime, false); | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Consider mentioning a few other features, such as NEON, SVE2 or DotProd and please explain what the NoRunTime target does. |
||
|
||
// --- Define the pipeline --- | ||
// Define variables | ||
Var x("x"), y("y"); | ||
|
||
// Define input parameter | ||
ImageParam input(UInt(8), 2, "input"); | ||
|
||
// Create a clamped function that limits the access to within the image bounds | ||
Func clamped("clamped"); | ||
clamped(x, y) = input(clamp(x, 0, input.width()-1), | ||
clamp(y, 0, input.height()-1)); | ||
|
||
// Now use the clamped function in processing | ||
RDom r(0, 3, 0, 3); | ||
Func blur("blur"); | ||
|
||
// Initialize blur accumulation | ||
blur(x, y) = cast<uint16_t>(0); | ||
blur(x, y) += cast<uint16_t>(clamped(x + r.x - 1, y + r.y - 1)); | ||
|
||
// Then continue with pipeline | ||
Func blur_div("blur_div"); | ||
blur_div(x, y) = cast<uint8_t>(blur(x, y) / 9); | ||
|
||
// Thresholding | ||
Func thresholded("thresholded"); | ||
Expr t = cast<uint8_t>(128); | ||
thresholded(x, y) = select(blur_div(x, y) > t, cast<uint8_t>(255), cast<uint8_t>(0)); | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why is this different than before if the goal is to achieve the same thing? |
||
|
||
// Simple scheduling | ||
blur_div.compute_root(); | ||
thresholded.compute_root(); | ||
|
||
// --- AOT compile to a file --- | ||
thresholded.compile_to_static_library( | ||
output_basename, // base filename | ||
{ input }, // list of inputs | ||
"blur_threshold", // name of the generated function | ||
target | ||
); | ||
|
||
return 0; | ||
} | ||
``` | ||
|
||
The program takes at least one command-line argument, the output base name used to generate the files (e.g., “blur_threshold_android”). Here, the target architecture is explicitly set within the code to Android ARM64: | ||
|
||
```cpp | ||
// Configure Halide Target for Android | ||
Halide::Target target; | ||
target.os = Halide::Target::OS::Android; | ||
target.arch = Halide::Target::Arch::ARM; | ||
target.bits = 64; | ||
target.set_feature(Target::NoRuntime, false); | ||
``` | ||
|
||
We declare spatial variables (x, y) and an ImageParam named “input” representing the input image data. We use boundary clamping (clamp) to safely handle edge pixels. Then, we apply a 3x3 blur with a reduction domain (RDom). The accumulated sum is divided by 9 (the number of pixels in the neighborhood), producing an average blurred image. Lastly, thresholding is applied, producing a binary output: pixels above a certain brightness threshold (128) become white (255), while others become black (0). | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This should be clear to the reader at this point, the operations didn't really change from last time. |
||
|
||
Simple scheduling directives (compute_root) instruct Halide to compute intermediate functions at the pipeline’s root, simplifying debugging and potentially enhancing runtime efficiency. | ||
|
||
We invoke Halide’s AOT compilation function compile_to_static_library, which generates a static library (.a) containing the optimized pipeline and a corresponding header file (.h). | ||
|
||
```cpp | ||
thresholded.compile_to_static_library( | ||
output_basename, // base filename for output files (e.g., "blur_threshold_android") | ||
{ input }, // list of input parameters to the pipeline | ||
"blur_threshold", // the generated function name | ||
target // our target configuration for Android | ||
); | ||
``` | ||
|
||
This will produce: | ||
* A static library (blur_threshold_android.a) containing the compiled pipeline. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Does this contain runtime functions of Halide for the specific target (i.e. arm-64-android), right? If true, I think it is worth noting. |
||
* A header file (blur_threshold_android.h) declaring the pipeline function for use in other C++/JNI code. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Please explain what JNI means. |
||
|
||
These generated files are then ready to integrate directly into an Android project via JNI, allowing efficient execution of the optimized pipeline on Android devices. The integration process is covered in the next section. | ||
|
||
## Compilation instructions | ||
To compile the pipeline-generation program on your host system, use the following commands (replace /path/to/halide with your Halide installation directory): | ||
```console | ||
export DYLD_LIBRARY_PATH=/path/to/halide/lib/libHalide.19.dylib | ||
g++ -std=c++17 camera-capture.cpp -o camera-capture \ | ||
-I/path/to/halide/include -L/path/to/halide/lib -lHalide \ | ||
$(pkg-config --cflags --libs opencv4) -lpthread -ldl \ | ||
-Wl,-rpath,/path/to/halide/lib | ||
``` | ||
|
||
Then execute the binary: | ||
```console | ||
./blur_android blur_threshold_android | ||
``` | ||
|
||
This will produce two files: | ||
* blur_threshold_android.a: The static library containing your Halide pipeline. | ||
* blur_threshold_android.h: The header file needed to invoke the generated pipeline. | ||
|
||
We will integrate these files into our Android project in the following section. | ||
|
||
## Summary | ||
In this section, we’ve explored Halide’s powerful ahead-of-time (AOT) and cross-compilation capabilities, preparing an optimized image processing pipeline tailored specifically for Android devices. By using the host-based Halide compiler, we’ve generated a static library optimized for ARM64 Android architecture, incorporating safe boundary conditions, neighborhood-based blurring, and thresholding operations. This streamlined process allows seamless integration of highly optimized native code into Android applications, ensuring both development efficiency and runtime performance on mobile platforms. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please update the language to answer the rendered question (Upon completion of this learning path, you will be able to...)