Skip to content

LPs on Halide #1814

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
---
title: Halide Essentials. From Basics to Android Integration
minutes_to_complete: 180

who_is_this_for: This is an introductory topic for software developers interested in learning how to use Halide for image processing.

learning_objectives:

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please update the language to answer the rendered question (Upon completion of this learning path, you will be able to...)

- Introduction, Background, and Installation.
- Building a Simple Camera/Image Processing Workflow.
- Demonstrating Operation Fusion.
- Integrating Halide into an Android (Kotlin) Project

prerequisites:

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider adding:

  • basic C++ knowledge
  • Android Studio with emulator
    Or remove the prerequisites part altogether.



author: Dawid Borycki

### Tags
skilllevels: Introductory
subjects: Performance and Architecture

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add image processing, or this is chosen from a predefined list?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree and "computer vision" as well.

armips:
- Cortex-A
- Cortex-X
operatingsystems:
- Android
tools_software_languages:
- Android Studio
- Coding

further_reading:

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider directly linking the Halide tutorials for further reading.

- resource:
title: Halide 19.0.0
link: https://halide-lang.org/docs/index.html
type: website
- resource:
title: Halide GitHub
link: https://github.com/halide/Halide
type: repository


### FIXED, DO NOT MODIFY
# ================================================================================
weight: 1 # _index.md always has weight of 1 to order correctly
layout: "learningpathall" # All files under learning paths have this same wrapper
learning_path_main_page: "yes" # This should be surfaced when looking for related content. Only set for _index.md of learning path content.
---
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
---
# ================================================================================
# FIXED, DO NOT MODIFY THIS FILE
# ================================================================================
weight: 21 # Set to always be larger than the content in this path to be at the end of the navigation.
title: "Next Steps" # Always the same, html page title.
layout: "learningpathall" # All files under learning paths have this same wrapper for Hugo processing.
---

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
@@ -0,0 +1,142 @@
---
# User change
title: "Ahead-of-time and cross-compilation"

weight: 5

layout: "learningpathall"
---

## Ahead-of-time and cross-compilation
One of Halide’s standout features is the ability to compile image processing pipelines ahead-of-time (AOT), enabling developers to generate optimized binary code on their host machines rather than compiling directly on target devices. This AOT compilation process allows developers to create highly efficient libraries that run effectively across diverse hardware without incurring the runtime overhead associated with just-in-time (JIT) compilation.

Halide also supports robust cross-compilation capabilities. Cross-compilation means using the host version of Halide, typically running on a desktop Linux or macOS system—to target different architectures, such as ARM for Android devices. Developers can thus optimize Halide pipelines on their host machine, produce libraries specifically optimized for Android, and integrate them seamlessly into Android applications. The generated pipeline code includes essential optimizations and can embed minimal runtime support, further reducing workload on the target device and ensuring responsiveness and efficiency.

## Objective
In this section, we leverage the host version of Halide to perform AOT compilation of an image processing pipeline via cross-compilation. The resulting pipeline library is specifically tailored to Android devices (targeting, for instance, arm64-v8a ABI), while the compilation itself occurs entirely on the host system. This approach significantly accelerates development by eliminating the need to build Halide or perform JIT compilation on Android devices. It also guarantees that the resulting binaries are optimized for the intended hardware, streamlining the deployment of high-performance image processing applications on mobile platforms.

## Prepare Pipeline for Android
The procedure implemented in the following code demonstrates how Halide’s AOT compilation and cross-compilation features can be utilized to create an optimized image processing pipeline for Android. We will run Halide on our host machine (in this example, macOS) to generate a static library containing the pipeline function, which will later be invoked from an Android device. Below is a step-by-step explanation of this process.

Create a new file named blur-android.cpp with the following contents:

```cpp
#include "Halide.h"
#include <iostream>

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same issue regarding the headers as before.

using namespace Halide;

int main(int argc, char** argv) {
if (argc < 2) {
std::cerr << "Usage: " << argv[0] << " <output_basename> \n";
return 1;
}

std::string output_basename = argv[1];

// Configure Halide Target for Android
Halide::Target target;
target.os = Halide::Target::OS::Android;
target.arch = Halide::Target::Arch::ARM;
target.bits = 64;
target.set_feature(Target::NoRuntime, false);

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider mentioning a few other features, such as NEON, SVE2 or DotProd and please explain what the NoRunTime target does.


// --- Define the pipeline ---
// Define variables
Var x("x"), y("y");

// Define input parameter
ImageParam input(UInt(8), 2, "input");

// Create a clamped function that limits the access to within the image bounds
Func clamped("clamped");
clamped(x, y) = input(clamp(x, 0, input.width()-1),
clamp(y, 0, input.height()-1));

// Now use the clamped function in processing
RDom r(0, 3, 0, 3);
Func blur("blur");

// Initialize blur accumulation
blur(x, y) = cast<uint16_t>(0);
blur(x, y) += cast<uint16_t>(clamped(x + r.x - 1, y + r.y - 1));

// Then continue with pipeline
Func blur_div("blur_div");
blur_div(x, y) = cast<uint8_t>(blur(x, y) / 9);

// Thresholding
Func thresholded("thresholded");
Expr t = cast<uint8_t>(128);
thresholded(x, y) = select(blur_div(x, y) > t, cast<uint8_t>(255), cast<uint8_t>(0));

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this different than before if the goal is to achieve the same thing?


// Simple scheduling
blur_div.compute_root();
thresholded.compute_root();

// --- AOT compile to a file ---
thresholded.compile_to_static_library(
output_basename, // base filename
{ input }, // list of inputs
"blur_threshold", // name of the generated function
target
);

return 0;
}
```

The program takes at least one command-line argument, the output base name used to generate the files (e.g., “blur_threshold_android”). Here, the target architecture is explicitly set within the code to Android ARM64:

```cpp
// Configure Halide Target for Android
Halide::Target target;
target.os = Halide::Target::OS::Android;
target.arch = Halide::Target::Arch::ARM;
target.bits = 64;
target.set_feature(Target::NoRuntime, false);
```

We declare spatial variables (x, y) and an ImageParam named “input” representing the input image data. We use boundary clamping (clamp) to safely handle edge pixels. Then, we apply a 3x3 blur with a reduction domain (RDom). The accumulated sum is divided by 9 (the number of pixels in the neighborhood), producing an average blurred image. Lastly, thresholding is applied, producing a binary output: pixels above a certain brightness threshold (128) become white (255), while others become black (0).

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be clear to the reader at this point, the operations didn't really change from last time.


Simple scheduling directives (compute_root) instruct Halide to compute intermediate functions at the pipeline’s root, simplifying debugging and potentially enhancing runtime efficiency.

We invoke Halide’s AOT compilation function compile_to_static_library, which generates a static library (.a) containing the optimized pipeline and a corresponding header file (.h).

```cpp
thresholded.compile_to_static_library(
output_basename, // base filename for output files (e.g., "blur_threshold_android")
{ input }, // list of input parameters to the pipeline
"blur_threshold", // the generated function name
target // our target configuration for Android
);
```

This will produce:
* A static library (blur_threshold_android.a) containing the compiled pipeline.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this contain runtime functions of Halide for the specific target (i.e. arm-64-android), right? If true, I think it is worth noting.

* A header file (blur_threshold_android.h) declaring the pipeline function for use in other C++/JNI code.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please explain what JNI means.


These generated files are then ready to integrate directly into an Android project via JNI, allowing efficient execution of the optimized pipeline on Android devices. The integration process is covered in the next section.

## Compilation instructions
To compile the pipeline-generation program on your host system, use the following commands (replace /path/to/halide with your Halide installation directory):
```console
export DYLD_LIBRARY_PATH=/path/to/halide/lib/libHalide.19.dylib
g++ -std=c++17 camera-capture.cpp -o camera-capture \
-I/path/to/halide/include -L/path/to/halide/lib -lHalide \
$(pkg-config --cflags --libs opencv4) -lpthread -ldl \
-Wl,-rpath,/path/to/halide/lib
```

Then execute the binary:
```console
./blur_android blur_threshold_android
```

This will produce two files:
* blur_threshold_android.a: The static library containing your Halide pipeline.
* blur_threshold_android.h: The header file needed to invoke the generated pipeline.

We will integrate these files into our Android project in the following section.

## Summary
In this section, we’ve explored Halide’s powerful ahead-of-time (AOT) and cross-compilation capabilities, preparing an optimized image processing pipeline tailored specifically for Android devices. By using the host-based Halide compiler, we’ve generated a static library optimized for ARM64 Android architecture, incorporating safe boundary conditions, neighborhood-based blurring, and thresholding operations. This streamlined process allows seamless integration of highly optimized native code into Android applications, ensuring both development efficiency and runtime performance on mobile platforms.
Loading