AAP 0.7.x releases

[atsushieno]

I'll be recording release notes here, kind of.

Today I tagged 0.7.3 release.

The 0.7.2 release had a lot of fundamental changes regarding extensions ("AAPXS", as documented in EXTENSIONS.md).

The foundation was almost set in stone there, but 0.7.3 brought it in real world usage, namely for plugin state load/save support, which made it possible to remove getState() / getState() methods from the AAP AIDL. It is basically the biggest change that impacts AAP, in terms of backward compatibility. Basically we use org.androidaudioplugin.AudioPluginService.V1 as the new intent action to differentiate from older ones i.e. older AAPs will not be found on newer plugin clients (and vice versa).

Apart from that, 0.7.2 should have involved a lot of changes to the client API too, but it was also a lot of work, especially the Kotlin client API was based on AIDL directly. Since the new extension foundation requires the native API (namely aap::RemotePluginInstace), it was not possible to support it without the whole rewrite that happened in this 0.7.3 release.

A handful of (but not all) existing aap-lv2 and aap-juce plugin ports are also updated to 0.7.3. Namely, Dexed and Odin2 were also updated to the latest upstream sources and they are now CMake based ports.

I am not certain what 0.7.4 will be like, but my current plan is to implement dynamic ports configuration (described at NEW_PORTS.md design doc).

[atsushieno]

4 month are unexpectedly long time for only 0.1 version bump, but today I finally tagged v0.7.4 release.

v0.7.4 brings in a big paradigm shift that are partly planned as "new ports" things. Namely...

• Now ports are used only for audio and MIDI.
• And MIDI port are only MIDI 2.0 UMP ports now. aap-lv2 and aap-juce automatically translate them.
• Ports for parameters are gone. They are instead changed through UMP parameter change sysex8 messages.

There are some other changes, for example...

• There is now "plugin-info" host extension that a plugin (especially plugin wrappers) can retrieve host-managed plugin information (as stored in aap::PluginInformation) within the plugin itself. AAP plugin API does not provide access to the information of the plugin itself. We could make the plugin API changes so that the plugin itself can implement the API and use it by itself, but so far host has it (via aap_metadata.xml) and providing it as a host plugin makes it simpler (also contributes to "single source of truth").
• PluginPreview has "MIDI" button, instead of "UI" button which was not really in use. Before this change, we had to always resort to external MIDI keyboard app (such as atsushieno/kmmk) which makes debugging not intuitive, but now it is easy.
• AudioPluginMidiDeviceService seems to cause no (or, much less) glitches than before. I know it is still not really safely implemented and causes key drops, but the overall quality is better.

It's been tough months to struggle with the frequent design changes (in early stage the parameter changes were implemented as Assignable Controllers, until I noticed that I'm going to kill NRPN at all, just like VST3 CCs), but I think it's in general good now.

[atsushieno]

Just a followup, those apps (hosts and plugins) are updated to the new V2 protocol:

• aap-lv2-mda
• aap-lv2-sfizz
• aap-lv2-fluidsynth
• aap-lv2-guitarix
• aap-lv2-string-machine
• aap-lv2-dragonfly-reverb (still has its own issue)
• aap-juce-plugin-host (not really practically useful at this state, outputs are all weird)
• aap-juce-obxd
• aap-juce-dexed
• aap-juce-odin2
• aap-juce-hera
• aap-juce-vital
• aap-juce-frequalizer
• aap-juce-simple-reverb (parameter changes do not work)

I don't think I keep updating everything (I didn't this time either).

[atsushieno]

Today I released AAP v0.7.5, AAP-LV2 v0.2.5, and AAP-JUCE v0.4.5.

These months have been intensive on AAP land, and while there are still many many things to do I'm kind of proud of the overall improvements. Things happened in these two months (actually mostly in January; I was kinda busy in December on other stuff):

• we started using Wiki pages to describe the current development status etc. Notably List of AAP plugins and hosts and Are We Plugin Format Yet?
• rebranding: it is Audio Plugins For Android (posted here)
• practical Presets integration: aaphostsample (or MainActivity for non-juce plugins) lets you select a preset before applying it.
• A new MIDI mapping feature: now we map Control Changes, Assignable Controllers (NRPNs in MIDI 1.0), including Per-Note ones, and some Universal SysEx8 messages to parameter changes, as well as Program Changes to Preset changes, in our MidiDeviceService. Plugins can still indicate that which messages are supposed to be passed directly so that we don't bring in unwanted absorption and conflicts.
• MIDI extension: formerly we had aap-midi2.h which was supposed to manage switches between MIDI 1.0 and MIDI 2.0. But now we are totally based on MIDI 2.0 (like CoreMIDI is nowadays), it should play different roles. So far, the MIDI mapping feature explained above is now part of this extension.
• With the latest aap-juce version I brought two new ports: aap-juce-byod and aap-juce-os251. BYOD is undoubtedly one of the best open source plugins in 2022, and it's also easy to play with the built-in presets. OS-251 is a leading open source use case for react-juce, which means we can cover such a complicated setup scenario too, without too much tweaks I believe.
• Now we have an experimental AAP APK Installer that lets you install those AAP hosts and plugins from GitHub build artifacts and releases, so that you don't have to build everything locally. Note that I still don't plan to publish them on Google Play Store as things don't really "work as expected" and collect a bunch of 1-star reviews, so it will be the closest way to store distribution for now (and for a while).
• AAP-JUCE hosting had some blocking issue (literally I say; it was not working due to blocking main thread) that should be fixed in this release. I also revisited my experimental Helio Workstation integration and now I can instantiate and even play sequences on it using AAP (see the video recording linked on its README) - kind of (still horrible output quality). This is what I want to achieve; any DAW with any audio plugins.

[atsushieno]

I pinned this issue because GitHub Discussions somehow gone buggy and it never treats this thread as "open".

[atsushieno]

Today (actually yesterday in JST) I tagged v0.7.6 in aap-core, along with aap-lv2 v0.2.6 and aap-juce v0.4.6.

There has been a lot of work over stability, audio quality improvements, clearer API and design documentation, and last but not least, initial set of GUI support (finally).

Almost all the existing host and plugin ports that were maintained until v0.7.5 are updated to use v0.7.6. Except for aap-juce-adlplug that is facing some build issues. (I was trying to create a CMake build but that's also facing some problem. It may take a while.)

Just a quick note that currently AAP APK Installer has problem that crashes at attempt to see any app details. It is likely due to the new state of having multiple release artifacts. I will fix it hopefully sooner (but I'm also packed right now, might take a while). => see my followup comment.

aap-core 0.7.6

• Web UI support work has finally started. Everything is subject to change, but we have draft version of Web UI spec on docs/design/GUI.md.
  • At this state we can send MIDI2 parameter changes and note on/off messages (or more complicated stuff if you implement on your own Web UI).
  • all those up-to-date ports come with the default Web UI that only lists parameters along with a demo keyboard.
  • It shows up on aap-juce-helio too, but helio-workstation does not seem to activate audio processing when its playback is stopped and thus those knobs and the MIDI keyboard do not work like they do on aap-juce-simple-host.
• We also have preliminary native UI support which is based on System Alert Window (therefore not really recommended). We have some demo UI that simply shows the WebView (which is just loading the Web UI) as the native view.
  • Currently there is no plugin that provides reusable native UI - juce_gui_basics does not work as an isolated UI from its JuceActivity model (it does not even let us launch other Activity). To change this, we will have to make a bunch of changes to JUCE sources, which is not going to be a maintainable JUCE fork anymore.
  • LV2 plugins - we do not have viable solution for Android. A foreseeable future option is DPF based plugins that could incorporate the Web UI solution, but not any solid plan.
• AAP metadata: manufacturer is replaced by developer (BREAKING CHANGE)
• There have been various improvements over realtime stability. We have solid concurrency manager over the RT audio thread and other threads in androidaudioplugin-midi-device-service and some other places.
• There are improvements over instance lifetime management in PluginPreview (ui-compose backend).
• New plugin buffer API: AndroidAudioPluginBuffer is gone and a void* buffer is acquired via aap_buffer_t struct via its member functions. (BREAKING CHANGE)
• Minor but incompatible change to AIDL so that process() now takes frameCount. There was an implementation mismatch between what aap-core expects and aap-juce actually does, which was often causing unwanted audio glitches. With this change, we are now based on "V3" protocol. (BREAKING CHANGE)
• Extension functions are now annotated with RT_SAFE and RT_UNSAFE distinction.
  • We still haven't implemented but to address RT-safe audio processing, we have solid guideline on how extension functions can be invoked between host and plugin (extension over MIDI2 transport on RT mode, or AIDL on non-RT mode).
• We have clearer definitions on how an extension is composed: it comes with C header, AAPXS, and extension XML items (with RELAX NG schemas for standard extensions).
• We use oboe StabilizedCallback for AAP MidiDeviceService, which brought in great audio output stability especially over aap-juce plugins.
• Added ATrace support. Profiling audio processing is quite easy with Trace profiler with Perfetto. Check out the relevant section on docs/DEVELOPERS.md for details.
• Revamped AAPXS (extension services) API. Eliminated AndroidAudioPluginExtensionTarget struct for more intuitive API.

aap-lv2 0.2.6

(Not really a remarkable change, it's just following aap-core changes.)

aap-juce 0.4.6

• Now we enforce JUCE_DONT_AUTO_OPEN_MIDI_DEVICES_ON_MOBILE. JUCE standalone application has a terrible default behavior that it tries to open any MIDI device (yes, either input or output device), including our own AAP MidiDeviceServices. It then acquires AAudio realtime audio loop and blocks RT audio processing in our actual JUCE host apps. Therefore this device opener is forcibly disabled. Everyone who makes use of JUCE on Android should enforce this too, otherwise any audio apps on your users device may perform in poor quality.
  • This improved overall audio processing quality in JUCE plugin a lot.
• Fixed the awkward module name flipping - juceaap_audio_plugin_client is now aap_audio_processor (because it was for plugins) and juceaap_audio_processors is now aap_audio_plugin_client (because it was for plugin clients).
• Those modules are also relocated from modules to aap-modules in aap-juce repo (because there was some plugin that had conflicting Modules directory; it could happen in other plugins).

AAP project overall

• Build scripts are now mostly compatible with macOS development environment (I have been using M2 for a while). Most of the projects now build without problem. Our normative build is still Linux and any other platforms are optionally supported.

New apps/ports

• There is a new host for testing, aap-juce-simple-host which should give you hassle-free experience on devices unlike AudioPluginHost. You can stack plugins without connecting audio/MIDI ports.
• New plugin port: aap-juce-xenos, port of raphaelradna/xenos.

[atsushieno]

It turned out that I had been hitting "rate limit exceeded" errors on AAP APK Installer, without PAT. It may be just because I had been spamming GitHub Actions builds over ~30 repos where each release build takes 30 min.-ish, but if the UI is not responding without PAT you might want to try it with your PAT.

[atsushieno]

Today I tagged aap-core 0.7.7 along with aap-lv2 0.2.7 and aap-juce 0.4.7.

At this state, less plugin ports were updated to 0.7.7 (there are too many for one developer to take care). Those plugins were what I thought could get broken with the latest changes in aap-core and aap-juce (some of them indeed exposed regressions).

This set of releases includes:

• The Native UI foundation is now based on SurfaceControlViewHost. It requires Android 11 or later, but provides what we exactly needed - plugin UI on DAWs without special permissions. We reuse existing org.androidaudioplugin.AudioPluginViewFactory as the plugin View class, but the content is now transferred to the SurfaceView that the DAW hosts. It is still preliminary, but you'd be way more confident than now-deprecated System Alert Window approach.
• For the new native UI, we have a refresh default view implementation based on Jetpack Compose. It comes with the new compose-audio-controls Compose Multiplatform library. Existing Web UI based AudioPluginWebViewFactory is not deprecated and we will keep this for Web UI based plugins (that will require more hand-written cross-process interactions so far though).
• Dynamic code-driven parameters (or parameterless metadata) became really useful, in that we can fully populate the parameter list in aap-lv2 and aap-juce plugins. That results in those metadata generator almost unnecessary, which especially saves aap-juce porting work a lot. Formerly, we had to build "AAP incorporated" version of JUCE plugins for desktop, to run metadata generator on desktop (because it should be done on development host), and it was tricky and often impossible. With this simplification, we could finally port Surge XT to AAP. Note that these changes involved breaking changes in parameters extension and our extension URI differs now ("2.1").
• Enumeration and properties in parameters are newly supported (in both code-driven parameters and aap_metadata.xml). Property access is based on integer constant ID (can be common, can be unique, may conflict with others but would not matter unless they conflict each other within one plugin). aaphostsample, PluginDetails, and plugin native UI support the enumeration views in each way. We can also define new parameter properties without breaking API in the future. For example, AAP_PARAMETER_PROPERTY_PRIORITY is defined for "default compact view" GUI like AudioUnit V3 plugins can offer.
• updated MIDI 2.0 support to the latest June 2023 specification. Our MidiDeviceService now detects UMP Stream Configuration messages instead of MIDI-CI Set New Protocol which is now deprecated. We would also assume that some transport controls such as tempo changes are also sent as UMP Flex Data messages.
• Somewhat better support for LV2 parameters conversion to AAP: now it retrieves only ControlPort (traditional approach) to populate parameters (formerly we had some garbage parameters). Note that we still do not support modern Atom port based parameters - it is what we want to support but not in any progress yet.
• We are now publishing aap-core and aap-lv2 AARs to Maven Central which had happened only once in a while.

There have been new plugin ports since 0.7.6 release. Note that everything is nothing but a simple code porting without tailoring for Android / AAP.

• aap-juce-adlplug-ae: it is actually a refresh version of ADLplug that builds on JUCE7 and JUCE CMake (ADLplug supports CMake in its own way, which prevented CMake based porting). No FM bank file loader replacement for Android so far.
• aap-juce-ddsp: DDSP-VST port using aap-juce. It involves a handful of deps (such as tensorflow) which kind of complicates Android builds.
• aap-lv2-aida-x AIDA-X port using aap-lv2. Simple porting, with no actual support for preset file selector. It is also likely the latest normative import of DPF project import.
• aap-juce-surge: Surge-XT port using aap-juce, finally. As mentioned earlier, it became possible due to parameterless metadata.

[atsushieno]

Today I have tagged aap-core 0.7.8, aap-lv2 0.2.8, and aap-juce 0.4.8 on their repos respectively.

This version is kind of transitive release before making a handful of breaking changes (to be mentioned later).

This time I'm going to apply the new versions only to the limited hosts and plugins that are enough for dogfooding. In case we have to maintain this version we will eventually get more updated ports namely:

• aap-lv2-mda
• aap-lv2-string-machine
• aap-lv2-sfizz
• aap-juce-simple-host
• aap-juce-helio
• aap-juce-adlplug-ae
• aap-juce-obxd

Here are the notable changes:

(1) Refresh default plugin activity: we have long been using androidaudioplugin.ui.compose.PluginListActivity and its plugin details composables as the "default plugin activity UI". It did not provide audio plugin features in dynamic "live" way, and the gap between what AAP provides and the UI is capable was getting bigger and bigger.

So this time we have introduced the new featureful activity as androidaudioplugin.ui.compose.app.PluginManagerActivity (and so on).

plugin preview	plugin details

It features:

• dynamic and realtime plugin player implementation
• All the features are now implemented in C++, and Kotlin API only works as the facade.
• MIDI-like keyboard as well as effect application (to wav/ogg/etc.; we do not have file chooser yet though)
• can launch Web UI and Native UI, and they are now fully working i.e. they interact with the plugin instance

The old UI Activity is still alive in this version, but it will vanish in the next version, along with androidaudioplugin-samples-host-engine aar.

(2) Realtime AAP Extension Service (AAPXS) messaging

A bit of background: AAP extensions need to work across the processes (host app vs. plugin app), thus it is defined as part of Binder IPC AIDL messages (extension() method). When we run audio processing, the binder should be busy only for audio processing. Therefore, there is no room for extension() and thus extension calls were always regarded as non-RT safe.

I thought that it is fine if we just mark AAP extensions as non-RT-safe. However it is not just about RT safety of extensions; if those non-RT-safe methods are called, the audio processing thread may be still blocked, because there will be only one binder method invocation (because the IPC shares the proxy). Thus, we need RT-safe way to invoke those AAP extension functions, even if the functions are not RT-safe.

To make it possible, we make use of MIDI2 ports. It is similar to LV2 Atom ports, or CLAP event channel (which specifies there is only one for input and for output, respectively). We defined a new UMP message format called AAPXS SysEx8. This also opens the door for AAP extensions to become really RT-safe.

This effort is however, just started and we need further implementation. This version includes some preliminary version of it. We will have to introduce breaking changes to fully implement it (for example, issue #171), thus we are releasing this version before working on it further.

(3) Miscellaneous changes

MidiDeviceService now works in stereo mode.

aap-juce-helio is now updated to run the latest Helio Workstation with JUCE 7.0.5.

The project structure is now kind of awkward due to some Gradle/AGP issues that fail to function effect applier (issue #174) - they are temporarily structured to load the aar modules from git submodules. Also due to this issue we are likely unable to release AAR package for aap-lv2 (sorry for inconvenience in case it matters to anyone, but I'm also a victim from Gradle/AGP either).

We do not have any new plugin ports since 0.7.7 release (we could do that too, but focused on the work above instead).