AudYoFlo: Recipes: Creating an In‐Place Audionode
An in-place audionode is one that processes input audio data into output audio data. The term in-place indicates that the input data format is identical to that of the output data. Therefore, the processing may operate on the output buffers directly since input and output data buffers may be identical. Note, however, that even though foreseen as an in-place processor, it may be the case that input and output buffers are not identical buffers - e.g., if the node is operated in a multithreaded, pipelined environment.
In this section, we will start to add a signal processing node which we will then configure to become an audio node. This audio node will then become part of an application and involved in the processing chain. Finally, we will add a C library to realize the core algorithm. The new class is part of a project denoted as ayfbinrender which is not part of the AudYoFlo repository. This project shall be extended by a cross talk canceller component which is our new component. Even though the project ayfbinrender is not available here, the structure of the project is identical to all other sub-projects. Hence, all relations described in the following also apply to every other sub project.
At first, we will create and involve a signal processing audio node. This implies multiple steps to be described in this chapter.
There is a base class which acts as a template for in-place components: class CjvxBareNode1io. We start the development of a new component by creating a project which exposes a component. We can do so by copying the example code from
sources/jvxComponents/jvxSignalProcessingNodes/templates/jvxSpNSimpleZeroCopy
to the desired location of our new component:
Then, we change the name of the folder and the project to yield the new project denoted as ayfAuNCTC in this example:
Then, we need to involve the new project to become part of our cmake build tree. In the example, the project ayfbinrender of which the new component shall become part of, has a project specific CMakeLists.txt file which we need to edit at the following location:
Finally, our new project must get a unique name. We therefore open the local CMakeLists.txt file to edit the project name:
We can still keep the original names of the involved source files for now and then run the main cmake configuration step:
If this step was succesful, the compile-script run will be terminated normally:
Now, the new project is part of the build solution in Visual Studio:
In the next step, we need to adapt the filenames to match our new project. We can do this in Visual Studio, e.g.:
Also, we need to adapt the CMakeLists.txt file as follows:
Finally, we need to adapt the file componentEntry.cpp to make sure the modified filenames, the desired component type as well as the descriptor tags match our need,
and the include statement in the file CayfAuNCTC.cpp to match the new filenames,
Next, the name of the main component class shall be modified. This can be done by editing the header file, the implementation and the file componentEntry.cpp:
In the last step we may modify the name of the property code generation in file exports_node.pcg:
and need to adapt that name also in the component class definition:
and in the class implementation by search and replace,
We re-compile the component in order to check functionality:
The new class implements only the most commonly used member functions:
class CAyfAuNCTC: public CjvxBareNode1io, public genAuNCTC_node
{
private:
public:
JVX_CALLINGCONVENTION CAyfAuNCTC(JVX_CONSTRUCTOR_ARGUMENTS_MACRO_DECLARE);
~CAyfAuNCTC();
virtual jvxErrorType JVX_CALLINGCONVENTION activate()override;
virtual jvxErrorType JVX_CALLINGCONVENTION deactivate()override;
// ===============================================================
virtual jvxErrorType JVX_CALLINGCONVENTION process_buffers_icon(jvxSize mt_mask, jvxSize idx_stage)override;
// ===============================================================
virtual jvxErrorType JVX_CALLINGCONVENTION put_configuration(jvxCallManagerConfiguration* callMan,
IjvxConfigProcessor* processor,
jvxHandle* sectionToContainAllSubsectionsForMe,
const char* filename,
jvxInt32 lineno)override;
virtual jvxErrorType JVX_CALLINGCONVENTION get_configuration(jvxCallManagerConfiguration* callMan,
IjvxConfigProcessor* processor,
jvxHandle* sectionWhereToAddAllSubsections)override;
// ===============================================================
JVX_PROPERTIES_FORWARD_C_CALLBACK_DECLARE(set_bypass_mode);
};
This includes the activate/deactivate functions called on the beginning of the lifecycle of the component and the function
process_buffers_icon which is the main processing bufferswitch callback. The functions put_configuration/get_configuration are optional to store and read entries in the configuration file. The function set_bypass_mode is a callback in the property code that is called whenever the property bypass is set.
The new component is now ready to be used in an application to test parameter forwarding and signal processing.
The new component is involved in the main host application by attaching the static library to the main application and exposing the entry functions.
At first, we add the include path for our component to the search path of the host application:
Then, we add the library:
Note that the component may be required in different versions of the host application, e.g. the web host or the flutter import library dll. In the example, we do add it only the QT host application.
The components required in the application are deployed to the application in the call to function jvx_access_link_objects which is called in the application initialization code. Therefore, we add header file include as well as initialization code to the file ayfbinrender-components.cpp which is part of the QT application host application,
Here, we add the include file and an entry in function jvx_access_link_objects on the case of component type JVX_COMPONENT_AUDIO_NODE:
and
In this case, the new component is the component with id 7 for the type JVX_COMPONENT_AUDIO_NODE.
Now, we can already run the application and see how the new component is loaded by revieweing the applications output:
Another way to review it is to list all available audio node components in the UI:
The new component is loaded but it is not yet active.
The activation of the new component at boot time can be achieved by adding an entry to the table componentsOnLoad_algorithms which is located in the file ayfbinrender-hostconfig.cpp in our example file:
This field is associated in function jvx_configure_factoryhost_features which is also part of the file ayfbinrender-hostconfig.cpp:
Note that the component can only be activated if the number of slots is setup large enough. This happens further down in function jvx_configure_factoryhost_features:
In this case it means that the number of audio nodes is unlimited.
The new component can be reviewed in the host in slot #1 as demonstrated in the following:
In the current setup, the new component is not part of the signal processing chain:
The green connectors indicate that no connections are established. The connections are defined as a connection rule which is setup in file ayfbinrender-connectrules.cpp in function jvx_default_connection_rules_add:
The new component shall run between the binaural audio mixer and the output device connector:
The component is addressed by the slot id which is "1" for the new component. The bridge defined by
res = theDataConnectionDefRuleHdl->add_bridge_specification(
jvxComponentIdentification(JVX_COMPONENT_AUDIO_NODE, 0, 0),
"*", "default",
jvxComponentIdentification(JVX_COMPONENT_AUDIO_NODE, 1, 0),
"*", "default", "ABridge_mixer_to_ctc_node", false, false);
defines the linkage between the binaural mixer and the CTC component.
When starting the host this time, the module shows up as connected:
The component also shows up in the generic property viewer,
At this position, the connection parameters are valid since the new component integrates into the overall data signal processing chain.
The new component is now engaged into the signal processing chain. The processing parameters, however, are not constraint when using the default configuration. The component accepts all kinds of data at any samplerate in buffers of any size for any numbr of channels. The only constraint is that input and output data is identical - yielding in-place processing.
The default behavior can be modified as follows:
The involved processing is in-place - that is, the input and the output buffers are identical. This requires that the data struct to link the objects is forwarded in the prepare step. We can prevent this by setting
_common_set_ldslave.zeroCopyBuffering_cfg = false;
in the constructor. In that case, however, we need to obtain input and output buffers in the processing callback and move data forward from the input to the output to prevent that the output is silence.
The new component principally accepts all processing formats on the input side. Due to its in-place procesing, the output side must always be identical to the input side with respect to the processing parameters.
If we desire to only accept one specific configuration or to provide a range of processing parameter configurations, we can do this by using the member variable neg_input.
If, e.g., we want to support only 2 channels, we can specify
neg_input._update_parameters_fixed(2);
in the constructor. Within the function _update_parameters_fixed, all processing parameters can be adapted given the value at the appropriate position in the call. The default values - in most cases JVX_SIZE_SELECTED - causes the function not to update the constraining parameter.
Note that to transform a constrained parameter into an unconstrained parameter requires to call the function _clear_parameters_fixed to clear the appropriate positions.
The zerocopy components typically can not modify the output processing parameters if the input parameters are given since the zerocopy requires that input and output are equal. In that case, the requested output parameter can be forwarded to the components successor in the backward negotiation. This is at the same time the default behavior. If the successor agrees with the proposed modification of the paramaters as passed from the zerocopy component, it changes its output parameter which is a new input parameter for the zerocopy component such that successor as well as zerocopy component can change the processing parameter if it is allowed.
If a zerocopy component shall not forward the request to change processing parameters, it can be setup accordingly in the constructor:
forward_complain = false;
The proper design of a class to allow automatic parameter setup sometimes can be tricky. Principally, the base classes provide member variables to paramterize the negotiation constraints and all the rest is setup without any additional code being required. However, sometimes, it is needed to debug the test function run to find possible problems.
It is recommended to add the member function for the test function in the new class with
jvxErrorType test_connect_icon(JVX_CONNECTION_FEEDBACK_TYPE(fdb)) override;
to become part of the class declaration and
jvxErrorType
CAyfAuNCTC::test_connect_icon(JVX_CONNECTION_FEEDBACK_TYPE(fdb))
{
return CjvxBareNode1io::test_connect_icon(JVX_CONNECTION_FEEDBACK_CALL(fdb));
}
to become part of the class implementation. This function only calls the base class function but it may be used in the debugger to add a break point and follow the called function with a debugger stepping.