Skip to content

Commit

Permalink
Implement generic and strongly typed wrappers for MSparseArray togeth…
Browse files Browse the repository at this point in the history
…er with documentation and unit tests.
  • Loading branch information
rafal-c committed Feb 24, 2021
1 parent 267c6be commit b5ddc70
Show file tree
Hide file tree
Showing 29 changed files with 1,159 additions and 77 deletions.
5 changes: 3 additions & 2 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ if(APPLE AND NOT DEFINED ENV{MACOSX_DEPLOYMENT_TARGET})
endif()

project(LLU
VERSION 3.0.1
VERSION 3.1.0
DESCRIPTION "Modern C++ wrapper over LibraryLink and WSTP"
LANGUAGES CXX)

Expand Down Expand Up @@ -69,7 +69,8 @@ if(NOT TARGET LLU)
${LLU_SOURCE_DIR}/FileUtilities.cpp
${LLU_SOURCE_DIR}/TypedMArgument.cpp
${LLU_SOURCE_DIR}/Containers/DataStore.cpp
${LLU_SOURCE_DIR}/Containers/NumericArray.cpp)
${LLU_SOURCE_DIR}/Containers/NumericArray.cpp
${LLU_SOURCE_DIR}/Containers/SparseArray.cpp)

#add the main library
add_library(LLU ${LLU_SOURCE_FILES})
Expand Down
1 change: 0 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -172,7 +172,6 @@ Limitations with respect to LibraryLink
There are a few LibraryLink features currently not covered by LLU, most notably:
- Sparse Arrays
- Tensor subsetting: `MTensor_getTensor`
- Callbacks
- Wolfram IO Library (asynchronous tasks)
Expand Down
2 changes: 1 addition & 1 deletion docs/conf.py.in
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ language = None

# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
exclude_patterns = ['_build', 'DoxygenMain.md']
exclude_patterns = ['_build', 'DoxygenMain.md', 'tutorial']

# The reST default role (used for this markup: `text`) to use for all
# documents.
Expand Down
1 change: 0 additions & 1 deletion docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -205,7 +205,6 @@ Limitations with respect to LibraryLink

There are a few LibraryLink features currently not covered by LLU, most notably:

- Sparse Arrays
- Tensor subsetting: `MTensor_getTensor`
- Callbacks
- Wolfram IO Library (asynchronous tasks)
Expand Down
83 changes: 81 additions & 2 deletions docs/modules/containers.rst
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ generic, datatype-agnostic wrappers and full-fledged wrappers templated with the
+----------------------------------------------+----------------------------------------------+----------------------------------------+
| :ref:`DataStore <datastore-label>` | :ref:`GenericDataList <genericdl-label>` | :ref:`DataList\<T> <datalist-label>` |
+----------------------------------------------+----------------------------------------------+----------------------------------------+
| :ref:`MSparseArray <msparsearray-label>` | | |
| :ref:`MSparseArray <msparsearray-label>` | :ref:`GenericSparseArray <genericsa-label>` | :ref:`SparseArray\<T> <spararr-label>` |
+----------------------------------------------+----------------------------------------------+----------------------------------------+

Memory management
Expand Down Expand Up @@ -275,6 +275,23 @@ perform operations on the underlying data.
.. doxygenclass:: LLU::MContainer< MArgumentType::Tensor >
:members:


.. _genericsa-label:

:cpp:type:`LLU::GenericSparseArray`
------------------------------------

GenericSparseArray is a light-weight wrapper over :ref:`msparsearray-label`. It offers the same API that LibraryLink has for MSparseArray, except for access
to the underlying array data because GenericSparseArray is not aware of the array data type. Typically one would use GenericSparseArray to take an MSparseArray
of an unknown type from LibraryLink, investigate its properties and data type, then upgrade the GenericSparseArray to the strongly-typed one in order to
perform operations on the underlying data.

.. doxygentypedef:: LLU::GenericSparseArray

.. doxygenclass:: LLU::MContainer< MArgumentType::SparseArray >
:members:


Typed Wrappers
============================

Expand Down Expand Up @@ -302,7 +319,7 @@ DataList is a strongly-typed wrapper derived from GenericDataList in which all n
+-------------------------+--------------------------+------------------------+
| NodeType::Tensor | LLU::GenericTensor | MTensor |
+-------------------------+--------------------------+------------------------+
| NodeType::SparseArray | MSparseArray | MSparseArray |
| NodeType::SparseArray | LLU::GenericSparseArray | MSparseArray |
+-------------------------+--------------------------+------------------------+
| NodeType::NumericArray | LLU::GenericNumericArray | MNumericArray |
+-------------------------+--------------------------+------------------------+
Expand Down Expand Up @@ -557,6 +574,68 @@ On the Wolfram Language side, we can load and use this function as follows:
:members:


.. _spararr-label:

:cpp:class:`LLU::SparseArray\<T> <template\<typename T> LLU::SparseArray>`
-------------------------------------------------------------------------------

:cpp:expr:`LLU::SparseArray` is a wrapper over an MSparseArray which holds elements of type ``T``. MSparseArray supports only 3 types of data,
meaning that :cpp:class:`template\<typename T> LLU::SparseArray` class template can be instantiated with only 3 types ``T``:

- ``mint``
- ``double``
- ``std::complex<double>``


Here is an example of the SparseArray class in action:

.. code-block:: cpp
:linenos:
template<typename T>
void sparseModifyValues(LLU::SparseArray<T>& sa, LLU::TensorTypedView<T> newValues) {
// extract a Tensor with explicit values of the SparseArray
// this does not make a copy of the values so modifying the Tensor will modify the values in the SparseArray
auto values = sa.explicitValues();
if (values.size() < newValues.size()) {
throw std::runtime_error {"Too many values provided."};
}
// copy new values in place of the old ones
std::copy(std::cbegin(newValues), std::cend(newValues), std::begin(values));
// Recompute explicit positions (necessary since one of the new values might be equal to the implicit value of the SparseArray)
sa.resparsify();
}
LLU_LIBRARY_FUNCTION(ModifyValues) {
auto sp = mngr.getGenericSparseArray<LLU::Passing::Shared>(0);
auto values = mngr.getGenericTensor<LLU::Passing::Constant>(1);
// Operate on the GenericSparseArray as if its type was known
LLU::asTypedSparseArray(sp, [&values](auto&& sparseArray) {
using T = typename std::remove_reference_t<decltype(sparseArray)>::value_type;
sparseModifyValues(sparseArray, LLU::TensorTypedView<T> {values});
});
}
On the Wolfram Language side, we can load and use this function as follows:

.. code-block:: wolfram-language
(* Our function takes a shared SparseArray to modify it in-place. The SparseArray can be of any type. *)
`LLU`PacletFunctionSet[$ModifyValues, {{LibraryDataType[SparseArray, _, _], "Shared"}, {_, _, "Constant"}}, "Void"];
sparse = SparseArray[{{3.5, 0., 0., 0.}, {.5, -7., 0., 0.}, {4., 0., 3., 0.}, {0., 0., 0., 1.}}];
$ModifyValues[sparse, {3.5, .5, -7.}];
Normal[sparse]
(* Out[] = {{3.5, 0., 0., 0.}, {.5, -7., 0., 0.}, {4., 0., 3., 0.}, {0., 0., 0., 1.}} *)
.. doxygenclass:: LLU::SparseArray
:members:

Iterators
========================

Expand Down
4 changes: 2 additions & 2 deletions docs/tutorial/IntegratingCppWithWL.md
Original file line number Diff line number Diff line change
Expand Up @@ -659,8 +659,8 @@ using Complex = std::complex<double>;
/// Tensor stands for a GenericTensor - type agnostic wrapper over MTensor
using Tensor = MContainer<MArgumentType::Tensor>;

/// SparseArray type corresponds to the "raw" MSparseArray as LLU does not have its own wrapper for this structure yet
using SparseArray = MSparseArray;
/// SparseArray stands for a GenericSparseArray - type agnostic wrapper over MSparseArray
using SparseArray = MContainer<MArgumentType::SparseArray>;

/// NumericArray stands for a GenericNumericArray - type agnostic wrapper over MNumericArray
using NumericArray = MContainer<MArgumentType::NumericArray>;
Expand Down
28 changes: 16 additions & 12 deletions include/LLU/Containers/Generic/Base.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -74,8 +74,10 @@ namespace LLU {
* @return reference to this object
*/
MContainerBase& operator=(MContainerBase&& mc) noexcept {
reset(mc.container, mc.owner);
mc.container = nullptr;
if (this != &mc) {
reset(mc.container, mc.owner);
mc.container = nullptr;
}
return *this;
}

Expand Down Expand Up @@ -198,17 +200,19 @@ namespace LLU {
* @param newOwnerMode - owner of the new container
*/
void reset(Container newCont, Ownership newOwnerMode = Ownership::Library) noexcept {
switch (owner) {
case Ownership::Shared:
disown();
break;
case Ownership::Library:
free();
break;
case Ownership::LibraryLink: break;
if (newCont != container) {
switch (owner) {
case Ownership::Shared:
disown();
break;
case Ownership::Library:
free();
break;
case Ownership::LibraryLink: break;
}
container = newCont;
}
owner = newOwnerMode;
container = newCont;
}

private:
Expand All @@ -231,7 +235,7 @@ namespace LLU {

/**
* @class MContainer
* @brief MContainer is an abstract class template for generic containers. Only specializations shall be used.
* @brief MContainer is an abstract class template for generic containers. Only specializations shall be used.
* @tparam Type - container type (see MArgumentType definition)
*/
template<MArgumentType Type, typename std::enable_if_t<Argument::ContainerTypeQ<Type>, int> = 0>
Expand Down
174 changes: 174 additions & 0 deletions include/LLU/Containers/Generic/SparseArray.hpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,174 @@
/**
* @file SparseArray.hpp
* @brief GenericSparseArray definition and implementation
*/
#ifndef LLU_CONTAINERS_GENERIC_SPARSEARRAY_HPP
#define LLU_CONTAINERS_GENERIC_SPARSEARRAY_HPP

#include "LLU/Containers/Generic/Base.hpp"
#include "LLU/Containers/Generic/Tensor.hpp"

namespace LLU {

template<>
class MContainer<MArgumentType::SparseArray>;

/// MContainer specialization for MSparseArray is called GenericSparseArray
using GenericSparseArray = MContainer<MArgumentType::SparseArray>;

/**
* @brief MContainer specialization for MSparseArray
*/
template<>
class MContainer<MArgumentType::SparseArray> : public MContainerBase<MArgumentType::SparseArray> {
public:
/// Inherit constructors from MContainerBase
using MContainerBase<MArgumentType::SparseArray>::MContainerBase;

/**
* @brief Default constructor, the MContainer does not manage any instance of MSparseArray.
*/
MContainer() = default;

/**
* @brief Create a new SparseArray from positions, values, dimensions and an implicit value
* @param positions - positions of all the explicit values in the array
* @param values - explicit values to be stored in the array
* @param dimensions - dimensions of the new SparseArray
* @param implicitValue - implicit value (the one that is not stored) of the new SparseArray
* @see <https://reference.wolfram.com/language/LibraryLink/ref/callback/MSparseArray_fromExplicitPositions.html>
*/
MContainer(const GenericTensor& positions, const GenericTensor& values, const GenericTensor& dimensions, const GenericTensor& implicitValue);

/**
* @brief Create a new SparseArray from data array and an implicit value
* @param data - a tensor whose contents will be copied and sparsified
* @param implicitValue - implicit value (the one that is not stored) of the new SparseArray
* @see <https://reference.wolfram.com/language/LibraryLink/ref/callback/MSparseArray_fromMTensor.html>
*/
MContainer(const GenericTensor& data, const GenericTensor& implicitValue);

/**
* @brief Create a copy of given SparseArray with different implicit value
* @param s - other SparseArray
* @param implicitValue - implicit value (the one that is not stored) of the new SparseArray
* @see <https://reference.wolfram.com/language/LibraryLink/ref/callback/MSparseArray_resetImplicitValue.html>
*/
MContainer(const GenericSparseArray& s, const GenericTensor& implicitValue);

/**
* @brief Clone this MContainer, performs a deep copy of the underlying MSparseArray.
* @note The cloned MContainer always belongs to the library (Ownership::Library) because LibraryLink has no idea of its existence.
* @return new MContainer, by value
*/
MContainer clone() const {
return MContainer {cloneContainer(), Ownership::Library};
}

/**
* @brief Get the implicit value of this sparse array.
* @return Rank 0 tensor of the same type as the value type of this sparse array.
* @see <https://reference.wolfram.com/language/LibraryLink/ref/callback/MSparseArray_getImplicitValue.html>
*/
GenericTensor getImplicitValueAsTensor() const;

/**
* @brief Change the implicit value of this array.
* @param implicitValue - new implicit value
* @note The underlying MSparseArray object may be replaced in the process.
* @see <https://reference.wolfram.com/language/LibraryLink/ref/callback/MSparseArray_resetImplicitValue.html>
*/
void setImplicitValueFromTensor(const GenericTensor& implicitValue);

/**
* @brief Get the rank (number of dimensions) of this sparse array.
* @return the rank of this array
* @see <https://reference.wolfram.com/language/LibraryLink/ref/callback/MSparseArray_getRank.html>
*/
mint getRank() const {
return LibraryData::SparseArrayAPI()->MSparseArray_getRank(this->getContainer());
}

/**
* @brief Get dimensions of this sparse array.
* @return a read-only raw array of container dimensions
* @see <https://reference.wolfram.com/language/LibraryLink/ref/callback/MSparseArray_getDimensions.html>
*/
mint const* getDimensions() const {
return LibraryData::SparseArrayAPI()->MSparseArray_getDimensions(this->getContainer());
}

/**
* @brief Get a tensor with the values corresponding to the explicitly stored positions in the sparse array.
* @return GenericTensor of rank 1 with length equal to the number of explicit positions in the array or an empty wrapper for "pattern sparse arrays"
* @see <https://reference.wolfram.com/language/LibraryLink/ref/callback/MSparseArray_getExplicitValues.html>
*/
GenericTensor getExplicitValues() const;

/**
* @brief Get a row pointer array for this sparse array.
* @details The values returned are the cumulative number of explicitly represented elements for each row, so the values will be non-decreasing.
* @return GenericTensor of rank 1 and integer type or an empty GenericTensor
* @see <https://reference.wolfram.com/language/LibraryLink/ref/callback/MSparseArray_getRowPointers.html>
*/
GenericTensor getRowPointers() const;

/**
* @brief Get the column indices for the explicitly stored positions in this sparse array.
* @details The first dimension of the resulting tensor is the number of explicit positions, and the second dimension is equal to getRank() - 1.
* @return GenericTensor of rank 2 or an empty GenericTensor
* @see <https://reference.wolfram.com/language/LibraryLink/ref/callback/MSparseArray_getColumnIndices.html>
*/
GenericTensor getColumnIndices() const;

/**
* @brief Get the explicitly specified positions in this sparse array.
* @details The first dimension of the resulting tensor is the number of explicit positions, and the second dimension is equal to getRank().
* @return GenericTensor of rank 2 or an empty GenericTensor
* @see <https://reference.wolfram.com/language/LibraryLink/ref/callback/MSparseArray_getExplicitPositions.html>
*/
[[nodiscard]] GenericTensor getExplicitPositions() const;

/**
* @brief Expand this sparse array to a regular tensor
* @return GenericTensor of the same data type as this array
* @see <https://reference.wolfram.com/language/LibraryLink/ref/callback/MSparseArray_toMTensor.html>
*/
[[nodiscard]] GenericTensor toGenericTensor() const;

/**
* @brief Use current implicit value to recalculate the sparse array after the data has been modified.
*/
void resparsify();

/**
* @brief Get the data type of this MSparseArray
* @return type of elements (MType_Integer, MType_Real or MType_Complex)
*/
mint type() const;

private:

/**
* @brief Make a deep copy of the raw container
* @see <http://reference.wolfram.com/language/LibraryLink/ref/callback/MSparseArray_clone.html>
**/
Container cloneImpl() const override;

/**
* @copydoc MContainer<MArgumentType::Image>::shareCount()
* @see <http://reference.wolfram.com/language/LibraryLink/ref/callback/MSparseArray_shareCount.html>
*/
mint shareCountImpl() const noexcept override {
return LibraryData::SparseArrayAPI()->MSparseArray_shareCount(this->getContainer());
}

///@copydoc MContainer<MArgumentType::DataStore>::pass
void passImpl(MArgument& res) const noexcept override {
MArgument_setMSparseArray(res, this->getContainer());
}
};

} // namespace LLU

#endif // LLU_CONTAINERS_GENERIC_SPARSEARRAY_HPP
Loading

0 comments on commit b5ddc70

Please sign in to comment.