SFML Backend

[jasonbeach]

I'm sure I'm like many who are frustrated with the extremely limited options there are for plotting in C++. I really like this library (see here for a particle filter demo I made), but like others who want real-time performance are left wanting with the gnu-plot backend (it absolutely blows my mind the only API to that library is through pipes). I am not super familiar with OpenGL, but was looking at SFML since they recently released version 3.0 of their library. I had their demo built and running in less that one minute after cloning the repo and had a simple ball bouncing around the screen in 15 minutes without looking at the documentation. It was really pretty intuitive.

I'm curious what interest there would be in pulling in an SFML backend if I started working on one.

[alandefreitas]

Yes. I'm also frustrated by that. :)

Gnuplot was a terrible choice, and we learned a lot from other mistakes in this library. However, I don't have the time to implement a new version. If anyone is willing to take over v2 of this library, I'm happy to help and guide them.

• Gnuplot provides no control over details, and the high-level features break between versions. My intuition is also that the backend in Matplot v2 should be SFML+Cairo. Or SDL+Cairo. The backend interface should be more like Python's matplotlib interface: just a few functions to plot lines, etc. The library would handle all other transformations because they're always the same. Gnuplot makes that impossible. But once we get this right, people could write their own backends more easily and the choice of SFML+Cairo or SDL+Cairo is not even that relevant anymore. A basic backend could just write the plot to the console. What's important is getting rid of Gnuplot. Once we do that and the interface is simplified, people can easily write whatever backend they want (OpenGL, QT, Webassembly, ImGui, the console, or whatever) by defining a few functions.

• Another problem with backends now is that people want different kinds of representations for points and colors. This is not really a concern in MATLAB or Python's matplotlib but it makes a huge difference in C++ applications. This probably implies we would need templated canvases/backends and bring over some concepts from Boost.GIL to matplot. This way, the backend member functions can have any type that satisfies the point/line/color/... concept instead of relying on a specific representation. Of course, these templated backends could be (in later iterations of the library) type erased in the implementation to reduce compile times.

• A third problem is the dual interface. Python's matplotlib provides an interface based on classes and an interface based on free functions. The rationale for the second interface is to make it easier for people to migrate from MATLAB. We inherited this dual interface, but nowadays, that makes no sense at all. Very few people are still migrating from MATLAB, and the free function API is even less suitable for C++ applications. This problem is easy to solve as we have to remove the free functions.

Still, at this point, most member functions in the figure and axes classes also need to be removed. A figure has at least one member function for each plot type. And the functions are duplicated on the axes and figure classes. Calling the function on the axes puts more content on the chart, and calling the function on the figure redirects the call to the default chart. This is repetitive and prone to errors. What's worse, that pattern from Python, when translated to C++ means we have to have all headers for all plot types included in figure header file. There should be no member functions attached to plot types. A figure is simply a sequential container of axes and axes are just sequential containers of plots. This way, users can also get used to the idea of writing their own plot types, by just defining how the inputs should be represented as fundamental data in the axes. If some plot type becomes very popular, we can port it to the main library. A minor point here: these names also came from matplotlib who took them from MATLAB, but the names are really unintuitive. It makes sense to rename them to something like figure -> figure, axes -> subplot, plot -> ?, backend -> canvas.

A fourth point we need to improve is the documentation. The documentation is only composed of lots of examples: code and figures. The code examples are used as tests, but this is not great. We should have the manually written exposition describing the concepts in an intelligible order and the reference for the headers, which can be generated with mrdocs.

If anyone is willing to take over v2 of this library, I'm happy to help. I'd like to see this situation improve, even if it's by some new library or by someone else.

[jasonbeach]

I've been working through the white paper and I must say overall it's very impressive what you've been able to do and the sheer number of different plotting use-cases that are supported.

Unfortunately I don't have time either to take over (as much as part of me would like to say I could) but I think it would be useful to start documenting these ideas (which I think I agree with all of them) in something like a "v2_planning.md". That is something I can help with and hopefully that would provide an idea of the level of effort required for a v2 re-write. My gut feeling (which is probably very naive) is that rewriting the core of a v2 with a couple of basic plot types, while non-trivial wouldn't be that difficult. I assume that 80% of the work to get back to parity with the current version would be in converting the all of the different plot types once the core was re-written.

[jasonbeach]

I'd really like to see this improve as well. I've been poking through the code and for the figure class, I don't see the duplicated functions you mention. I do see them in the axes_type.h header and freestanding/plot.h. It appears these functions form the first abstraction layer that is built on top of the classes that derived from axes_object.

I think a v2 of the library would have sort of the following structure:

The primary class is still a figure similar to how it is now. One of the objects in that class would be a canvas (to borrow from your terminology). The canvas class would be a type-erased container that would hold what is currently called a backend.

A figure is simply a sequential container of axes and axes are just sequential containers of plots.

I like that although I might add one addition level of "plot window" which is roughly equivalent to a subplot. So a figure is a sequential container of 1..n plot windows, plot windows are a sequential container of 1..2 axes and axes are sequential containers of 1..n "plots". A "plot" again is a type-erased object container. So all the specific plot functions from the axes_type.h header would be removed and converted to stand-alone plot object classes that really wouldn't have to know anything about a figure, axes, etc. and a figure and axes objects would only have to know about the type-erased plot container object. So the call to create a plot would be something like:

fig.axes.back().emplace_back(scatter(x,y)); // we can add syntactic sugar on this to make it a little nicer

One thing I'm really curious about and is what I think you hinted at in your second point is moving from a struct-of-arrays concept (i.e. plot(vector<double> x, vector<double> y)) to an array-of-structs concept (i.e. plot(vector<point2d>, Projection proj) where vector could actually be any "range" whose element has something like a point2d. The projection converts an element in that container to something the library understands like a point2d. (I'm unfortunately not super familiar with C++ ranges yet, but I think they could be helpful here.) So something like this:

struct particle{
 double position_x;
 double position_y
 double vel_x;
 double vel_y;
};
using particles = std::vector<particle>
...
particles current_dist;
...
f.gca().emplace_back(scatter(current_dist, [](const auto&p){return point2d{p.position_x, p.position_y};})):

[alandefreitas]

I don't see the duplicated functions you mention

Here they are: https://github.com/alandefreitas/matplotplusplus/tree/master/source/matplot/freestanding

The primary class is still a figure similar to how it is now. One of the objects in that class would be a canvas (to borrow from your terminology). The canvas class would be a type-erased container that would hold what is currently called a backend.

Yes.

One thing I was thinking about is that type-erasing the canvas would be important and kind of complex at the same time because the canvas would support different structures to represent points. That's one of the main limitations stopping other backends from existing now. But the abstract type used for type-erasing could just have variants of the same function to accepts different data structures for points.

A figure is simply a sequential container of axes and axes are just sequential containers of plots.

Exactly.

I like that although I might add one addition level of "plot window" which is roughly equivalent to a subplot.

We usually refer to axes as a subplot. matplotlib inherited this convention from here: https://www.mathworks.com/help/matlab/ref/subplot.html The user refers to it a subplots because that's the public API but that's stored internally as this Axes class.

That's why I wanted to rename axes to subplot. The name is awful (plural for one thing and singular for something completely different) and subplot is the name the user interacts with.

I'm not sure this equivalence of axes and subplot is what you meant by "plot window" but the rest of your description makes sense with this assumption.

fig.axes.back().emplace_back(scatter(x,y));

Yes. That's exactly what I was thinking about. This way, you don't have to have included headers for plot types you don't need.

we can add syntactic sugar on this to make it a little nicer

Yes, we would have a helper function like:

fig.push_back(scatter(x,y));

that would be internally equivalent to:

fig.subplots_[default_subplot_idx_].push_back(scatter(x,y));

Minor detail: notice I used push_back because emplace_back should be reserved for in-place construction:

fig.emplace_back<scatter>(x,y);

So something like this:

struct particle{
 double position_x;
 double position_y
 double vel_x;
 double vel_y;
};

Exactly. To be useful in C++, we need to support other types of points. For instance, OpenGL applications usually require the points to be float. There's not much point in using double for graphical applications. Sometimes, we might even need to attach the color and alpha to the points.

[jasonbeach]

One thing I was thinking about is that type-erasing the canvas would be important and kind of complex at the same time because the canvas would support different structures to represent points. That's one of the main limitations stopping other backends from existing now. But the abstract type used for type-erasing could just have variants of the same function to accepts different data structures for points.

I think that would really be a challenge. On the one hand if you use templates, you would have to template the type-erased container itself which would mean you would have to template the figure object with I don't think is desirable. If you have overloads of the same function, the type-erased container would have to be aware of what the contained backend type is so it would know which overload to call which again I think is not desirable.

I've been thinking about what the main interface between the the library and the backend should be and to that end, I've really been exploring how SFML does things and it really comes down to everything is a vertex array. My best idea is that essentially the backend should output a something like a vertex array in the form of triangles that draws lines, axes, etc. and a vector of strings for labels, legends, annotations, etc. Maybe also need a vector circles. The way SFML formats them is very close to what OpenGL expects. So while a plot class may take a range of an arbitrary type, by the time the backend receives that data it's transformed into a common format that is highest performing for the default backend.

Here's a straw implementation: https://godbolt.org/z/1hcboT4Mn hopefully to illustrate what I mean.

[alandefreitas]

If you have overloads of the same function, the type-erased container would have to be aware of what the contained backend type is

We can avoid that by providing overloads for all combinations of point types that we can type erase. Still a hassle, but it works.

it really comes down to everything is a vertex array

Yes. Something equivalent to a vertex would be the main point type. For instance, an OpenGL application can also use it because it contains all the information contiguously. The plots can do the transformations for other point types to become vertices and extensions to the backend would mostly related to plotting 3d points directly, but we can skip this for now (or potentially forever: anyone who needs it can propose their adaptation to the canvas type).

Here's a straw implementation: https://godbolt.org/z/1hcboT4Mn hopefully to illustrate what I mean.

Yes. That's precisely what we need! That implementation looks great.