6.3.0: Encodings, new stream classes, deprecations, and more.

This is a big release, with lots of changes. It paves the way for some even more drastic changes to come in 7.0.

First let me tell you what's new in 6.3, and then I'll go into my plans for 7.0.

What's new in 6.3

Thanks to some major contributions from Joseph "JadeMatrix" Durel, we now have:

Type-safe replacements for tablereader and tablewriter: stream_from and stream_to. They support tuples (or tuple-like types) to represent database rows on the client side, so you could stream a std::tuple or std::vector straight into a database row, or vice versa.

Broader support for variable-width multibyte client encodings. UTF-8 has always worked, but other encodings such as SJIS might confuse the code by embedding bytes that looked like ASCII quotes or backslashes and such inside a multibyte character. From now on, the library should work properly with all client encodings that PostgreSQL supports.

In other news:

• libpqxx lets you assign names to some types of objects, such as cursors or transactions. These names are now more tolerant of "special" characters such as embedded quotes or non-ASCII letters.
• The CMake build has been overhauled. There may be teething problems, so please report anything you run into.
• Copying result objects for long SQL queries is now faster.

For the really big changes though, we're going to have to break stuff. Many items have also been deprecated. See the section on my 7.0 plans below.

Also, many items which had been documented as being deprecated are now actually marked deprecated, using the C++ deprecated attribute if your compiler supports it. This means that you may start seeing warnings where your code has been using deprecated features. Those features really will start disappearing in 7.0.

Preparing for 7.0

We'll see some profound change in libpqxx 7.0. Some of details still have to be worked out, but in 6.3 you'll start seeing deprecation warnings for features that will no longer work as before.

What can you expect from 7.0? What will you need to be ready for? Read on.

C++ upgrade

C++11 has been great, but now I want more. There are some wonderful things we can do if we bump the minimum requirement to C++17. Is your compiler ready for them?

The C++17 features I'd particularly like to use are:

• std::optional as, eventually, the one standard way to accommodate null values.
• Nested namespaces. It'll make pqxx::internal easier for me to manage.
• std::string_view support in places where we already accept std::string or C-style strings.
• std::to_chars and std::from_chars could replace a lot of libpqxx code.

If your compiler is not ready for any of these new features, please file a bug on the ​Github page so I can avoid breaking your build.

Connection classes

The connection class hierarchy is going to change drastically. If you have defined your own custom connection classes, these will no longer work, and you will start to see deprecation warnings in libpqxx 6.3. The new design is not final, but it will be a lot simpler than what we have now. Some esoteric features are going to disappear, and in return you'll get a move constructor and a much easier class hierarchy to work with.

The plan is to fold essentially the entire class hierarchy into one single class. It will support the current built-in connection types, but it won't be extensible. We have always had an API for defining your own connection policies, and the library uses it to implement different types of connections, but I've never heard of anyone else using this API to define their own connection types. The existing built-in connection types will just be thin wrappers for the single connection class, and eventually the single connection class should replace them all.

Replacing lazyconnection and asyncconnection

If you're using lazyconnection or asyncconnection, the API will change. You'll have to do a bit of extra work.

In older libpqxx versions, you got a connection object which looks just like a regular connection but only completes its actual connection to the database once you actually start using it. Your code will have to go through an explicit check-or-wait step before using the connection. It may end up looking similar to std::future, or it may just be a member function. I'm not sure yet.

No more connection reactivation

It's just a fact of life: network connections to the database can break. It could happen because of a network problem, or the database may have been restarted. The existing connection classes hide this from you. They notice that the connection is broken and quietly try to re-establish it for you. This feature is going to be removed.

That may sound like a problem. But your application already knows how to deal with a broken connection, right? It may simply happen slightly more often.

I believe doing it this way will make the application design issues clearer, eliminate a duplication of effort between libpqxx and your application, and clear out some dark corners where bugs might hide. It will also remove an enormous source of complexity inside the connection classes.

As a bonus, several operations on connection objects can finally get the const qualifier. That's because they will no longer try to re-establish a broken connection, or finalise an incomplete lazy or asynchronous connection. It will be a lot more obvious which operations make changes to your connection and which ones will not. Some references in your code that don't look like they modify the connection may even become const.

Birthday release: libpqxx turns 18!

Today marks 18 years since my initial commit in libpqxx's original CVS repo. This library is now officially a grownup.

Changes:

• Removed deprecated pqxx-config.
• Build fix on Visual C++ when not defining NOMINMAX.
• Reduce setup code for string conversions, hopefully improving speed.
• Allow nul bytes in tablereader.
• Support defining string conversions for enum types.
• Fixed const/pure attributes warning in gcc 8.
• Updated build documentation to mention CMake option.

6.2.4: Build fixes

The build machinery now tries to cope with non-ASCII characters in paths. It's not perfect, because filesystem paths are raw bytes — it may still break if those raw bytes fail to decode to Unicode characters.

If your application's build breaks with an error about std::experimental not existing, try defining the new macro PQXX_HIDE_EXP_OPTIONAL. It will suppress libpqxx support for std::experimental::optional. This can be needed if your copy of libpqxx was built in a compiler environment which had that template, and your application is being built in an environment which does not.

6.2.3: build fixes

Thanks to all the people who contributed fixes and improvements!

Small fixes

Now that we're on github, libpqxx is getting a lot more scrutiny. Thanks to all those who submitted bugs and fixes!

This update mostly addresses compile issues, but also one actual bug: in some circumstances an error string would be left empty.

Array parsing, and bug fixes.

Adds a first-generation parser for SQL arrays. We'll want to improve on this later. See the pqxx/array header.

This release also fixes some bugs which would break builds under specific circumstances — in some cases builds of the library, in other cases builds of client software. So, if you had trouble compiling with 6.0, try 6.1.

Dependencies between library headers have changed. It's possible that some code may need an extra #include here or there, if your code implicitly relied on some libpqxx headers including other libpqxx headers.

Smaller, modernised, all-C++11 libpqxx

This is a major overhaul. Changes are everywhere:

• C++11 is now required. Your compiler must have shared_ptr, noexcept, etc.
• Removed configure.ac.in; we now use configure.ac directly.
• Removed pqxx::items. Use the new C++11 initialiser syntax.
• Removed maketemporary. We weren't using it.
• Can now be built outside the source tree.
• New, simpler, lambda-friendly transactor framework.
• New, simpler, prepared statements and parameterised statements.
• Result rows can be passed around independently.
• New exec0(): perform query, expect zero rows of data.
• New exec1(): perform query, expect (and return) a single row of data.
• New exec_n(): perform query, expect exactly n rows of data.
• No longer defines Visual Studio's NOMINMAX in headers.
• Much faster configure script.
• Most configuration items are gone.
• Retired all existing capability flags.
• Uses WSAPoll() on Windows.
• Documentation on readthedocs.org, thanks Tim Sheerman-Chase.

Most old code should still compile against the new library, but you may find incompatibilities in uncommon usage.

Everything is more modern now. The code lives on GitHub (with automated test runs thanks to CircleCI), and documentation on ReadTheDocs. Release procedures were rigid and cumbersome and kept me from doing any releases at all; now they're light and fast again. I couldn't get access to the mailing lists, so I dropped them. A lot of workarounds for old compilers and postgres versions went out the window. As a result, the configure script runs in about as many seconds as it used to take minutes!

Adding exec1() functions (to execute a query that returns exactly 1 row) may seem like trivia. But it's actually a big deal. Everything had to move and change in very painful ways in order to make that possible! A row object now keeps its result object alive, so you no longer need a result somewhere in order to have a row variable. The same thing happened between fields and rows. The side effect is that the library is now cleaner, and easier to reason about. To me, looking at the inside, it's like the whole library is new again.

For you, I hope it will mainly be simpler. There's less crud. Prepared statements and parameterised statements no longer look like a clever hack. Builds are faster. It's harder to make subtle mistakes.

5.0.1

Expose SQLSTATE error codes in sql_error exceptions.