Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Getting parameterised query text without placeholders #902

Open
GoldAndPurple opened this issue Nov 12, 2024 · 3 comments
Open

Getting parameterised query text without placeholders #902

GoldAndPurple opened this issue Nov 12, 2024 · 3 comments

Comments

@GoldAndPurple
Copy link

Is there a way to know the proper text of the executed query, with placeholder text replaced with bound parameters? Relevant for exception handling and logging purposes, where you might not know the parameters used, but need information the most.
Right now there is a weird pattern for more complicated queries. It's better to format query-strings with escaped parameters in the application itself, "on your own", and only then execute the resulting statement with no parameters.

@jtv
Copy link
Owner

jtv commented Nov 12, 2024

Alas, I don't know if any way to get that version of the query string.

I guess it would be possible to reconstruct it, but that can go wrong in exactly the kind of situation where you need it most.

@GoldAndPurple
Copy link
Author

I suppose the main problem is that postgres itself doesn't always log parameters, so we have to rely on either the library, or the application to remember them.
But what are the pitfalls of reconstructing the query post-execution? The only thing that wouldn't be exactly the same is raw binary data (and its user-defined equivalents), right? Everything else is passed as text, so what's the problem with replacing placeholders with pqxx-escaped parameters?

@jtv
Copy link
Owner

jtv commented Nov 16, 2024

Text is no longer simple. :-( Yes, it can be done, but it's starting to sound like work.

Imagine you're working in an encoding like SJIS. Your parameters look like $1, $2, etc. But your query can also contain a multi-byte character where the 2nd or 3rd byte happens to match the ASCII byte value for $. Maybe it' s a Japanese multi-byte character that libpqxx doesn't support properly yet, or there's an obscure bug in text parsing. Now you may have a phantom parameter, and the parameter value may splice into the middle of a singe character.

Or consider the complexity of support for complex data types — support that may live in your application rather than in libpqxx itself. I'm not sure the quoting and escaping rules will always be appropriate. Will libpqxx need to figure out the parameters' SQL types? How do we keep those choices consistent with libpq's? And how do we represent binary?

In theory it's all doable. But it does add complexity and scope for error, and error handling is the worst place for that.

What might work better is to show the query as-is, and then also make the parameter values available. However I do kind of hate to leave that responsibility on your shoulders because it exposes your application to encoding issues. For instance, you might be running on Windows in UTF-16, but with UTF-8 as your postgres client encding — that probably means you can't just print these values. And you may want to truncate long parameter values for display, but you'll need to take care not to truncate in the middle of a character. (And then in some languages, there are places where truncating between characters doesn't make sense in some cases, but that's a minor thing so let's not even get into that.)

In short: I hate parsing text in dynamically determined encodings. :-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants