Cache querying and bulk skeleton endoints #261
Conversation
| file_content = ( | ||
| response.content.decode() | ||
| if output_format == "swc" | ||
| else SkeletonClient.decompressBytesToString(response.content) |
There was a problem hiding this comment.
This should not be required.. if the repsonse content is advertized as having the right content-encoding responses should decompress it automatically.
There was a problem hiding this comment.
I experimented with using the after_request and accept-encoding methodology and couldn't get it to work properly. I can try to recreate the debugging code I tore down for the PR to see if I can get this going again.
There was a problem hiding this comment.
A few things are going on here. The code you highlighted above is for the SWC scenario. You'll notice that I'm handling both cases (native SWC, whether internally compressed or sent raw, and my manual compression method). The switch you highlighted does what you are suggesting in the native code, taking response.content.decode() directly, without further processing, i.e., regardless of whether the service compressed it and/or gave it a compression encoding, the client doesn't do anything explicit to handle the compression. It just gets the SWC data for free. But the same switch highlighted above also handles my experimental manual compression scenario, in which I handle the compression without Flask's help. So, the question is, should I be doing that or should I let Flask do the compression for me?
As I showed in some detailed experiments and resulting plots last week, it turns out that SWC doesn't benefit from compression of any sort (internal Flask compression or manual compression) because SWC data is small enough to fall within the overhead of the overarching communication inefficiencies. So, one outcome of my experiment was that in the case of an SWC request, I may as well dispense with any compression complexity (which is work I will defer to the orthogonal task of simplifying the skeleton services output format options; this PR is about the new cache querying and bulk skeleton APIs and I'm frustrated that my unrelated experiments vis-a-via compression possibilities are a distraction here; I guess properly I should have attempted to maintain these two lines of work in unrelated git branches or something to that effect, but I didn't compartmentalize the tasks that cleanly as I was picking away at the code. I'm sorry I have multiple tasks going on at once in the same codebase, I realize this isn't the cleanest approach).
Of course, I'm worried there may be larger neurons and skeletons out there that I haven't caught in my experimental net yet in which compression (of one sort or another) would be beneficial. If that turns out to be the case, should I just rely on Flask's cleaner handling of compression for me? Well, read on...
Now, in the case of JSON, the plots I shared last week made it clear that there are substantial benefits in compressing the JSON data before transmitting it. But should I let Flask handle it for me? Well, observe:
For a test root id, if I pass uncompressed JSON back, the data size and round-trip time are:
len(response.content) received by client: 2,587,008
Wall time: 3.3s
If I use Flask jsonify() (which is slow, it takes at least a second) along with manual compression in after_request() and then let Flask automatically decompress the data for the client, I get:
Precompressed data size in after_request: 1847682
Compressed data size in after_request: 284732
len(response.content) received by client: 1,847,682
Wall time: 3.4s
This experiment shows that if you ask Flask for the response.content size on the receiving end, you get the size after decompression. You can't see the size of the response package that was received, as it is already decompressed by the time you ask for that. The more perplexing question is why the response.data is 1.8M in this case and 2.6M in the first case, in which I can't really see the response at all since I just return the dictionary from the service directly to Flask, which then packages it up on its own somehow. For some reason, that produces a 2.6M package, whereas building a response around it starts out at 1.8M before any compression. Incidentally, if I print out len(json.dumps(cached_skeleton), i.e., the original dictionary, I get 1,077,899. It's plainly clear in the debugger that Flask's conversion of the dictionary to a serializable byte string introduces a bunch of prettyprint formatting, hence the bloated sizes shown above, either 2.6M or 1.8M, one way or the other.
If I use my current method of handling compression manually, without Flask's help, I get:
Precompressed data size in after_request: 261925
Compressed data size in after_request: 251381
len(response.content) received by client: 261,925
Wall time: 1.1s
Curiously, gzipping my manually gzipped data second time punches it down a tad bit more, but that isn't too relevant to this experiment.
My method achieves slightly tighter compression and is the only method of the three to make any temporal gains. Why would this be? I can think of two broad categories of explanation. The first is that I've botched it up somehow and I'm not using the internal compression method correctly, in which case I need to figure out what's wrong and fix it. The second possible explanation is that if I handle the compression myself, I'm avoiding all the infrastructure of some multi-layered, general purpose, all-consuming library behemoth that offers the ease-of-use of a do-it-all-behind-the-scenes library, but at the corresponding cost too. Maybe Flask's handling of compression is just so jammed up with complexity that the gains are only appreciable on, oh say, much larger data than a mere 2-3 MBs. Maybe it would show real rewards on 100MBs but the overhead of descending into Flask's subterranean library routines just doesn't pay off on small payloads. I don't know. All I know is the experimental results shown above, in which letting Flask handle the compression, doesn't seem to pay off (assuming I didn't simply screw it up somehow).
There was a problem hiding this comment.
I would like to merge this PR, which pertains to cache querying and bulk skeleton routines, and to defer the compression issue to future (ongoing for a while now) work that is not yet fully resolved. Is that an option, or is the current code considered too messy to deploy, even if the end-user use cases aren't directly affected? What is the right approach here?
There was a problem hiding this comment.
Some research into the prettyprint formatting suggested that an earlier version of Flask facilitated this and it was deprecated without a clear explanation of how to achieve the same result going forward. I'm sure it can be done, but I haven't found it yet.
There was a problem hiding this comment.
i understand what you are saying here, but i'm talking about the client side processing of a request response... if the response is compressed, and the content-encoding is set correctly, the requests library will uncompress it automatically. So .. if Flask is compressing the response it should set the content-encoding flag to gzip in the response and on the client side it should uncompress it accordingly.
There was a problem hiding this comment.
also, if you set the content-encoding like that, and then leave this code like this, the client will decompress it twice and that will fail.
There was a problem hiding this comment.
I don't understand. I realize the switch between the two formats was a little confusing, but I tried to clarify that it actually does just call response.content.decode() on the internally compressed result of compress inside after_request. As you point out, it was double-compressing as a result (I showed that in the comment above where I remarked that the compression JSON I produced achieved a little more compression on the second pass inside after_request. So, from Flask's perspective, it decompresses back to my explicitlly gzipped data...which I then decompress on my own. I realize that's probably abstruse and confusing. I don't think I appreciated that it was doing a double-compression previously.
There was a problem hiding this comment.
but here the client didn't set the Accept-Encoding to include gzip so flask should not have been compressing it in the after_request.. so i don't think you are getting double compression.
What should happen is that the somewhere in flask there should be one compression based on what is in the accept-encoding header, and then the content-encoding confirms whether it is compressed or not. If you use gzip or zlib then requests handles the decompression for you and you don't have to have this code in the client which will go out of style, and at some point there will be new compression algorithms and the server will only utilize them if the client says they can handle them, so it's a nice two way contract.
Now your compression that is happening outside the after_request is not checking the accept-encoding header, so the client has no choice but to accept the compression you are giving it regardless of whether it knows how to decode it.
Think forward 5 years to some new magic "INSIDEOUT" compression algorithm and imagine that you don't have control over what version of the client somebody is running, then you upgrade to this compression algorithm and break things for all the old clients that you had hard coded to decompress via gzip, but now the server actually responded with a different compression type.
No description provided.