Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature th http client for RTS-258 #1371

Closed

Conversation

lehoff
Copy link
Contributor

@lehoff lehoff commented Mar 14, 2016

Re-worked to use Webmachine better.

Open issues:

  • The query API only accepts POST and uses a ?query="..." query string in the URL to pass in the query. This is to make it consistent across HTTP methods if we want to match the side effects of the query with the HTTP method.
  • The output from list_keys is some hrefs inside a <html> block. Perhaps we should change the design and provide something more useful. It is not of much use when doing a curl or http (httpie.org) request on the command line.
  • Dialzyer is not fully happy about the listkeys module, but it smells like a Dialyzer bug - see comment in that file.
  • The commits have not been squashed - open to doing that before doing the final PR.

Jon Meredith and others added 30 commits January 26, 2016 16:40
Queries still require equality on family and series with a range
of timestamps, but will now retrieve records with local keys
defined after the partition key.
…ldb backend. Depends on same branch in eleveldb
…ch replies or should we just use standard put replies?
…atches if the total number of records is less than the max per batch
…ive, to ensure it is always greater than the start key.
…t/flexi_keys

Conflicts:
	src/riak_kv_qry_compiler.erl
Flexible keys

Reviewed-by: andytill
Output some query debug information

Reviewed-by: javajolt
Torben Hoffmann added 14 commits March 28, 2016 19:31
* removed malformed_request
* changed CB_RV_SPEC to cb_rv_spec(T) type
* simplified forbidden/2
* removed check_permissions/2 as it is no longer needed (is_authorized/2 handles that job now)
* removed utf8_to_binary/1 as it is no longer used (riak_kv_wm_ts_util has it for those functions who needs it).
* error_out, handle_error and flat_format removed as they are no longer used.
…right order.

And all helper functions at the end of the module.
@hmmr
Copy link
Contributor

hmmr commented Mar 29, 2016

@lehoff have a look how keys are sent in standard riak: https://github.com/basho/riak_kv/blob/riak_ts-develop/src/riak_kv_wm_keylist.erl#L230, and how they are received: https://github.com/basho/riak-erlang-http-client/blob/riak_ts-develop/src/rhc_listkeys.erl#L86.

There is also my branch in riak-erlang-http-client, https://github.com/basho/riak-erlang-http-client/tree/feature/az/http-client, where I have implemented all the TS-specific API calls. I then used riakhttpc as the alternative client in ts_cluster_comprehensive (basho/riak_test#1016) to run the same scenario as it normally does with the protobuffer client (riakc). The wrappers in riak-erlang-http-client will probably need to be modified per your changes in riak_kv.

Torben Hoffmann added 3 commits March 30, 2016 08:53
…re-th-http_client-rebase

# Conflicts:
#	src/riak_kv_pb_timeseries.erl
#	src/riak_kv_ts_api.erl
#	src/riak_kv_ts_util.erl
#	src/riak_kv_wm_timeseries.erl
#	src/riak_kv_wm_timeseries_listkeys.erl
#	src/riak_kv_wm_timeseries_query.erl
#	src/riak_kv_wm_ts_util.erl
@lehoff
Copy link
Contributor Author

lehoff commented Mar 30, 2016

It should be stream - fix for that coming up.
For the list_keys the idea was to return a stream of {"url": "http://..."} pairs.
The html tag is a leftover that should have been removed.
All the other timeseries HTTP endpoints return JSON, so it might be best to continue that.
So chunks in JSON arrays seems best. But should it just be an array of `["http://...", "http://..."] or should we tag them with "url" in pairs I originally envisioned?

@hmmr
Copy link
Contributor

hmmr commented Apr 4, 2016

The query callback still needs some tweaks to accept the recently implemented WITH-clause for the CREATE query. The changes are in lehoff-b#2, for you to consider and pull.

With those changes, I can confirm the test in basho/riak_test#1016 passes.

Edit: The riak_test PR is 1015, not 1016.

…develop

amend create_table code path to take into account WITH-props
@hmmr
Copy link
Contributor

hmmr commented Apr 5, 2016

+1 383e049

The tests (basho/riak_test#1015) all pass.

borshop added a commit that referenced this pull request Apr 5, 2016
Feature th http client for RTS-258

Reviewed-by: hmmr
@jonmeredith
Copy link
Contributor

... and the dialyzer says no, with a strange unrelated error to eleveldb.

eleveldb.erl:324: Call to missing or unexported function lager:error/2

@hmmr
Copy link
Contributor

hmmr commented Apr 6, 2016

Okaay, it appears @lehoff forgot to accept lehoff-b#3, something I had presumed he did..

I have opened a yet another PR, with identical contents as this one plus the last commit to fix the dialyzer issue: #1382.

@lehoff
Copy link
Contributor Author

lehoff commented Apr 19, 2016

Should be covered by #1398

@lehoff lehoff closed this Apr 19, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants