Delay Request Connection

[cebe]

I am working on a project which will make a lot of connections to external APIs and I need to queue the requests so that I can limit the number of active requests. This is needed to manage the load on the API (also may be useful for APIs that enforce a request limit).

For this reason I want to build the request and attach all event handlers, also write some data, but not yet send it.

Currently write() will directly open the connection and send the data.

http-client/src/Request.php

Lines 103 to 107 in d779a3b

	// otherwise buffer and try to establish connection
	$this->pendingWrites .= $data;
	if (self::STATE_WRITING_HEAD > $this->state) {
	$this->writeHead();
	}

I suggest to add a method that allows adding data to $pendingWrites without sending the request.

[kelunik]

How's the first related to the latter?

If you want to buffer some data but not send it, you can just buffer it yourself and delay your call to writeHead().

[cebe]

I'd like to prepare a Request object with all headers, body and events attached without sending it and put that into the queue. The only thing that I currently can not add is the request body as adding it will cause the request to make the connection. So I suggest to add that option too.

I can of course wrap the Request in another object, but that is more complex than it needs to be.

[cebe]

More generally I'd like to have more control over when the request is actually sent. The current programming interface hides the details about sending the request, so by creating a new request object and calling methods on it it is not quite clear at which point the request is being sent.

[kelunik]

Oh, I see what you mean. I was under the impression that this package used $client->request($request), but it does not. Nevermind then. It's easiest to store this in your own object then or to use another library until such a new API is implemented.

[clue]

@cebe More generally I'd like to have more control over when the request is actually sent. The current programming interface hides the details about sending the request, so by creating a new request object and calling methods on it it is not quite clear at which point the request is being sent.

Thank you bringing up this issue! I agree that this is a relevant issue and this is certainly something that I would like to address in a future update.

@kelunik If you want to buffer some data but not send it, you can just buffer it yourself and delay your call […].

I think that this approach makes perfect sense and this is the kind of approach that I would prefer for a future implementation.

We're currently looking into implementing PSR-7 interfaces via #41. Once this is in, we will provide a way to pass a request object to our API which will then take over responsibility for sending the request and receiving a response in return.

I think that this should fulfill your use case and I don't see anything that needs to be done beyond this. In particular, there are currently no plans to queue pending requests as far as I'm aware. That being said, I think implementing some kind of concurrency limit on top of this sounds reasonable. If you feel this is needed, I would suggest filing a new ticket to keep track of this and/or file a PR to implement this feature

I believe this has been answered, so I'm closing this for now. Please come back with more details if this problem persists and we can reopen this 👍

[cebe]

Thanks for the detailed answer! Switching to using PSR-7 will likely solve this, so we can keep this issue closed. Limiting the number of requests highly depends on the application so that is something I am implementing in the app itself. There are a lot of variables like limiting per host, per timeframe, per endpoint, etc. I do not see a good way to create a generic implementation for this that would fit for the library.