feat: endpoint reachability plugin

[5war00p]

Endpoint Reachability Plugin

• Brief Description of Changes
  Added new plugin that checks and validates whether an endpoint (url) is reachable or unreachable.

Motivation: (optional)

• Why this change is necessary or beneficial
  Its a preventive check to not do unnecessary actions without a proper input

Related Issues: (optional)

• [Feature] Endpoint Is Reachable Plugin #512

[roh26it]

Thanks for the PR @5war00p!

Does this only check the entire content of the request to assume that it contains a URL?

We also have another plugin - https://github.com/Portkey-AI/gateway/blob/main/plugins/default/validUrls.ts

Is that conflicting?

[5war00p]

This just does almost same, but instead of GET request I'm using HEAD request also we as we need to validate endpoint is reaching or not just header is fine no need validate dns separately.

[narengogi]

Looks good, just that this assumes that the response only contains a URL. in comparison,
https://github.com/Portkey-AI/gateway/blob/main/plugins/default/validUrls.ts feels more complete, I think it's a good idea to make a HEAD request to check the validity of a URL, you can perhaps add it as a fallback in the code of the existing plugin. That would be a 2 line change and more robust

[5war00p]

To your point, validUrl ideally should be a regexp check thats what name speaks about. Where as endpointReachability make sure it reaches. If we really don't need 2 separate then we can merge, but validUrl can be used to validate a url input but if we make sure its a reachable one then merging both will be good idea.

[b4s36t4]

My suggestion here is use regex to get all the URLs available from text and validate them using URL or some other way.

With this we can have config weather to run plugin for all the endpoints or first endpoint found within the text.

What do you think? @5war00p

[5war00p]

Makes sense if text might contain >1 url, however this issue is to build a plugin for endpoint reachability right? and we should use this util func in another which extracts urls and check each url using this plugin. Correct me if I went off.

[b4s36t4]

Your thought makes sense, but this PR is a plugin PR which I think doesn't make sense to use it as utility function. I think we should focus on keep this as individual plugin. keeping that I prefer we should go for regex match.

As an enhancement I think we shouldn't just blindly hit the URL, we should ask user which type of method they're thinking of ex: GET and POST and let plugin do OPTIONS request and make sure the endpoint(s) support the user requested methods.

let me know if this makes sense to you.

[5war00p]

I think if thats the case we should make it as a new separate plugin.

[narengogi]

To your point, validUrl ideally should be a regexp check thats what name speaks about. Where as endpointReachability make sure it reaches. If we really don't need 2 separate then we can merge, but validUrl can be used to validate a url input but if we make sure its a reachable one then merging both will be good idea.

sorry for the late reply @5war00p , as you might have noticed during your testing, the plugins act on the request and response body, so your plugin assumes it recieves only the URL in the body. Hence the plugin is not of much use.

But it's a cool implementation like I said, but for now this cannot be merged

[5war00p]

@narengogi In that case we can do as @b4s36t4 suggested, extract links and verify endpoint reachability, but question is do we need to do that for all links in found in entire text and how we do we determine that?

[narengogi]

@5war00p you can use regex to extract the URLs

[VisargD]

Hey folks! Are we moving forward with this implementation?