My project based on PyWorkflow

[cfhamlet]

Hi guys, I have created a project os-pywf which wrap the PyWorkfow provide high level APIs and command line tools.
I have released Requets like APIs and provide a command line tool same as curl but more flexible work with Python scripts.
My project can be installed with pip

pip install os-pywf

During developing, I find some features depond on PyWorkflow, need your help

• HttpTask do not support proxies
• I do not konw how to add client cert when using HttpTask
• Running task can not cancel, so on some situation (slow response for example) I have to force quit the program or wait a long time

[Barenboim]

To use http proxy, you may create an http task with proxy's URL like this: http://proxy-host-name:port/ ，set the request URI: task.get_req().set_request_uri("http://www.sogou.com/"), and then set the "Host" header: task.get_req().set_header_pair("Host", "www.sogou.com")
Unfortunately we do not support https proxy currently.

[cfhamlet]

Thanks, let me have a try

[cfhamlet]

To use http proxy, you may create an http task with proxy's URL like this: http://proxy-host-name:port/ ，set the request URI: task.get_req().set_request_uri("http://www.sogou.com/"), and then set the "Host" header: task.get_req().set_header_pair("Host", "www.sogou.com")
Unfortunately we do not support https proxy currently.

It works. Expect solution for https proxy.

[kedixa]

GOOD NEWS

pyworkflow support visit http(s) server with http proxy now ( #19 )

you can use it as simple as normal http tasks

import pywf as wf

def http_callback(t):
    if t.get_state() != wf.WFT_STATE_SUCCESS:
        print(wf.get_error_string(t.get_state(), t.get_error()))
        return

    resp = t.get_resp()
    print("{} {} {}".format(
        resp.get_http_version(), resp.get_status_code(), resp.get_reason_phrase()
    ))

task = wf.create_http_task(
    url = "https://www.sogou.com/",
    proxy_url = "http://user:[email protected]:port", # Your http proxy here
    redirect_max = 2,
    retry_max = 2,
    callback = http_callback)
task.start()
wf.wait_finish()

[cfhamlet]

@kedixa Good job, it's GREAT.