Promises acting synchronously

[blitzmann]

Greetings!

When it comes to promises, I hail from the land of AngularJS, which includes a few nice wrappers. I've only had to deal with pure promises a few times, so this issue is most likely me not understanding how to set this up (and there is a lack of basic examples in the README, instead directing readers to the spec)

This is the current code I have:

from promise import Promise
import urllib.request

url = 'https://jsonplaceholder.typicode.com/photos'

def success(data):
    print("promise success!")
    # print(data)

def reject(data):
    print("rejected!")
    print(data)

def getData(url):
    def func(resolve, reject):
        
        req = urllib.request.urlopen(url)
        if req.getcode() == 200:
            resolve(req.read())
        else:
            reject(Exception(req.getcode()))

    return Promise(func)

print("getting resource")
p = getData(url)
p.then(success, reject)
print("Waiting on promise")

This seems to work synchronously: once getData() is called, it returns a promise, but it also blocks execution. This is what the output looks like:

getting resource
promise success!
Waiting on promise

The order I was expecting:

getting resource
Waiting on promise
promise success!

I've also tested this with a simple time.sleep(5) and resolving immediately after that. Again, once getData() is run, it simply sleeps for 5 seconds without hitting the "Waiting" print, then continues onword.

Sorry if this is a bit of an obtuse question, I acknowledge I'm probably not working with it correctly, but I can't seem to figure it out.

[LivInTheLookingGlass]

The dev is aware of it, but doesn't want to make a change. Apparently the A+ spec says to do it this way, even though its both less obvious and less useful. I've opened an issue about it in the past.

[blitzmann]

Maybe my understanding of promises is completely different then, but... isn't the whole point of a promise to do work asynchronously and then return a result sometime in the future? Is this package simply to get the Promise syntax?

After looking around some more, seems like maybe this might be a feature that is currently being worked on? Found this (seems to be your implementation): https://pypi.python.org/pypi/async-promises/ (merged with #20, but without creation of new Thread for resolvers). I am also not too familiar with best practices involving this stuff (Threads vs asyncio). I haven't tested it, but since I'm currently starting new Threads in my application to handle stuff, so I'm not too concerned about the additional overhead

I will take a look at your fork, is anyone else aware of similar projects that might be useful?

@syrusakbary would still be interested in hearing if this is a planed feature currently being worked on (and if so, current state of work), or if it's been decided against implementing it async.

Thanks!

[chtseac]

To everyone who stumble into this issue having their expectations mismatched that this would work like the promises in <your language>:

Here is how I added a thing to have a decorator that acts like Promise.promisify except it also starts a thread. Be warned of thread safety - this isn't javascript where code is run in a single-thread event loop.

from functools import wraps
from threading import Thread
from promise import Promise

def threadedPromisify(func):
    prom_func = Promise.promisify(func)
    @wraps(prom_func)
    def wrapper(*args, **kwds):
        def call_func_async(res, rej):
            thread = Thread(
                target=lambda: prom_func(*args, **kwds).then(res, rej)
            )
            thread.start()
        return Promise(call_func_async)
    return wrapper

# if you want a function that simply calls a function
# (there's probably a better way of doing this since I wrote the promisify thing first)
def callAsync(fun):
  return threadedPromisify(fun)()

Usage:

@threadedPromisify
def someFunction():
  return 10

# prints 10 asynchronously
someFunction().done(lambda i: print(i))

# prints content of README.md asynchronously
callAsync(lambda: open('README.md')).then(lambda f: f.read()).done(lambda t: print(t))

[BryceCicada]

Looking at the code, it seems that these promises are synchronous by default. This was a surprise to me too having come straight from javascript promises. However, a little more digging shows that it's already considered by the developers.

Here's a small change to the code in the original post that uses AsyncIO.

from promise import Promise, set_default_scheduler
from promise.schedulers.asyncio import AsyncioScheduler
import urllib.request

url = 'https://jsonplaceholder.typicode.com/photos'

def success(data):
    print("promise success!")
    # print(data)

def reject(data):
    print("rejected!")
    print(data)

def getData(url):
    def func(resolve, reject):

        req = urllib.request.urlopen(url)
        if req.getcode() == 200:
            resolve(req.read())
        else:
            reject(Exception(req.getcode()))

    return Promise(func)


set_default_scheduler(AsyncioScheduler())

print("getting resource")
p = getData(url)
p2 = p.then(success, reject)
print("Waiting on promise")
Promise.wait(p2)

Alternatively, you can pip install gevent and then use

from promise.schedulers.gevent import GeventScheduler
...
set_default_scheduler(GeventScheduler())
...

There's also a ThreadScheduler in this package but, for me, that seems to give the same results observed in the original post. Maybe there's something going on with the GIL. Anyhows, event-loops make me happier.

[chtseac]

Oh, this is awesome! Why this is not in the readme? We should totally add it to the documentation!

…

On Jan 26, 2018 7:20 PM, "Chris Nix" ***@***.***> wrote: Looking at the code, it seems that these promises are synchronous by default. This was a surprise to me too having come straight from javascript promises. However, a little more digging shows that it's already considered by the developers. Here's a small change to the code in the original post that uses AsyncIO. from promise import Promise, set_default_scheduler from promise.schedulers.asyncio import AsyncioScheduler import urllib.request url = 'https://jsonplaceholder.typicode.com/photos' def success(data): print("promise success!") # print(data) def reject(data): print("rejected!") print(data) def getData(url): def func(resolve, reject): req = urllib.request.urlopen(url) if req.getcode() == 200: resolve(req.read()) else: reject(Exception(req.getcode())) return Promise(func) set_default_scheduler(AsyncioScheduler()) print("getting resource") p = getData(url) p2 = p.then(success, reject) print("Waiting on promise") Promise.wait(p2) Alternatively, you can pip install gevent and then use from promise.schedulers.gevent import GeventScheduler ... set_default_scheduler(GeventScheduler()) ... There's also a ThreadScheduler in this package but, for me, that seems to give the same results observed in the original post. Maybe there's something going on with the GIL. Anyhows, event-loops make me happier. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#45 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ANTPa1Zb7AkBwQ55loQXxp5P3H-XVkofks5tObUEgaJpZM4QCVqh> .

[pkopac]

Hi! I've just tried a variant of your code example with time measuring and multiple promises locally and... it really doesn't work (python 3.6):

Bug 1: Promise(func) executes immediately and blocks main thread, whatever the scheduler.

Bug 2: Promise.promisify(func) returns a function, not a promise.

Executing promises
0
Time elapsed (hh:mm:ss.ms) 0:00:02.121857
1
Time elapsed (hh:mm:ss.ms) 0:00:06.121378
2
Time elapsed (hh:mm:ss.ms) 0:00:11.926724
[<Promise at 0x7f00fa7180b8 pending>, <Promise at 0x7f00fa701a58 pending>, <Promise at 0x7f00fa7289e8 pending>]
Waiting on all promises
promise success!
promise success!
promise success!
Time elapsed (hh:mm:ss.ms) 0:00:11.928643

With ThreadScheduler:

Executing promises
0
promise success!
Time elapsed (hh:mm:ss.ms) 0:00:01.692070
1
promise success!
Time elapsed (hh:mm:ss.ms) 0:00:03.782801
2
promise success!
Time elapsed (hh:mm:ss.ms) 0:00:05.826248
[<Promise at 0x7fd12df93a90 fulfilled with None>, <Promise at 0x7fd12de42cc0 fulfilled with None>, <Promise at 0x7fd12de5a320 fulfilled with None>]
Waiting on all promises
Time elapsed (hh:mm:ss.ms) 0:00:05.827753

My full code:

from promise import Promise, set_default_scheduler, get_default_scheduler
from promise.schedulers.asyncio import AsyncioScheduler
from promise.schedulers.gevent import GeventScheduler
from promise.schedulers.thread import ThreadScheduler
from datetime import datetime
import urllib.request

# set_default_scheduler(AsyncioScheduler())
set_default_scheduler(ThreadScheduler())
# set_default_scheduler(GeventScheduler())

url = 'https://jsonplaceholder.typicode.com/photos'
def getData(url):
    def func(resolve, reject):
        req = urllib.request.urlopen(url)
        if req.getcode() == 200:
            resolve(req.read())
        else:
            reject(Exception(req.getcode()))

    return Promise(func)

start_time = datetime.now()

def success(data):
    print("promise success!")
    # print(data)

def reject(data):
    print("rejected!")
    print(data)

def print_elapsed_time():
    time_elapsed = datetime.now() - start_time
    print('Time elapsed (hh:mm:ss.ms) {}'.format(time_elapsed))

print("Executing promises")
l = []
for x in range(0, 3):
    print(x)
    l.append(getData(url).then(success, reject))
    print_elapsed_time()

print(l)
print("Waiting on all promises")
Promise.all(l).get()
print_elapsed_time()

I was able to get it work as async promises with a really ugly syntax:

def getDataHack(url):
    def func(value):
        req = urllib.request.urlopen(url)
        if req.getcode() == 200:
            Promise.resolve(req.read())
        else:
            Promise.reject(Exception(req.getcode()))

    return Promise.resolve("foo").then(func, None)

Executing promises
0
Time elapsed (hh:mm:ss.ms) 0:00:00.001408
1
Time elapsed (hh:mm:ss.ms) 0:00:00.001448
2
Time elapsed (hh:mm:ss.ms) 0:00:00.001474
[<Promise at 0x7f8e75321eb8 pending>, <Promise at 0x7f8e75321fd0 pending>, <Promise at 0x7f8e7533a0f0 pending>]
Waiting on all promises
promise success!
promise success!
promise success!
Time elapsed (hh:mm:ss.ms) 0:00:05.912553

[pkopac]

OK, @chtseac commented chartmogul/chartmogul-python#18 (comment) about promisify, so I tried it again:

def getDataPromisify(url):
    def func():
        urllib.request.urlopen(url).read()

    return Promise.promisify(func)()

...
print("Executing promises")
l = []
for x in range(0, 3):
    l.append(getDataPromisify(url).then(success, reject))
    print_elapsed_time()
...

Output (with ThreadScheduler):

$ python3.6 main.py 
Executing promises
promise success!
Time elapsed (hh:mm:ss.ms) 0:00:02.328537
promise success!
Time elapsed (hh:mm:ss.ms) 0:00:04.316264
promise success!
Time elapsed (hh:mm:ss.ms) 0:00:07.016003
[<Promise at 0x7fb336305e10 fulfilled with None>, <Promise at 0x7fb3361cc048 fulfilled with None>, <Promise at 0x7fb3384d4630 fulfilled with None>]
Waiting on all promises
Time elapsed (hh:mm:ss.ms) 0:00:07.016739

The () in the Promise.promisify(func)() was unexpected for me and is a bit awkward IMHO. It also executed immediately.

[chtseac]

@pkopac I trimmed down the code to a minimal case and it is clearly not working.

from promise import Promise, set_default_scheduler
from promise.schedulers.asyncio import AsyncioScheduler
from promise.schedulers.gevent import GeventScheduler
from promise.schedulers.thread import ThreadScheduler
import time

# set_default_scheduler(AsyncioScheduler())
set_default_scheduler(ThreadScheduler())
# set_default_scheduler(GeventScheduler())

@Promise.promisify
def sleep():
	print('sleep start')
	time.sleep(2)
	print('sleep end')
	return 'promise return'

sleep().then(print)

Ouptut:

sleep start
sleep end
promise return

pip list output:

$ pip list
asyncio (3.4.3)
gevent (1.2.2)
greenlet (0.4.13)
pip (8.1.1)
pkg-resources (0.0.0)
promise (2.1)
setuptools (20.7.0)
six (1.11.0)
typing (3.6.4)

[flewellyn]

Given this issue is still open, and I'm running in to the same problem, how do we solve it?

I'm trying to use the Dataloader with Graphene and SQLalchemy, and everything either runs sequentially, or if I try to use a scheduler, it gives me the error "call() missing 1 required positional argument: 'fn'".

Any ideas?

[pollaro]

I'm having trouble with my DataLoader queueing things. It seems to load the data immediately when I call load() instead of waiting.