Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implementing "tick" to avoid using sleep #127

Open
kaiku opened this issue Sep 9, 2016 · 14 comments
Open

Implementing "tick" to avoid using sleep #127

kaiku opened this issue Sep 9, 2016 · 14 comments

Comments

@kaiku
Copy link

kaiku commented Sep 9, 2016

Great work on this library. Just wondering, have you considered adding a tick method or some sort of frozen time mode to FakeRedis that allows programmatic advancing of time to avoid using an actual sleep in the test code?

@jamesls
Copy link
Owner

jamesls commented Dec 7, 2016

Wouldn't be opposed to a feature like that. Marking as HelpWanted.

@charettes
Copy link
Contributor

freezegun can be used for this purpose and works like a charm with fakeredis because the later uses the stdlib datetime module to determine if keys are expired. Feels like it's not something that should be implemented in this package.

@bmerry
Copy link
Collaborator

bmerry commented Nov 18, 2020

Thanks, I haven't come across freezegun before. From a quick look at the docs it appears to mock the functions that get time, but not time.sleep - in which case it would presumably cause an infinite loop since the timeout would never be reached?

@charettes
Copy link
Contributor

@bmerry DataBase.time is retrieved from time.time() on command processing

now = time.time()
for db in self._server.dbs.values():
db.time = now

This method is mocked by freezegun which means tests such as

@pytest.mark.slow
def test_expireat_should_expire_key_by_timestamp(r):
r.set('foo', 'bar')
assert r.get('foo') == b'bar'
r.expireat('foo', int(time() + 1))
sleep(1.5)
assert r.get('foo') is None
assert r.expire('bar', 1) is False

Could be rewritten as

@freeze_time(as_kwarg='frozen_time')
def test_expireat_should_expire_key_by_timestamp(r, frozen_time):
    r.set('foo', 'bar')
    assert r.get('foo') == b'bar'
    r.expireat('foo', int(time() + 1))
    frozen_time.tick(1.5)
    assert r.get('foo') is None
    assert r.expire('bar', 1) is False

@charettes
Copy link
Contributor

charettes commented Nov 18, 2020

In the case of internal usages of time.sleep

time.sleep(0.01)

They can be mocked with mock.patch('time.sleep', frozen_time.tick). There even seems to be a solution for asyncio.sleep spulec/freezegun#290

@bmerry
Copy link
Collaborator

bmerry commented Nov 18, 2020

I think one problem is that the tests run against both real redis and fake redis to ensure that the tests have correct expectations, and of course freezegun won't work with real redis. But I can probably build a pytest fixture to do the mocking in the appropriate cases, which would at least halve the sleep time in tests. I'll take a look at some point when I have free time - thanks for the suggestion.

@adamchainz
Copy link

FYI I built a faster alternative to freezegun called time-machine. See https://adamj.eu/tech/2020/06/03/introducing-time-machine/ .

@slothyrulez
Copy link

slothyrulez commented Nov 23, 2023

@adamchainz sadly I'm trying this right now and does not seem to work 😢

def test_cache_expiration(self,...):
        frozen_time = pytz.UTC.localize(datetime.datetime(2023, 11, 23, 14, 52))
        with time_machine.travel(frozen_time, tick=False) as travel_time:
            print(datetime.datetime.now())
            print(cache.has_key('XXX'))
            print(cache.pttl('XXX'))
            ...
            # some caching with django cache and fakeredis
            # some asserts...
            ...
            before_expire_delta = datetime.timedelta(seconds=(CACHE_TIEMOUT - 10))
            travel_time.shift(before_expire_delta)
            print(datetime.datetime.now())
            print(cache.has_key('XXX'))
            print(cache.pttl('XXX'))
            ...
            # some asserts
            ...
            after_expire_delta = datetime.timedelta(seconds=(CACHE_TIEMOUT + 10))
            travel_time.shift(after_expire_delta)
            print(datetime.datetime.now())
            print(cache.has_key('XXX'))
            print(cache.pttl('XXX'))

2023-11-23 14:52:00                                                                                                                                                                                                                                                                                                                                                        
False                                                                                                                                                                                                                                                                                                                                                                      
0 
...   
2023-11-23 14:56:50                                                                                                                                                                                                                                                                                                                                                        
True                                                                                                                                                                                                                                                                                                                                                                       
299999                                                                                                                                                                                                                                                                                                                                                                     
...
2023-11-23 15:02:00                                                                                                                                                                                                                                                                                                                                                        
True                                                                                                                                                                                                                                                                                                                                                                       
299991                                                                                                                                                                                                                                                                                                                                                                     
...

@adamchainz
Copy link

@slothyrulez Hmm I am afraid I can't see why exactly. It looks like fakeredis (the new maintained repo) calls time.time() on each command: https://github.com/cunla/fakeredis-py/blob/04d4703d6bf7bc7c9d98d2d128cd206de80787b3/fakeredis/_basefakesocket.py#L271

time-machine definitely mocks time.time() just fine. If you add time.time() calls in your test they should show updates just like datetime.now().

Maybe the above code path isn't running, so the server time variable is not updating correctly. Calling the time command would probably be instructive.

@slothyrulez
Copy link

Ummmmm, should I open a discussion on time-machine or in fakeredis ?

Definitively, thanks for your response @adamchainz

@adamchainz
Copy link

I think this is an issue with how fakeredis reads the time. I’d only want a time-machine issue if you can show there’s an underlying cause that needs a fix.

@slothyrulez
Copy link

slothyrulez commented Nov 24, 2023

@slothyrulez Hmm I am afraid I can't see why exactly. It looks like fakeredis (the new maintained repo) calls time.time() on each command: https://github.com/cunla/fakeredis-py/blob/04d4703d6bf7bc7c9d98d2d128cd206de80787b3/fakeredis/_basefakesocket.py#L271

time-machine definitely mocks time.time() just fine. If you add time.time() calls in your test they should show updates just like datetime.now().

Maybe the above code path isn't running, so the server time variable is not updating correctly. Calling the time command would probably be instructive.

BEFORE time_machine.travel --
datetime.datetime.now()=datetime.datetime(2023, 11, 24, 6, 1, 23, 8122)
time.time()=1700805683.0081432
cache.client.get_client().time()=(1700805683, 8477)
cache.has_key("XXXX")=False
cache.pttl("XXXX")=0

AFTER time_machine.travel --
datetime.datetime.now()=datetime.datetime(2023, 11, 23, 14, 52)
time.time()=1700751120.0
cache.client.get_client().time()=(1700805683, 16603)
cache.has_key("XXXX")=False
cache.pttl("XXXX")=0
--
CACHED DURING 300 secs (5min.)
--
TRAVEL 4m50s to the future
datetime.datetime.now()=datetime.datetime(2023, 11, 23, 14, 56, 50)
time.time()=1700751410.0
cache.client.get_client().time()=(1700805683, 38205)
cache.has_key("XXXX")=True
cache.pttl("XXXX")=299999
--
TRAVEL 15s to the future
datetime.datetime.now()=datetime.datetime(2023, 11, 23, 14, 57, 5)
time.time()=1700751425.0
cache.client.get_client().time()=(1700805683, 47826)
cache.has_key("XXXX")=True
cache.pttl("XXXX")=299989

@slothyrulez
Copy link

Cross-posting here for future visibility cunla/fakeredis-py#253
Thanks @jamesls and sorry for the noise

@earonesty
Copy link

freezegun doesn't work with aioredis, unfortunately.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

7 participants