Skip to content

Poor pickling performance on macOS #360

Open
@lohedges

Description

@lohedges

As reported by @msuruzhon, pickling/unpickling large BioSimSpace objects has very poor performance on macOS. Using System.pickle.tar.gz we see:

import BioSimSpace.Sandpit.Exscientia as BSS

import pickle
import time

a = time.time()
s = pickle.load(open("System.pickle", "rb"))
b = time.time()
print(f"Unpickling BioSimSpace system took {b - a} seconds")

a = time.time()
with open("tmp.pickle", "wb") as f:
    pickle.dump(s, f)
b = time.time()
print(f"Pickling BioSimSpace system took {b - a} seconds")

a = time.time()
with open("tmp.pickle", "wb") as f:
    pickle.dump(s._sire_object, f)
b = time.time()
print(f"Pickling Sire system took {b - a} seconds")

Linux:

Unpickling BioSimSpace system took 2.6445674896240234 seconds
Pickling BioSimSpace system took 2.548236131668091 seconds
Pickling Sire system took 1.7895822525024414 seconds

macOS:

Unpickling BioSimSpace system took 3.2096989154815674 seconds
Pickling BioSimSpace system took 125.13923001289368 seconds
Pickling Sire system took 2.2229161262512207 seconds

(The Linux system is newer/faster than the macOS one, but the discrepancy is consistent with different macOS machines.)

While the Sire object pickling takes 1.3 times as long on macOS, the BioSimSpace object takes 50 times longer!

I'll try to figure out what attribute in the Python object is making the pickling so slow.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions