Generic push_to_hub by adding optional save_fn

[nateraw]

Something I've encountered in #284 and elsewhere (when integrating 3rd party libraries directly), is that folks may have to rewrite push_to_hub to fit their needs. However, it seems to me the only change you end up needing to make is specifying what needs to be saved.

I propose we do something like this:

1. add a generic push_to_hub function.
2. add a kwarg to this function, save_fn, that will allow users to pass a function that saves any assets specific to their model.

The logic would then look like the following:

def push_to_hub(model, save_directory, ..., save_fn=None):
    # Init/clone repo
    # ...

    # Run save func if provided or run the default if its not
    save_fn = save_fn or default_save_fn
    
    with repo.commit(commit_message):
        save_fn(model, save_directory, config)

    return repo.remote_url

What do you think? @LysandreJik @osanseviero

[osanseviero]

I like the idea but let's see how to avoid duplication. This sounds very similar to the generic mixin, which has a generic push_to_hub method and users just need to overwrite _save_pretrained.

Anyways, +1 to this since we have push_to_hub repetition across multiple libraries, although let's be careful since each has some different things

• https://github.com/explosion/spacy-huggingface-hub/blob/main/spacy_huggingface_hub/push.py
• https://github.com/UKPLab/sentence-transformers/blob/master/sentence_transformers/SentenceTransformer.py#L415
• Proposal of utility function + command to push models to HF Hub allenai/allennlp#5370