Skip to content

Optimisations for compiling large fluent files #32

Open
@leamingrad

Description

@leamingrad

Hi 👋. Firstly, thanks for this library - it is really useful!

I'm currently working with fluent-compiler (via django-ftl) as part of a large Django project. Its generally been great, but unfortunately compiling our fluent files contributes quite a lot (~10s) to our apps startup time.

With that in mind, I was hoping to contribute a couple of optimisations to speed up the compilation of large fluent files, and have raised this issue to track everything.

Current PRs

I've raised the following PRs, and am happy to make fixups as needed:

Potential next steps

After removing the set copies, the biggest contributor to compile times is the span_to_position function.

The issue is that the function requires us to scan through the text of the fluent file from the start in order to work out the row number of the element.

Ideally we would have these positions when the fluent file is parsed, but python-fluent does not do this for us. I think there are two ways to optimise here:

  1. Update python-fluent to include position information with span information
  2. Tag the FluentParser output with position information inside fluent-compiler

I've got a proof-of-concept for 2 which shaves ~2.5s from the 10K benchmark, but this isn't ideal as it imposes overhead on fluent-compiler.

What do you think the best way forward is? I'd be happy to put a PR up for python-fluent, but I'm not sure if it is worth also putting up a PR for option 2 here in the meantime (since I'm not sure how quickly python-fluent does releases.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions