SearchFilter with unaccent

[gdvalderrama]

Currently SearchFilter offers 4 restrictions:

• '^' Starts-with search.
• '=' Exact matches.
• '@' Full-text search. (Currently only supported Django's PostgreSQL backend.)
• '$' Regex search.

It would be great to have the possibility of using __unaccent lookup.

Example:

Search for "Héctor" would also return "Hector".

[tomchristie]

That's an interesting idea yup.

Questions:

• Does the __unaccent filter end up being case sensitive or case insensitive?
• How can you control if it is case sensitive or case insensitive?

I suppose a suggestion here might be to use the existing functionality, and tweak the lookup prefixes?...

Eg. either this...

class UnaccentedSearchFilter(SearchFilter):
    lookup_prefixes = {
        '^': 'istartswith',
        '=': 'unaccent',
        '@': 'search',
        '$': 'iregex',
    }

Or this...

class UnaccentedSearchFilter(SearchFilter):
    lookup_prefixes = {
        '^': 'istartswith',
        '=': 'iexact',
        '~': 'unaccent',
        '@': 'search',
        '$': 'iregex',
    }

Which perhaps resolves the use-case without requiring any code changes in the framework?

[carltongibson]

Adding a new class UnaccentedSearchFilter is probably the way to go for this.

So @gdvalderrama — could that live in a third-party package at the first pass to see what it looks like?
(Then it can either live there to share the maintenance burden or be brought in, as appropriate.)

[gdvalderrama]

Adding a new class UnaccentedSearchFilter is probably the way to go for this.

So @gdvalderrama — could that live in a third-party package at the first pass to see what it looks like?
(Then it can either live there to share the maintenance burden or be brought in, as appropriate.)

Making a third party package for this sounds a bit overboard.
Would this be preferred over making a PR with a new UnaccentedSearchFilter class?

[tomchristie]

Intuitively I'd expect that most search functionality ought to typically be unaccented, case-insensitive full-text search.

One option here might be to add some docs about how to achieve that with a Postgres database, without making any code changes. (What would that documentation look like, exactly? Presumably we'd also want to mention the UnaccentExtension migration.)

Also it looks to me like there might be some nuances here... I'm not sure from the Postgres docs if full text search automatically becomes always-unaccented-lookups once the "unaccent extension" has been activated. (Or if you still need to adapt the SQL query)

[kmahelona]

We'd also appreciate this feature. In the meantime we subclassed SearchFilter. This worked for us:

class UnaccentFilter(SearchFilter):
    lookup_prefixes = {
        '^': 'istartswith',
        '=': 'iexact',
        '@': 'search',
        '$': 'iregex',
        '&': 'unaccent__icontains',
    }

Most cases we'll want case-insensitive. It might be better though to update this to be more flexible. For example I tried to just do search_fields = ['name__unaccent__icontains'] but then I looked at the code and was like oh, search_fields = ['name__unaccent'], but then there's a check that saw that unaccent isn't a field on name and raised an error.

[gdvalderrama]

Also it looks to me like there might be some nuances here... I'm not sure from the Postgres docs if full text search automatically becomes always-unaccented-lookups once the "unaccent extension" has been activated. (Or if you still need to adapt the SQL query)

Nope, you still have to specify you want to use unaccent in each query.
So just documenting how this works with Postgres is not enough, the actual query has to be adapted in the search filter.