Index topics and extensions and various other tweaks #11
Conversation
| // I wonder if we could use something similar to https://stackoverflow.com/a/74737474/5043585 | ||
| // to have some sort of weight in the documents and prioritize some of them |
There was a problem hiding this comment.
I think you want https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-rank-feature-query.html
We need a PageRank :)
There was a problem hiding this comment.
Seriously though, we could implement a very simple rank based on which guide is referenced by others. The Hibernate ORM guide would automatically get a higher rank than e.g. Panache or Hibernate Search, and that's what we want.
| // to have some sort of weight in the documents and prioritize some of them | ||
| // problem will be to find the right balance because the weight would be always on | ||
| // another option could be to use the keyword to trick some searches | ||
| // I suspect we should make the keywords a List separated by , and a proper list in the index rather than what it is currently |
There was a problem hiding this comment.
Not sure we can expect all documents to reliably use the proper syntax...
Regardless, in the full-text index you'll get the exact same result, whether you split the list yourself or you let the tokenizer do it. So unless you need spaces or punctutation in keywords, or you need faceting, I don't see what this would bring.
| } | ||
|
|
||
| return f.bool().must(f.simpleQueryString() | ||
| .field("title").boost(10.0f) |
There was a problem hiding this comment.
I thought topics/keywords were things we wanted to match in priority, that's why the title boost was lower.
If you want to change this, you should probably change it for title_autocomplete as well?
There was a problem hiding this comment.
Keep in mind that we had a very naive solution for search until now. So keywords was basically: "let's add some additional words that are not in the title or preamble", and nothing more.
We could change the definition but that's what it is atm.
As for topics, it might make sense to boost them more in the future but I would suggest to do the tweaking when all the guides are indexed and we have a proper idea of what we want (and also maybe once the PageRank thing is in).
yrodiere
force-pushed
the
index-topics-extensions
branch
from
593ad82 to
9b9535b
Compare
Also adjust the scoring a bit: - keywords should be at the same level of the title as they are just additional keywords on top of the title - when topics contains compatibility, lower the score of the document Adjust the dataset a bit to be more representative and also add a few explanations about why the documents are popping.
(Even if the changes seem a bit counterproductive, we will need a better way to adjust the Hibernate Reactive ranking)
yrodiere
force-pushed
the
index-topics-extensions
branch
from
9b9535b to
b22fcae
Compare
No description provided.