Use an AlternativeValueBridge for localized fields V2

[marko-bekhta]

fixes #33

soooo here's an alternative using alternative 😃 value bridge.

[yrodiere]

Looks simpler than the previous PR to me... WDYT?

[marko-bekhta]

Some more progress on this and a couple of findings so far:

• For ES site:
  • https://github.com/quarkusio/es.quarkus.io/tree/master/l10n/po/es_ES/_data missing any translation files for quarkus metadata, and that breaks the general pattern compared to other localized sites
  • uses a master instead of main as a branch for sources, we probably can create some exception in the code and work around the difference but would've been much easier if the project gets updated.
• For CN site:
  • https://github.com/quarkusio/cn.quarkus.io/tree/main/l10n/po/zh_CN/_data/versioned missing a few versions
  • the main version is missing a quarkiverse translations file
• org.fedorahosted.tennera:jgettext fails in native mode somewhere on antlr 😔 with null pointers, didn't spend to much time looking into it for now
• started looking into configuring analyzers differently for different languages,
  • found this article on Japanese analyzer where they are suggesting adding plugins https://www.elastic.co/blog/how-to-implement-japanese-full-text-search-in-elasticsearch; which makes me wonder if we can do that easily on our app instance or not, or should we limit things to simple n-gram?
  • for other languages like Spanish or Portuguese it seems that the main difference is in adding the language to the stopwords filter or to the stemmer (https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-lang-analyzer.html#portuguese-analyzer) I suppose we can try that, but then since these aren't actually translated, should we even bother?
• Still need to update the part where we read quarkiverse data, since for now it remains untranslated...

[yrodiere]

For ES site:
uses a master instead of main as a branch for sources, we probably can create some exception in the code and work around the difference but would've been much easier if the project gets updated.

Hey @rdnovell, do you think you could rename the master branch of https://github.com/quarkusio/es.quarkus.io to main? I think that's the recommended practice in the Quarkus org nowadays, and it would help us avoid an edge case here.

[yrodiere]

For ES site:
https://github.com/quarkusio/es.quarkus.io/tree/master/l10n/po/es_ES/_data missing any translation files for quarkus metadata, and that breaks the general pattern compared to other localized sites

For CN site:
https://github.com/quarkusio/cn.quarkus.io/tree/main/l10n/po/zh_CN/_data/versioned missing a few versions
the main version is missing a quarkiverse translations file

I think you'll have to make sure we're resilient to such problems, and use the english version as a fallback. Might be worth adding notes about these in #64

org.fedorahosted.tennera:jgettext fails in native mode somewhere on antlr 😔 with null pointers, didn't spend to much time looking into it for now

I'll disable native tests. We don't deploy in native mode anyway, because of a different bug in jgit.

EDIT: See #72

found this article on Japanese analyzer where they are suggesting adding plugins https://www.elastic.co/blog/how-to-implement-japanese-full-text-search-in-elasticsearch; which makes me wonder if we can do that easily on our app instance or not, or should we limit things to simple n-gram?

In prod, that's doable: we already alter the container image before running it.

In dev mode, I'm less sure, as I don't know if dev services can use dockerfiles.

We use OpenSearch, though. So maybe the plugins come pre-installed. Yeah I dream sometimes.

for other languages like Spanish or Portuguese it seems that the main difference is in adding the language to the stopwords filter or to the stemmer (https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-lang-analyzer.html#portuguese-analyzer) I suppose we can try that, but then since these aren't actually translated, should we even bother?

Probably better for the es/pt analyzers to assume english for now, if the guide title/summary/content are in English.

Still need to update the part where we read quarkiverse data, since for now it remains untranslated...

You can probably assume it won't get translated. At least not in the near future.

[marko-bekhta]

For Japanese:

q: Getting Started with Security :

{
      "url": "https://ja.quarkus.io/guides/security-getting-started-tutorial",
      "type": "tutorial",
      "origin": "quarkus",
      "title": "Basic認証とJakarta Persistenceを使ったセキュリティ入門",
      "summary": "組み込みのQuarkus Basic認証とJakarta Persistence IDプロバイダーを使用してQuarkusアプリケーションのエンドポイントを保護し、ロールベースのアクセス制御を有効にすることで、Quarkus <span class=\"highlighted\">Security</span>を開始できます。",
      "content": [
        "<span class=\"highlighted\">Getting</span> <span class=\"highlighted\">Started</span> with <span class=\"highlighted\">Security</span> using Basic authentication and Jakarta Persistence <span class=\"highlighted\">Get</span> <span class=\"highlighted\">started</span> with Quarkus <span class=\"highlighted\">Security</span> by <span class=\"highlighted\">securing</span>",
        "Use Curl or a browser to test your application 下記の例のように、PostgreSQL サーバーを起動します: docker run --rm=true --name <span class=\"highlighted\">security</span>-<span class=\"highlighted\">getting</span>-<span class=\"highlighted\">started</span>"
      ]
    }

q: ブロッキング:

{
      "url": "https://ja.quarkus.io/guides/hibernate-reactive",
      "type": "guide",
      "origin": "quarkus",
      "title": "Hibernate Reactiveの使用",
      "summary": "Hibernate Reactiveは、Hibernate ORMのリアクティブAPIで、ノン<span class=\"highlighted\">ブロッキング</span>データベースドライバとデータベースとのリアクティブスタイルのインタラクションをサポートしています。",
      "content": [
        "Hibernate Reactiveの使用 Hibernate Reactiveは、Hibernate ORMのリアクティブAPIで、ノン<span class=\"highlighted\">ブロッキング</span>のデータベースドライバと、データベースとのインタラクションのリアクティブなスタイルをサポート"
      ]
    }

so it looks like this mix works, but would be better if someone with language knowledge looks at the results 🙈 😃

[marko-bekhta]

I'll pause on pushing more to this PR 🙈

[yrodiere]

A few comments on the latest version of this; I don't see any major problems.

We might need to handle the master branch name for es in order to expedite things, as I didn't get an answer... Would that be complex? It'll need to be handled in QuarkusIOSample too, I suppose. Maybe put this info in application.properties and load it from QuarkusIOSample too (along with .env?

Also... good luck rebasing this :D

[rdnovell]

Some more progress on this and a couple of findings so far:

• For ES site:

  • https://github.com/quarkusio/es.quarkus.io/tree/master/l10n/po/es_ES/_data missing any translation files for quarkus metadata, and that breaks the general pattern compared to other localized sites
  • uses a master instead of main as a branch for sources, we probably can create some exception in the code and work around the difference but would've been much easier if the project gets updated.

• For CN site:

  • https://github.com/quarkusio/cn.quarkus.io/tree/main/l10n/po/zh_CN/_data/versioned missing a few versions
  • the main version is missing a quarkiverse translations file

• org.fedorahosted.tennera:jgettext fails in native mode somewhere on antlr 😔 with null pointers, didn't spend to much time looking into it for now

• started looking into configuring analyzers differently for different languages,

  • found this article on Japanese analyzer where they are suggesting adding plugins https://www.elastic.co/blog/how-to-implement-japanese-full-text-search-in-elasticsearch; which makes me wonder if we can do that easily on our app instance or not, or should we limit things to simple n-gram?
  • for other languages like Spanish or Portuguese it seems that the main difference is in adding the language to the stopwords filter or to the stemmer (https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-lang-analyzer.html#portuguese-analyzer) I suppose we can try that, but then since these aren't actually translated, should we even bother?

• Still need to update the part where we read quarkiverse data, since for now it remains untranslated...

Hello guys, I'm working in renaming the master branch to main, I was in PTO sorry for the delay.

[yrodiere]

Hello guys, I'm working in renaming the master branch to main, I was in PTO sorry for the delay.

No worries, thanks!

[yrodiere]

LGTM; @marko-bekhta please merge when you're sure it will work in prod (branches renamed, etc.)

[marko-bekhta]

I've rebased it and tried to regenerate the samples as @rdnovell already created the main branch in the es repository. All worked fine, thanks! Let's wait a bit so that the branch transition is completed (still need to set it to be a default) and then merge.

[ynojima]

For ES site:
https://github.com/quarkusio/es.quarkus.io/tree/master/l10n/po/es_ES/_data missing any translation files for quarkus metadata, and that breaks the general pattern compared to other localized sites
uses a master instead of main as a branch for sources, we probably can create some exception in the code and work around the difference but would've been much easier if the project gets updated.
For CN site:
https://github.com/quarkusio/cn.quarkus.io/tree/main/l10n/po/zh_CN/_data/versioned missing a few versions
the main version is missing a quarkiverse translations file

Oh, sorry for differences between these languages.
I'll align es.quarkus.io and cn.quarkus.io to ja.quarkus.io (pt.quarkus.io is already aligned. These sites can be aligned at this extent.)

[yrodiere]

Merging as per #86 (comment)

[yrodiere]

Thanks for your work Marko :)

We'll see if indexing succeeds on OpenShift first; once that works, we'll need @rdnovell / @ynojima 's help to determine the quality of the search :) We'll ping you again!

[ynojima]

Now es.quarkus.io and cn.quarkus.io repositories have main branches and their structures are aligned with pt.quarkus.io and ja.quarkus.io repositories.

Oh, sorry for differences between these languages.
I'll align es.quarkus.io and cn.quarkus.io to ja.quarkus.io (pt.quarkus.io is already aligned. These sites can be aligned at this extent.)

Originally posted by @ynojima in #71 (comment)

If there is anything I can help, please let me know.