Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add non-English search support #2392

Open
wants to merge 6 commits into
base: master
Choose a base branch
from
Open

Add non-English search support #2392

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
30f9e71
Add non-English search support (for HTML renderer)
Sunshine40 May 20, 2024
a2ee372
Implement "fallback" searching strategy and remove dedicated Chinese …
Sunshine40 Jun 4, 2024
e6771d8
fix compiler error when `search-non-english` feature is enabled.
Sunshine40 Jun 5, 2024
af1aca2
Apply rustfmt
Sunshine40 Jun 5, 2024
0d53d6c
Fix unhandled exception when search results contains an item with an …
Sunshine40 Jun 8, 2024
e06cbc1
Avoid the experimental `Iterator.prototype.map()` API
Sunshine40 Jun 8, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
503 changes: 495 additions & 8 deletions Cargo.lock
View file
Open in desktop

Large diffs are not rendered by default.

4 changes: 4 additions & 0 deletions Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,10 @@ default = ["watch", "serve", "search"]
watch = ["dep:notify", "dep:notify-debouncer-mini", "dep:ignore", "dep:pathdiff", "dep:walkdir"]
serve = ["dep:futures-util", "dep:tokio", "dep:warp"]
search = ["dep:elasticlunr-rs", "dep:ammonia"]
search-non-english = ["search", "elasticlunr-rs/ar", "elasticlunr-rs/da", "elasticlunr-rs/de", "elasticlunr-rs/du",
"elasticlunr-rs/es", "elasticlunr-rs/fi", "elasticlunr-rs/fr", "elasticlunr-rs/hu", "elasticlunr-rs/it",
"elasticlunr-rs/ja", "elasticlunr-rs/ko", "elasticlunr-rs/no", "elasticlunr-rs/pt", "elasticlunr-rs/ro",
"elasticlunr-rs/ru", "elasticlunr-rs/sv", "elasticlunr-rs/tr"]

[[bin]]
doc = false
Expand Down
2 changes: 2 additions & 0 deletions guide/src/format/configuration/general.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,8 @@ This is general information about your book.
key in the configuration file.
- **language:** The main language of the book, which is used as a language attribute `<html lang="en">` for example.
This is also used to derive the direction of text (RTL, LTR) within the book.
When it is specified to a non-English language, an alternative indexing / searching strategy would be applied to the search functionality provided by the HTML renderer.
When `search-non-english` feature is enabled, additional language-specific search support may kick in.
- **text-direction**: The direction of text in the book: Left-to-right (LTR) or Right-to-left (RTL). Possible values: `ltr`, `rtl`.
When not specified, the text direction is derived from the book's `language` attribute.

Expand Down
3 changes: 2 additions & 1 deletion guide/src/format/configuration/renderers.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -263,7 +263,8 @@ copy-js = true # include Javascript code for search
- **enable:** Enables the search feature. Defaults to `true`.
- **limit-results:** The maximum number of search results. Defaults to `30`.
- **teaser-word-count:** The number of words used for a search result teaser.
Defaults to `30`.
When `book.language` is set to a non-English language, this limit might
be exceeded in case too many keywords are matched. Defaults to `30`.
- **use-boolean-and:** Define the logical link between multiple search words. If
true, all search words must appear in each result. Defaults to `false`.
- **boost-title:** Boost factor for the search result score if a search word
Expand Down
28 changes: 19 additions & 9 deletions src/renderer/html_handlebars/hbs_renderer.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -539,6 +539,25 @@ impl Renderer for HtmlHandlebars {
fs::create_dir_all(destination)
.with_context(|| "Unexpected error when constructing destination path")?;

// Render search index
#[cfg(feature = "search")]
{
let search = html_config.search.clone().unwrap_or_default();
if search.enable {
let language = match book_config.language.as_deref() {
None => Err("en".to_string()),
Some(language) => language.parse(),
};
#[allow(unused_variables)]
let extra_language_subtag =
super::search::create_files(&search, language, destination, book)?;
#[cfg(feature = "search-non-english")]
if let Some(subtag) = extra_language_subtag {
data.insert("lunr_language_subtag".to_owned(), json!(subtag));
}
}
}

let mut is_index = true;
for item in book.iter() {
let ctx = RenderItemContext {
Expand Down Expand Up @@ -589,15 +608,6 @@ impl Renderer for HtmlHandlebars {
self.copy_additional_css_and_js(&html_config, &ctx.root, destination)
.with_context(|| "Unable to copy across additional CSS and JS")?;

// Render search index
#[cfg(feature = "search")]
{
let search = html_config.search.unwrap_or_default();
if search.enable {
super::search::create_files(&search, destination, book)?;
}
}

self.emit_redirects(&ctx.destination, &handlebars, &html_config.redirect)
.context("Unable to emit redirects")?;

Expand Down
Loading
Loading