Skip to content

Commit

Permalink
docs update
Browse files Browse the repository at this point in the history
  • Loading branch information
@jillesvangurp
jillesvangurp committed Dec 3, 2024
1 parent c4ef2f0 commit c4d8c6e
Show file tree
Hide file tree
Showing 3 changed files with 105 additions and 59 deletions.
83 changes: 53 additions & 30 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -114,30 +114,34 @@ And then add dependencies to jvm targets:

### Create a client

First we create a client.
We start by creating a client:

```kotlin
val client = SearchClient()
```

Kotlin has default values for parameters. So, we use sensible defaults for the
`host` and `port` variables to connect to `localhost` and `9200`.
`host` and `port` parameters to connect to `localhost` and `9200`. But of
course you can modify those:

```kotlin
val client = SearchClient(
KtorRestClient(host="localhost", port=9200)
)
```

If you need ro, you can also configure multiple hosts,
If you need to, you can also configure multiple hosts,
add ssl and basic authentication to connect to managed Opensearch or Elasticsearch clusters. If you use
multiple hosts, you can also configure a strategy for selecting the host to connect to. For more on
this, read the [manual](https://jillesvangurp.github.io/kt-search/manual/GettingStarted.html).
multiple hosts, you can also configure a strategy for selecting the host to connect to. And of course
you can completely customize how the client connects.

For more on
this, read the [manual](https://jillesvangurp.github.io/kt-search/manual/GettingStarted.html).

### Documents and data classes

In Kotlin, the preferred way to deal with data would be a data class. This is a simple data class
that we will use below.
that we will use as an example below.

```kotlin
@Serializable
Expand All @@ -149,11 +153,21 @@ data class TestDocument(

In the example below we will use this `TestDocument`, which we can serialize using the
[kotlinx.serialization](https://github.com/Kotlin/kotlinx.serialization)
framework. You can also pass in your own serialized json in requests,
so if you want to use e.g. jackson or gson instead,
you can do so easily.

### Creating an index
framework.

**Note**, the client provides a lot of flexibility including to your choice of JSON serialization. The only
part of the code dependent on kotlinx serialization is the client module. But for example the Search DSL
and the other DSLs are not actually dependent on this.

Additionally, if you chooes to use the `IndexRepository`, it comes with a ModelSerialization strategy
that abstracts how to parse/serialize your model classes. A kotlinx serialization implementation is included
but that's easily swapped out for something else. So, if you need to use e.g.
jackson or gson instead, you can do so easily. However, kotlinx serialization is of course the only thing
that works on multi platform.

### Creating an index
Before we can query for `TestDocument` documents, we need to create an index and store some objects:

```kotlin
val indexName = "readme-index"
Expand All @@ -172,20 +186,22 @@ client.createIndex(indexName) {
}
```

This creates the index and uses the mappings and settings DSL. With this DSL, you can map fields,
This creates the index and uses the **mappings and settings DSL**. With this DSL, you can map fields,
configure analyzers, etc. This is optional of course; you can just call it without the block
and use the defaults and rely on dynamic mapping.
You can read more about that [here](https://jillesvangurp.github.io/kt-search/manual/IndexManagement.html)

### Adding documents

To fill the index with some content, we need to use bulk operations.
To fill the index with some documents, we need to use bulk indexing operations.

In kt-search this is made very easy with a **Bulk Indexing DSL** that completely abstracts away the book keeping
that you need to do for this in other clients.

In kt-search this is made very easy with a DSL that abstracts away the book keeping
that you need to do for this. The bulk block below creates a `BulkSession`, which does this for you and flushes
The bulk block below creates a `BulkSession`, which does this for you and flushes
operations to Elasticsearch. You can configure and tailor how this works via parameters
that have sensible defaults. For example the number of operations that is flushed is something
that you'd want to probably configure.
that you'd want to probably configure and error handling is something you can customize as well.

The optional `refresh` parameter uses WaitFor as the default. This means that after the block exits, the documents
will have been indexed and are available for searching.
Expand All @@ -194,7 +210,7 @@ will have been indexed and are available for searching.
client.bulk(
refresh = Refresh.WaitFor,
// send operations every 2 ops
// default would be 100
// default and more sensible would be 100
bulkSize = 2,
) {
index(
Expand Down Expand Up @@ -222,7 +238,7 @@ client.bulk(
```

You can read more about
[bulk operations](https://jillesvangurp.github.io/kt-search/manual/BulkIndexing.html) in the manual.
[bulk operations](https://jillesvangurp.github.io/kt-search/manual/BulkIndexing.html) and how to customize it in the manual.

### Search

Expand All @@ -231,10 +247,11 @@ Now that we have some documents in an index, we can do some queries:
```kotlin
// search for some fruit
val results = client.search(indexName) {
// `this` is a SearchDSL instance in the block
query = bool {
must(
// note how we can use property references here
term(TestDocument::tags, "fruit"),
term("tags", "fruit"),
// note how we can also use property references here
matchPhrasePrefix(TestDocument::name, "app")
)
}
Expand Down Expand Up @@ -263,7 +280,10 @@ doc apple
{"name":"apple","tags":["fruit"]}
```

You can also construct complex aggregations with the query DSL:
You can also construct complex aggregations with the query DSL.
Aggregation queries are one of the more complex topics in Elasticsearch
and we worked hard to make constructing these programmatically from
Kotlin as easy as possible.

```kotlin
val resp = client.search(indexName) {
Expand All @@ -275,7 +295,8 @@ val resp = client.search(indexName) {
minDocCount = 1
})
}
// picking the results apart is just as easy.
// Despite aggregations JSON being very complicated,
// kt-search makes picking the results apart easy.
resp.aggregations
.termsResult("by-tag")
.parsedBuckets.forEach { bucket ->
Expand All @@ -295,16 +316,18 @@ These examples show off a few nice features of this library:

- Kotlin DSLs are nice, type safe, and easier to read and write than pure Json. And of course
you get auto completion too. The client includes more DSLs for searching, creating indices and mappings, datastreams,
index life cycle management, bulk operations, aggregations, and more.
index life cycle management, bulk operations, aggregations, and more. All this builds on
[JsonDSL](https://github.com/jillesvangurp/json-dsl), which is a library we created for easily creating
Kotlin DSLs for existing JSON dialects.
- Where in JSON, you use a lot of String literals, kt-search actually allows you to use
property references or enum values as well. So, refactoring your data model doesn't
property references or enum values as well. If you use those, refactoring your data model doesn't
break your mappings and queries.
- Kt-search makes complicated features like bulk operations, aggregations, etc. really easy
to use and accessible. And there is also the IndexRepository, which makes it extremely easy
to use and accessible. And there is also the `IndexRepository`, which makes it extremely easy
to work with and query documents in a given index or data stream.
- While a DSL is nice to have, sometimes it just doesn't have the feature you
need or maybe you want to work with raw json string literal. Kt-search allows you to do both
and mix schema less with type safe kotlin. You can easily add custom
- While a Kotlin DSL is nice to have, sometimes it just doesn't have the feature you
need or maybe you want to work with raw json and use Kotlin's multiline string literals.
Kt-search allows you to do both and mix schema less with type safe kotlin. You can easily add custom
properties to the DSL via a simple `put`. All `JsonDsl` are actually mutable maps.
- Kt-search is designed to be [extensible](https://jillesvangurp.github.io/kt-search/manual/ExtendingTheDSL.html).
It's easy to use the built in features. And you can easily add your own features. This also
Expand Down Expand Up @@ -382,8 +405,8 @@ There are several libraries that build on kt-search:

Additionally, I also maintain a few other search related projects that you might find interesting.

- [Rankquest Studio](https://rankquest.jillesvangurp.com) - A user friendly tool that requires no installation process that helps you build and run test cases to measure search relevance for your search products. Rankquest Studio of course uses kt-search but it is also able to talk directly to your API and is designed to work with any kind of search api or product that is able to return lists of results.
- [querylight](https://github.com/jillesvangurp/querylight) - Sometimes Elasticsearch is just overkill. Query light is a tiny in memory search engine that you can embed in your kotlin browser, server, or mobile applications. We use it at FORMATION to support e.g. in app icon search. Querylight comes with its own analyzers and query language.
- [Rankquest Studio](https://rankquest.jillesvangurp.com) - A user friendly tool that requires no installation process that helps you build and run test cases to measure search relevance for your search products. Rankquest Studio of course uses kt-search but it is also able to talk directly to your search API and is designed to work with any kind of search api or product that is able to return lists of results.
- [querylight](https://github.com/jillesvangurp/querylight) - Sometimes Elasticsearch/Opensearch is just overkill. Query light is a tiny but capable in memory search engine that you can embed in your kotlin browser, server, or mobile applications. We use it at FORMATION to support e.g. in app icon search. Querylight comes with its own analyzers and query language.

## Setting up a development environment

Expand Down
77 changes: 50 additions & 27 deletions docs/src/test/kotlin/documentation/projectreadme/readme.kt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ val projectReadme = sourceGitRepository.md {
+"""
### Create a client
First we create a client.
We start by creating a client:
""".trimIndent()

// this is what we show in the Readme :-)
Expand All @@ -53,7 +53,8 @@ val projectReadme = sourceGitRepository.md {

+"""
Kotlin has default values for parameters. So, we use sensible defaults for the
`host` and `port` variables to connect to `localhost` and `9200`.
`host` and `port` parameters to connect to `localhost` and `9200`. But of
course you can modify those:
""".trimIndent()
example {
val client = SearchClient(
Expand All @@ -62,17 +63,20 @@ val projectReadme = sourceGitRepository.md {
}

+"""
If you need ro, you can also configure multiple hosts,
If you need to, you can also configure multiple hosts,
add ssl and basic authentication to connect to managed Opensearch or Elasticsearch clusters. If you use
multiple hosts, you can also configure a strategy for selecting the host to connect to. For more on
this, read the [manual](${ManualPages.GettingStarted.publicLink}).
multiple hosts, you can also configure a strategy for selecting the host to connect to. And of course
you can completely customize how the client connects.
For more on
this, read the [manual](${ManualPages.GettingStarted.publicLink}).
""".trimIndent()

+"""
### Documents and data classes
In Kotlin, the preferred way to deal with data would be a data class. This is a simple data class
that we will use below.
that we will use as an example below.
""".trimIndent()

example {
Expand All @@ -86,11 +90,21 @@ val projectReadme = sourceGitRepository.md {
+"""
In the example below we will use this `TestDocument`, which we can serialize using the
[kotlinx.serialization](https://github.com/Kotlin/kotlinx.serialization)
framework. You can also pass in your own serialized json in requests,
so if you want to use e.g. jackson or gson instead,
you can do so easily.
framework.
**Note**, the client provides a lot of flexibility including to your choice of JSON serialization. The only
part of the code dependent on kotlinx serialization is the client module. But for example the Search DSL
and the other DSLs are not actually dependent on this.
Additionally, if you chooes to use the `IndexRepository`, it comes with a ModelSerialization strategy
that abstracts how to parse/serialize your model classes. A kotlinx serialization implementation is included
but that's easily swapped out for something else. So, if you need to use e.g.
jackson or gson instead, you can do so easily. However, kotlinx serialization is of course the only thing
that works on multi platform.
### Creating an index
### Creating an index
Before we can query for `TestDocument` documents, we need to create an index and store some objects:
""".trimIndent()
val indexName = "readme-index"
Expand All @@ -114,20 +128,22 @@ val projectReadme = sourceGitRepository.md {
}

+"""
This creates the index and uses the mappings and settings DSL. With this DSL, you can map fields,
This creates the index and uses the **mappings and settings DSL**. With this DSL, you can map fields,
configure analyzers, etc. This is optional of course; you can just call it without the block
and use the defaults and rely on dynamic mapping.
You can read more about that [here](${ManualPages.IndexManagement.publicLink})
### Adding documents
To fill the index with some content, we need to use bulk operations.
To fill the index with some documents, we need to use bulk indexing operations.
In kt-search this is made very easy with a **Bulk Indexing DSL** that completely abstracts away the book keeping
that you need to do for this in other clients.
In kt-search this is made very easy with a DSL that abstracts away the book keeping
that you need to do for this. The bulk block below creates a `BulkSession`, which does this for you and flushes
The bulk block below creates a `BulkSession`, which does this for you and flushes
operations to Elasticsearch. You can configure and tailor how this works via parameters
that have sensible defaults. For example the number of operations that is flushed is something
that you'd want to probably configure.
that you'd want to probably configure and error handling is something you can customize as well.
The optional `refresh` parameter uses WaitFor as the default. This means that after the block exits, the documents
will have been indexed and are available for searching.
Expand All @@ -136,7 +152,7 @@ val projectReadme = sourceGitRepository.md {
client.bulk(
refresh = Refresh.WaitFor,
// send operations every 2 ops
// default would be 100
// default and more sensible would be 100
bulkSize = 2,
) {
index(
Expand Down Expand Up @@ -164,7 +180,7 @@ val projectReadme = sourceGitRepository.md {
}
+"""
You can read more about
[bulk operations](${ManualPages.BulkIndexing.publicLink}) in the manual.
[bulk operations](${ManualPages.BulkIndexing.publicLink}) and how to customize it in the manual.
### Search
Expand All @@ -173,10 +189,11 @@ val projectReadme = sourceGitRepository.md {
example {
// search for some fruit
val results = client.search(indexName) {
// `this` is a SearchDSL instance in the block
query = bool {
must(
// note how we can use property references here
term(TestDocument::tags, "fruit"),
term("tags", "fruit"),
// note how we can also use property references here
matchPhrasePrefix(TestDocument::name, "app")
)
}
Expand All @@ -198,7 +215,10 @@ val projectReadme = sourceGitRepository.md {
}.printStdOut()

+"""
You can also construct complex aggregations with the query DSL:
You can also construct complex aggregations with the query DSL.
Aggregation queries are one of the more complex topics in Elasticsearch
and we worked hard to make constructing these programmatically from
Kotlin as easy as possible.
""".trimIndent()
example {
val resp = client.search(indexName) {
Expand All @@ -210,7 +230,8 @@ val projectReadme = sourceGitRepository.md {
minDocCount = 1
})
}
// picking the results apart is just as easy.
// Despite aggregations JSON being very complicated,
// kt-search makes picking the results apart easy.
resp.aggregations
.termsResult("by-tag")
.parsedBuckets.forEach { bucket ->
Expand All @@ -224,16 +245,18 @@ val projectReadme = sourceGitRepository.md {
- Kotlin DSLs are nice, type safe, and easier to read and write than pure Json. And of course
you get auto completion too. The client includes more DSLs for searching, creating indices and mappings, datastreams,
index life cycle management, bulk operations, aggregations, and more.
index life cycle management, bulk operations, aggregations, and more. All this builds on
[JsonDSL](https://github.com/jillesvangurp/json-dsl), which is a library we created for easily creating
Kotlin DSLs for existing JSON dialects.
- Where in JSON, you use a lot of String literals, kt-search actually allows you to use
property references or enum values as well. So, refactoring your data model doesn't
property references or enum values as well. If you use those, refactoring your data model doesn't
break your mappings and queries.
- Kt-search makes complicated features like bulk operations, aggregations, etc. really easy
to use and accessible. And there is also the IndexRepository, which makes it extremely easy
to use and accessible. And there is also the `IndexRepository`, which makes it extremely easy
to work with and query documents in a given index or data stream.
- While a DSL is nice to have, sometimes it just doesn't have the feature you
need or maybe you want to work with raw json string literal. Kt-search allows you to do both
and mix schema less with type safe kotlin. You can easily add custom
- While a Kotlin DSL is nice to have, sometimes it just doesn't have the feature you
need or maybe you want to work with raw json and use Kotlin's multiline string literals.
Kt-search allows you to do both and mix schema less with type safe kotlin. You can easily add custom
properties to the DSL via a simple `put`. All `JsonDsl` are actually mutable maps.
- Kt-search is designed to be [extensible](${ManualPages.ExtendingTheDSL.publicLink}).
It's easy to use the built in features. And you can easily add your own features. This also
Expand Down
Loading

0 comments on commit c4d8c6e

Please sign in to comment.