Add multi search helper (#1186)

This commit is contained in:
Tomas Della Vedova
2020-05-25 15:37:01 +02:00
committed by GitHub
parent 84217fc737
commit 1a25b623b0
6 changed files with 1102 additions and 19 deletions

View File

@ -7,6 +7,9 @@ CAUTION: The client helpers are experimental, and the API may change in the next
The helpers will not work in any Node.js version lower than 10.
=== Bulk Helper
~Added~ ~in~ ~`v7.7.0`~
Running Bulk requests can be complex due to the shape of the API, this helper aims to provide a nicer developer experience around the Bulk API.
==== Usage
@ -114,7 +117,7 @@ const b = client.helpers.bulk({
----
|`wait`
a|How much time to wait before retries in milliseconds.+
a|How much time to wait before retries in milliseconds. +
_Default:_ 5000.
[source,js]
----
@ -211,7 +214,123 @@ const result = await client.helpers.bulk({
console.log(result)
----
=== Multi Search Helper
~Added~ ~in~ ~`v7.8.0`~
If you are sending search request at a high rate, this helper might be useful for you.
It will use the mutli search API under the hood to batch the requests and improve the overall performances of your application. +
The `result` exposes a `documents` property as well, which allows you to access directly the hits sources.
==== Usage
[source,js]
----
const { Client } = require('@elastic/elasticsearch')
const client = new Client({ node: 'http://localhost:9200' })
const s = client.helpers.msearch()
// promise style API
s.search(
{ index: 'stackoverflow' },
{ query: { match: { title: 'javascript' } } }
)
.then(result => console.log(result.body)) // or result.documents
.catch(err => console.error(err))
// callback style API
s.search(
{ index: 'stackoverflow' },
{ query: { match: { title: 'ruby' } } },
(err, result) => {
if (err) console.error(err)
console.log(result.body)) // or result.documents
}
)
----
To create a new instance of the Msearch helper, you should access it as shown in the example above, the configuration options are:
[cols=2*]
|===
|`operations`
a|How many search operations should be sent in a single msearch request. +
_Default:_ `20`
[source,js]
----
const b = client.helpers.msearch({
operations: 10
})
----
|`concurrency`
a|How many request will be executed at the same time. +
_Default:_ `5`
[source,js]
----
const b = client.helpers.msearch({
concurrency: 10
})
----
|`retries`
a|How many times an operation will be retried before to resolve the request. An operation will be retried only in case of a 429 error. +
_Default:_ Client max retries.
[source,js]
----
const b = client.helpers.msearch({
retries: 3
})
----
|`wait`
a|How much time to wait before retries in milliseconds. +
_Default:_ 5000.
[source,js]
----
const b = client.helpers.msearch({
wait: 3000
})
----
|===
==== Stopping the Msearch Helper
If needed, you can stop a msearch processor at any time. The msearch helper returns a https://promisesaplus.com/[thenable], which has an `stop` method.
If you are creating multiple msearch helpers instances and using them for a limitied period of time, remember to always use the `stop` method once you have finished using them, otherwise your application will start leaking memory.
The `stop` method accepts an optional error, that will be dispatched every subsequent search request.
NOTE: The stop method will stop the execution of the msearch processor, but if you are using a concurrency higher than one, the operations that are already running will not be stopped.
[source,js]
----
const { Client } = require('@elastic/elasticsearch')
const client = new Client({ node: 'http://localhost:9200' })
const s = client.helpers.msearch()
s.search(
{ index: 'stackoverflow' },
{ query: { match: { title: 'javascript' } } }
)
.then(result => console.log(result.body))
.catch(err => console.error(err))
s.search(
{ index: 'stackoverflow' },
{ query: { match: { title: 'ruby' } } }
)
.then(result => console.log(result.body))
.catch(err => console.error(err))
setImmediate(() => s.stop())
----
=== Search Helper
~Added~ ~in~ ~`v7.7.0`~
A simple wrapper around the search API. Instead of returning the entire `result` object it will return only the search documents source.
For improving the performances, this helper automatically adds `filter_path=hits.hits._source` to the querystring.
@ -234,6 +353,9 @@ for (const doc of documents) {
----
=== Scroll Search Helper
~Added~ ~in~ ~`v7.7.0`~
This helpers offers a simple and intuitive way to use the scroll search API. Once called, it returns an https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/for-await...of[async iterator] which can be used in conjuction with a for-await...of. +
It handles automatically the `429` error and uses the client's `maxRetries` option.
@ -281,6 +403,8 @@ for await (const result of scrollSearch) {
=== Scroll Documents Helper
~Added~ ~in~ ~`v7.7.0`~
It works in the same way as the scroll search helper, but it returns only the documents instead. Note, every loop cycle will return you a single document, and you can't use the `clear` method.
For improving the performances, this helper automatically adds `filter_path=hits.hits._source` to the querystring.