mirrors
/
elasticsearch
mirror of https://mirror.osredm.com/root/elasticsearch.git
2
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

ESQL: Add more details on ENRICH vs. LOOKUP JOIN to docs (#125487) 

* Add more details on ENRICH vs. LOOKUP JOIN
* Move example, fix syntax formatting
This commit is contained in:
Alexander Spies 2025-03-24 16:26:28 +01:00 committed by GitHub
parent f83ca0c6b7
commit f8536aadda
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
3 changed files with 17 additions and 15 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

16
docs/reference/query-languages/esql/esql-commands.md
View File

@ -677,17 +677,11 @@ This functionality is in technical preview and may be changed or removed in a fu
**Syntax**
```
```esql
FROM <source_index>
| LOOKUP JOIN <lookup_index> ON <field_name>
```
```esql
FROM firewall_logs
| LOOKUP JOIN threat_list ON source.IP
| WHERE threat_level IS NOT NULL
```
**Parameters**
`<lookup_index>`
@ -717,6 +711,14 @@ FROM firewall_logs
| LOOKUP JOIN threat_list ON source.IP
```
To filter only for those rows that have a matching `threat_list` entry, use `WHERE ... IS NOT NULL` with a field from the lookup index:
```esql
FROM firewall_logs
| LOOKUP JOIN threat_list ON source.IP
| WHERE threat_level IS NOT NULL
```
**Host metadata correlation**: This query pulls in environment or ownership details for each host to correlate with your metrics data.
```esql

4
docs/reference/query-languages/esql/esql-enrich-data.md
View File

@ -19,10 +19,10 @@ For example, you can use `ENRICH` to:
* Enrichment data doesn't change frequently
* You can accept index-time overhead
* You are working with structured enrichment patterns
* You can accept having multiple matches combined into multi-values
* You can accept being limited to predefined match fields
* `ENRICH` has a simplified security model. There are no restirctions to specific enrich policies or document and field level security.
* You do not need fine-grained security: There are no restrictions to specific enrich policies or document and field level security.
* You want to match using ranges or spatial relations
### How the `ENRICH` command works [esql-how-enrich-works]

12
docs/reference/query-languages/esql/esql-lookup-join.md
View File

@ -20,11 +20,11 @@ For example, you can use `LOOKUP JOIN` to:
* Your enrichment data changes frequently
* You want to avoid index-time processing
* You're working with regular indices
* You need to preserve distinct matches
* You want SQL-like behavior, so that multiple matches result in multiple rows
* You need to match on any field in a lookup index
* You use document or field level security
* You want to restrict users to a specific lookup indices that they can you
* You want to restrict users to use only specific lookup indices
* You do not need to match using ranges or spatial relations
## How the `LOOKUP JOIN` command works [esql-how-lookup-join-works]
@ -108,7 +108,7 @@ FROM employees
To use `LOOKUP JOIN`, the following requirements must be met:
* **Compatible data types**: The join key and join field in the lookup index must have compatible data types. This means:
* The data types must either be identical or be internally represented as the same type in Elasticsearch's type system
* The data types must either be identical or be internally represented as the same type in {esql}
* Numeric types follow these compatibility rules:
* `short` and `byte` are compatible with `integer` (all represented as `int`)
* `float`, `half_float`, and `scaled_float` are compatible with `double` (all represented as `double`)
@ -120,9 +120,9 @@ For a complete list of supported data types and their internal representations,
The following are the current limitations with `LOOKUP JOIN`
* `LOOKUP JOIN` will be successful if the join field in the lookup index is a `KEYWORD` type. If the main index's join field is `TEXT` type, it must have an exact `.keyword` subfield that can be matched with the lookup index's `KEYWORD` field.
* Indices in [lookup](/reference/elasticsearch/index-settings/index-modules.md#index-mode-setting) mode are always single-sharded.
* Cross cluster search is unsupported. Both source and lookup indices must be local.
* Currently, only matching on equality is supported.
* `LOOKUP JOIN` can only use a single match field and a single index. Wildcards, aliases, datemath, and datastreams are not supported.
* The name of the match field in `LOOKUP JOIN lu_idx ON match_field` must match an existing field in the query. This may require renames or evals to achieve.
* The name of the match field in `LOOKUP JOIN lu_idx ON match_field` must match an existing field in the query. This may require `RENAME`s or `EVAL`s to achieve.
* The query will circuit break if there are too many matching documents in the lookup index, or if the documents are too large. More precisely, `LOOKUP JOIN` works in batches of, normally, about 10,000 rows; a large amount of heap space is needed if the matching documents from the lookup index for a batch are multiple megabytes or larger. This is roughly the same as for `ENRICH`.