[DOCS] Update ESQL metadata fields page (#129939)

* [DOCS] Update ESQL metadata fields page

**esql-metadata-fields.md:**
- restructured from bullet list to table format for metadata fields
- added `_index_mode` and `_source` fields to available metadata
- improved field descriptions (more detailed)
- added "usage and limitations" section
- reorganized examples into subsections with headers
- added `_score` sorting example
- added tip box linking to search documentation

* 🚙Drive by updates to search functions ref page

moved tutorial link into tip box at top
added cross-reference to search overview documentation
minor text flow improvements and punctuation fixes

* Fix id typo

* Apply suggestions from review

Co-authored-by: Bogdan Pintea <sig11@mailbox.org>
This commit is contained in:
Liam Thompson 2025-06-25 12:00:28 +02:00 committed by GitHub
parent 138f350840
commit 4df0e9930c
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
2 changed files with 51 additions and 15 deletions

View File

@ -6,24 +6,40 @@ mapped_pages:
# {{esql}} metadata fields [esql-metadata-fields]
{{esql}} can access [metadata fields](/reference/elasticsearch/mapping-reference/document-metadata-fields.md).
{{esql}} can access [metadata fields](/reference/elasticsearch/mapping-reference/document-metadata-fields.md). The currently supported ones are:
* [`_index`](/reference/elasticsearch/mapping-reference/mapping-index-field.md): the index to which the document belongs. The field is of the type [keyword](/reference/elasticsearch/mapping-reference/keyword.md).
* [`_id`](/reference/elasticsearch/mapping-reference/mapping-id-field.md): the source documents ID. The field is of the type [keyword](/reference/elasticsearch/mapping-reference/keyword.md).
* `_version`: the source documents version. The field is of the type [long](/reference/elasticsearch/mapping-reference/number.md).
* [`_ignored`](/reference/elasticsearch/mapping-reference/mapping-ignored-field.md): the ignored source document fields. The field is of the type [keyword](/reference/elasticsearch/mapping-reference/keyword.md).
* `_score`: when enabled, the final score assigned to each row matching an ES|QL query. Scoring will be updated when using [full text search functions](/reference/query-languages/esql/functions-operators/search-functions.md).
To enable the access to these fields, the [`FROM`](/reference/query-languages/esql/commands/source-commands.md#esql-from) source command needs to be provided with a dedicated directive:
To access these fields, use the `METADATA` directive with the [`FROM`](/reference/query-languages/esql/commands/source-commands.md#esql-from) source command. For example:
```esql
FROM index METADATA _index, _id
```
Metadata fields are only available if the source of the data is an index. Consequently, `FROM` is the only source commands that supports the `METADATA` directive.
## Available metadata fields
Once enabled, these fields will be available to subsequent processing commands, just like other index fields:
The following metadata fields are available in {{esql}}:
| Metadata field | Type | Description |
|---------------|------|-------------|
| [`_id`](/reference/elasticsearch/mapping-reference/mapping-id-field.md) | [keyword](/reference/elasticsearch/mapping-reference/keyword.md) | Unique document ID. |
| [`_ignored`](/reference/elasticsearch/mapping-reference/mapping-ignored-field.md) | [keyword](/reference/elasticsearch/mapping-reference/keyword.md) | Names every field in a document that was ignored when the document was indexed. |
| [`_index`](/reference/elasticsearch/mapping-reference/mapping-index-field.md) | [keyword](/reference/elasticsearch/mapping-reference/keyword.md) | Index name. |
| `_index_mode` | [keyword](/reference/elasticsearch/mapping-reference/keyword.md) | [Index mode](/reference/elasticsearch/index-settings/index-modules.md#index-mode-setting). For example: `standard`, `lookup`, or `logsdb`. |
| `_score` | [`float`](/reference/elasticsearch/mapping-reference/number.md) | Query relevance score (when enabled). Scores are updated when using [full text search functions](/reference/query-languages/esql/functions-operators/search-functions.md). |
| [`_source`](/reference/elasticsearch/mapping-reference/mapping-source-field.md) | Special `_source` type | Original JSON document body passed at index time (or a reconstructed version if [synthetic `_source`](/reference/elasticsearch/mapping-reference/mapping-source-field.md#synthetic-source) is enabled). |
| `_version` | [`long`](/reference/elasticsearch/mapping-reference/number.md) | Document version number |
## Usage and limitations
- Metadata fields are only available when the data source is an index
- The `_source` type is not supported by functions
- Only the `FROM` command supports the `METADATA` directive
- Once enabled, metadata fields work like regular index fields
## Examples
### Basic metadata usage
Once enabled, metadata fields are available to subsequent processing commands, just like other index fields:
```esql
FROM ul_logs, apps METADATA _index, _version
@ -40,6 +56,8 @@ FROM ul_logs, apps METADATA _index, _version
| 14 | apps | 1 | apps_14 |
| 14 | ul_logs | 1 | ul_logs_14 |
### Metadata fields and aggregations
Similar to index fields, once an aggregation is performed, a metadata field will no longer be accessible to subsequent commands, unless used as a grouping field:
```esql
@ -51,3 +69,15 @@ FROM employees METADATA _index, _id
| --- | --- |
| 10100 | employees |
### Sort results by search score
```esql
FROM products METADATA _score
| WHERE MATCH(description, "wireless headphones")
| SORT _score DESC
| KEEP name, description, _score
```
:::{tip}
Refer to [{{esql}} for search](docs-content://solutions/search/esql-for-search.md#esql-for-search-scoring) for more information on relevance scoring and how to use `_score` in your queries.
:::

View File

@ -6,13 +6,19 @@ mapped_pages:
# {{esql}} Search functions [esql-search-functions]
:::{tip}
Get started with {{esql}} for search use cases with
our [hands-on tutorial](docs-content://solutions/search/esql-search-tutorial.md).
For a high-level overview of search functionalities in {{esql}}, and to learn about relevance scoring, refer to [{{esql}} for search](docs-content://solutions/search/esql-for-search.md#esql-for-search-scoring).
:::
{{esql}} provides a set of functions for performing searching on text fields.
Use these functions
for [full-text search](docs-content://solutions/search/full-text.md)
and [semantic search](docs-content://solutions/search/semantic-search/semantic-search-semantic-text.md).
Get started with {{esql}} for search use cases with
our [hands-on tutorial](docs-content://solutions/search/esql-search-tutorial.md).
Full text functions can be used to
match [multivalued fields](/reference/query-languages/esql/esql-multivalued-fields.md).
A multivalued field that contains a value that matches a full text query is
@ -20,7 +26,7 @@ considered to match the query.
Full text functions are significantly more performant for text search use cases
on large data sets than using pattern matching or regular expressions with
`LIKE` or `RLIKE`
`LIKE` or `RLIKE`.
See [full text search limitations](/reference/query-languages/esql/limitations.md#esql-limitations-full-text-search)
for information on the limitations of full text search.