Commit Graph

86883 Commits

Author SHA1 Message Date
Patrick Doyle b282b6e937
Revert changes to Text class (#128483)
* Revert "Fix the Text class package change in example plugins (#128316)"

This reverts commit cc486480e3.

* Revert "Update Text class to use native java ByteBuffer (#127666)"

This reverts commit db0c3c7a28.
2025-05-26 13:51:22 -07:00
Pablo 8929a64343
Add sum_over_time aggregation (#128413)
* Drafting SumOverTime transform for timeseries

* Fix test cases after figuring out how to run them
2025-05-26 10:32:45 -07:00
Martijn van Groningen c1df652256
CompositeSyntheticFieldLoader tweak (#128461)
SourceLoader.SyntheticFieldLoader#write(...) is hot code, avoid using streams api and use loop and compute sum.
2025-05-26 17:35:33 +02:00
Ignacio Vera b89b86d0c3
Adjust generated pushdown query for IndexOrDocValuesQuery (#128434) 2025-05-26 17:28:38 +02:00
Patrick Doyle 940bea79c3
Allow for mark == bytes.length. (#128340)
In the case that the mark is at the very end of the stream,
the slice's stream could already be closed by the time we
call reset(), and so the stream must be reopened.
2025-05-26 10:08:33 -04:00
Ignacio Vera 444d1486b7
Adjust generated query in MultiMatchQueryBuilderTests (#128436) 2025-05-26 14:54:44 +02:00
Martijn van Groningen 7e269a92a8
Remove tsdb_doc_values_optimized_merge feature flag. (#128435)
All the development work has been completed in #126111
and no regressions have been observer in nightly benchmarks
2025-05-26 13:57:05 +02:00
Fabrizio Ferri-Benedetti 314dc1a20e
Replace chars (#128439) 2025-05-26 11:01:12 +02:00
Martijn van Groningen 2fabf6ae60
Revert "Temporarily disable use_default_lucene_postings_format feature flag. …" (#128433)
This reverts commit 6f2d4c816a.
2025-05-26 10:04:43 +02:00
Johannes Fredén 74d025eda8
Fix FileSettingsServiceIT race condition (#128374)
Same issue as
https://github.com/elastic/elasticsearch-serverless/pull/3900 

Instead of using handler keys as condition that reserved state has been
updated, use the version. 

Resolves: https://github.com/elastic/elasticsearch/issues/128369
2025-05-26 17:40:35 +10:00
Ioana Tagirta 8554e19d89
Fix ForkIT.testProfile test (#128398) 2025-05-26 09:14:17 +02:00
Nhat Nguyen f76e201774
Increase concurrency for TS command (#128419)
Today, with `FROM`, we can partition a shard into multiple slices, 
allowing multiple drivers to execute against a single shard.

For time-series, specifically rate aggregation, the data must arrive in 
order; therefore, we currently use a single driver. With this change, we
split the pipeline into two parts—the time-series source and the
rest—allowing two drivers to run concurrently.
2025-05-25 19:58:51 -07:00
Fang Xing dfe1357e26
[ES|QL] Allow lookup join on mixed numeric fields (#128263)
* allow lookup join on mixed numeric fields
2025-05-25 14:56:07 -04:00
Craig Taverner 844ee68f0f
Added geometry validation for GEO types to exit early on invalid latitudes (#128259)
It turns out in #128234 that Lucene pushdown on ST_DISTANCE with invalid points (latitude out of range) will cause all documents to be returned, since DISJOINT on an invalid circle is true for all documents. We could either add an extra check for spatial pushdown that the geometries are valid, or add validation at geometry creation. This second option is much easier to implement, and a more comprehensive approach, as it prevents invalid geometries in many more places, hopefully reducing the likelihood of subtle and obscure bugs like #128234 happening in future.
2025-05-25 11:34:28 +02:00
elasticsearchmachine d815e28c16 Mute org.elasticsearch.xpack.esql.action.CrossClusterQueryWithFiltersIT testTimestampFilterFromQuery #127332 2025-05-25 16:18:12 +10:00
elasticsearchmachine 431a6654bd Mute org.elasticsearch.xpack.esql.action.ForkIT testProfile #128377 2025-05-25 08:08:50 +10:00
elasticsearchmachine 05cd728e7e Mute org.elasticsearch.xpack.search.CrossClusterAsyncSearchIT testCCSClusterDetailsWhereAllShardsSkippedInCanMatch #128418 2025-05-24 16:08:05 +10:00
elasticsearchmachine 6c7d43ce98 Mute org.elasticsearch.xpack.ml.integration.InferenceIngestIT testPipelineIngestWithModelAliases #128417 2025-05-24 15:47:04 +10:00
elasticsearchmachine a8739a755f Mute org.elasticsearch.index.query.MultiMatchQueryBuilderTests testToQuery #128416 2025-05-24 15:26:03 +10:00
elasticsearchmachine 80a41a7c99 Mute org.elasticsearch.xpack.esql.qa.single_node.PushQueriesIT testEqualityAndOther {semantic_text} #128414 2025-05-24 09:34:13 +10:00
Parker Timmins 51e87cb744
Use new source loader when lower docId is accessed (#128320)
When using synthetic source, runtime fields data may come from doc values. Doc values iterators can only be read once, and in increasing docId order. But if a runtime field is referenced multiple times in a query, currently the same doc value iterator will be used. This causes an error, as the second field reference will attempt to read the same iterator from a lower docId than was previously used. The fix is to create a new source loader, and thus a new doc value iterator, if the requested docId is lower than the last seen docId.
2025-05-23 15:57:15 -05:00
Patrick Doyle 13a0e97e9f
Improve TestBuildInfoPluginFuncTest (#128322)
* Use walkFileTree for extractModuleNameFromDirectory

* More tests in TestBuildInfoPluginFuncTest

* Remove stray line from debugging

* [CI] Auto commit changes from spotless

* Eliminate List.reversed() call

---------

Co-authored-by: elasticsearchmachine <infra-root+elasticsearchmachine@elastic.co>
2025-05-23 22:28:57 +02:00
Mark J. Hoy 2310bcada2
Remove Enterprise Search Service Account (#124655)
* remove Enterprise Search service account + tests

* [CI] Auto commit changes from spotless

* remove all tests referencing ent-search

* [CI] Auto commit changes from spotless

* ensure yaml tests have correct length

* set proper length expected

* use index for slow log test that works

* skip account REST tests with invalid length

* skip Test service account tokens test too

* remove added space

* add changelog for PR

* update

---------

Co-authored-by: elasticsearchmachine <infra-root+elasticsearchmachine@elastic.co>
2025-05-23 13:53:20 -04:00
Valeriy Khakhutskyy c822a575ff
[ML] Prevent retention classes from failing when deleting documents in read-only indices (#125408)
Classes like UnusedStatsRemover delete orphaned documents without an associated job. When the indices are made read-only it will start failing as read-only means no delete.

This PR ensures that the non-writable indices are not included in the delete-by-query requests.
2025-05-23 19:49:55 +02:00
Ioana Tagirta 55956d4029
Docs: Fix match_phrase docs for zero_terms_query (#128387) 2025-05-23 19:33:26 +02:00
Nik Everett 584c60e233
ESQL: Enable docs for ROUND_TO (#128378)
Plug the docs in for ROUND_TO.
2025-05-23 19:06:05 +02:00
Keith Massey 7207692056
Adding dry_run mode for setting data stream settings (#128269) 2025-05-23 11:29:00 -05:00
Nik Everett dd180be55d
ESQL: Fix docs for ROUND_TO (#128382)
The examples included a filter we use for testing by mistake.
2025-05-24 01:28:29 +10:00
David Kyle 5b3a4f3b14
[ML] Yaml test that runs inference as a non-admin user (#128363) 2025-05-23 16:03:04 +01:00
Ignacio Vera de7c91c1d9
Use IndexOrDocValuesQuery in NumberFieldType#termQuery implementations (#128293) 2025-05-23 16:58:50 +02:00
Brian Seeders a13a46a2da
[tests] StandaloneRestIntegTestTask should set a minimum maxParallelForks of 1 (#126346) 2025-05-23 10:42:34 -04:00
Brian Seeders 00c3146dcf
[CI] Add windows-2025 to windows testing (#127850) 2025-05-23 10:25:49 -04:00
Pat Whelan c96a99a97a
[ML] Pass timeout to chat completion (#128338) 2025-05-23 16:16:02 +02:00
Nik Everett 45bfaab448
ESQL: ROUND_TO function (#128278)
Creates a `ROUND_TO` function that rounds it's input to one of the
provided values. Like so:
```
ROUND_TO(v, 0, 5000, 10000, 20000, 40000, 100000)

   v   | ROUND_TO
     0 | 0
   100 | 0
  6000 | 5000
 45001 | 40000
999999 | 100000
```

For some sequences of numbers you could do this with the `/` operator -
but for arbitrary sequences of numbers you needed `CASE` which is quite
slow. And hard to read!

Rewriting the example above would look like:
```
CASE (
  v <   5000,     0,
  v <  10000,  5000,
  v <  20000, 10000,
  v <  40000, 20000,
  v < 100000, 40000,
  100000
)
```

Even better, this is *fast*:
```
        (operation)  Mode  Cnt    Score   Error  Units
round_to_4_via_case  avgt    7  138.124 ± 0.738  ns/op
         round_to_4  avgt    7    0.805 ± 0.011  ns/op
         round_to_3  avgt    7    0.739 ± 0.011  ns/op
         round_to_2  avgt    7    0.651 ± 0.009  ns/op
         date_trunc  avgt    7    2.425 ± 0.018  ns/op
```

I've included a comparison to `DATE_TRUNC` above because we should be
able to rewrite `DATE_TRUNC` into `ROUND_TO` when we know the date range
of the index. This doesn't do it now, but it should be possible.
2025-05-23 10:14:30 -04:00
Iván Cea Fontenla 0c3d47de10
ESQL: Fix validation NPE in Enrich and add extra @Nullable annotations (#128260)
Fixes https://github.com/elastic/elasticsearch/issues/126253
Fixes https://github.com/elastic/elasticsearch/issues/126297

`inputDataType` may be null when in mixed cluster (<8.14). So `validateTypes()` should take that into account. Similar fix to https://github.com/elastic/elasticsearch/pull/116583
2025-05-23 16:05:57 +02:00
Patrick Doyle 7e4d96e1d8
Platform-independent paths in TestBuildInfoPluginFuncTest (#128303) 2025-05-23 09:46:18 -04:00
elasticsearchmachine de25987f15 Mute org.elasticsearch.reservedstate.service.FileSettingsServiceIT testSymlinkUpdateTriggerReload #128369 2025-05-23 23:31:51 +10:00
Krishna Chaitanya Reddy Burri 169527ff6c
Add aws.config source indices to kibana_system role permissions (#128350)
Adding `logs-aws.config-*` data stream indices to the `kibana_system` privileges. 
This is required for the latest transform to work.

Related: 
- https://github.com/elastic/integrations/pull/13830#discussion_r2100362326
2025-05-23 18:17:28 +05:30
Jan Calanog cae06b8c54
Fix code-callout rendering (#128365) 2025-05-23 13:42:04 +02:00
Lorenzo Dematté e88bcaf4da
Better handling of node ids from shutdown metadata (#128298)
Currently, the DiskThresholdMonitor code considers shutdown metadata to identify nodes that are being replaced. If the node-to-be-replaced (source) leaves the cluster before the corresponding shutdown metadata is removed from the cluster state, we can have a NPE. This PR adds a test for that and improves a bit code to handle node ids for source and target from shutdown metadata.

Fixes #100201
2025-05-23 12:47:05 +02:00
Carles Salvador b977b04a2e
Rename Azure Active Directory to Microsoft Entra ID (#128351)
* Rename Azure Active Directory to Microsoft Entra ID 

Based on Microsoft change: https://learn.microsoft.com/en-us/entra/fundamentals/new-name

* Update docs/reference/search-connectors/es-connectors-sharepoint-online.md

Co-authored-by: Liam Thompson <32779855+leemthompo@users.noreply.github.com>

* Update docs/reference/search-connectors/es-connectors-sharepoint-online.md

Co-authored-by: Liam Thompson <32779855+leemthompo@users.noreply.github.com>

* Update docs/reference/search-connectors/es-connectors-sharepoint-online.md

Co-authored-by: Liam Thompson <32779855+leemthompo@users.noreply.github.com>

---------

Co-authored-by: Liam Thompson <32779855+leemthompo@users.noreply.github.com>
2025-05-23 11:40:01 +02:00
Brian Seeders d7aebb8d17
[release-notes] Add release-notes for 9.0.2 (#128319) 2025-05-23 11:14:04 +02:00
Ignacio Vera 8085eea9fc
Change exceptions to IllegalArgumentException in GroupingDocValuesSelector (#128348) 2025-05-23 10:45:32 +02:00
Ioana Tagirta 6aed71f945
Add profile description labels for FORK sub plans (#128318) 2025-05-23 10:42:09 +02:00
Johannes Fredén cc5aa91119
Ensure config reload on ..data symlink switch for CSI driver support (#127628)
* Reprocess all changed files in settings dir if symlink dir is updated
2025-05-23 10:40:37 +02:00
Lorenzo Dematté 6bf531660c
Fix NPE in APMTracer through RestController (#128314)
Our APMTracer doesn't like nulls - this is a sensible thing, as APM in general does not allow nulls (it only allows a precise set of types).
This PR changes the attribute to a sentinel "" in place of null values. It also makes a small change to APMTracer to give a better error message in case of null values in attributes.
2025-05-23 09:32:22 +02:00
elasticsearchmachine aeafcd26e3 Mute org.elasticsearch.packaging.test.TemporaryDirectoryConfigTests test21AcceptsCustomPathInDocker #128114 2025-05-23 17:11:45 +10:00
Ievgen Degtiarenko 2f01d1715c
Fix thread leak in ManyShardsIT (#128321) 2025-05-23 08:28:49 +02:00
elasticsearchmachine 7c39f26f3d Mute org.elasticsearch.packaging.test.DockerTests test120DockerLogsIncludeElasticsearchLogs #128117 2025-05-23 09:08:45 +10:00
Nik Everett cf1f48e74b
ESQL: Use less data in test (#128337)
This test is expected to complete a deeply abusive lookup join without
circuit breaking. It was sometimes circuit breaking. This lowers the
data we feed to the test so its slightly less abusive. It's still plenty
abusive.

Closes #127365
2025-05-23 08:31:21 +10:00