TransportStartDatafeedAction previously tried to validate remote index cluster
names in datafeed jobs, before checking if the local cluster had
remote_cluster_client role. Because this role enables retrieval of the remote
cluster names, the validation step would always fail with a no-such-cluster
exception. This was confusing. This change moves the remote_cluster_client check
ahead of cluster name validation, and adds a test.
Closes ES-11841
Closes#121149
There is an issue where for Flattened fields with synthetic source, if there is a key with a scalar value, and a duplicate key with an object value, one of the values will be left out of the produced synthetic source. This fixes the issue by replacing the object with paths to each of its keys. These paths consist of the concatenation of all keys going down to a given scalar, joined by a period. For example, they are of the form foo.bar.baz. This applies recursively, so that every value within the object, no matter how nested, will be accessible through a full specified path.
* Making progress on different request parameters
* Working tests
* Adding custom service validator for rerank
* Fixing embedding bug
* Adding transport version check
* Fixing tests
* Fixing license header
* Fixing writeTo
* Moving file and removing commented code
* Fixing test
* Fixing tests
* Refactoring and tests
* Fixing test
This test has been muted for a long time and only seemed to fail on v7
compatability tests. We're unmuting this to see if it's still relevant.
Closes#112189
* Field infos calculation method inside Engine
* buildSeqNoStats as static public method
So it can be overriden in stateless if/as needed.
Relates ES-11457
* Update kibana system user permissions for cases analytics index.
* [CI] Auto commit changes from spotless
* Add permissions for aliases.
Create constants for indexes and aliases.
Update tests.
* [CI] Auto commit changes from spotless
---------
Co-authored-by: elasticsearchmachine <infra-root+elasticsearchmachine@elastic.co>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* add index privileges to support reroute processor
* [CI] Auto commit changes from spotless
* add more indices
* update
* fix test
* remove unnecessary indices
---------
Co-authored-by: elasticsearchmachine <infra-root+elasticsearchmachine@elastic.co>
This PR addresses ES-12071.
We want to collect metrics for the time that is spent waiting for the next chunk of a bulk request. This can help with diagnosing high bulk latency in case the latency is attributable to external factors such as network connection.
Co-authored-by: Francisco Fernández Castaño <francisco.fernandez.castano@gmail.com>
When migrating RepositoriesMetadata from cluster custom to project
custom (#125398), we needed temporary BWC handling for clusters running
on a version that is before this change but after the initial MP change.
Such a cluster can only exist in the serverless environment which has
progressed way past any applicable versions. Therefore we no longer need
the BWC handling and this PR removes it.
Relates: #125398
This PR addresses a bug where aborted merges are blocked if there's
insufficient disk space.
Previously, the merge disk space estimation did not consider if the
operation has been aborted when/while it was enqueued for execution.
Consequently, aborted merges, for e.g. when closing a shard, were
blocked if their disk space estimation was exceeding the available disk
space threshold. In this case, the shard close operation would itself
block.
This fix estimates a disk space budget of `0` for aborted merges, and it
periodically checks if any enqueued merge tasks have been aborted (more
generally, it checks if the budget estimate for any merge tasks has
changed, and reorders the queue if so). This way aborted merges are
prioritized and are never blocked.
Closes https://github.com/elastic/elasticsearch/issues/129335
In the last two months a lot of tests were converted to use the newer rest test framework. Some tests start 1 node, other start 3 nodes, others even more, the framework runs tests in parallel but it doesn't know how many nodes its tests needs meaning that running 3 tests in parallel, for example, can be very different when they are single node clusters or 3 node clusters etc. During this execution we saw the 3x more CPU load than what we would want to have ideally.
Currently there is no good solution for this because if dial down the concurrency we will use the nodes inefficiently, but if we keep the concurrency to where it is we risk longer start up times. Considering that the starting time of elasticsearch is not related to this test, we choose to increase the timeout to reduce the noise.
* Enable And Disable Endpoint
* Status Endpoint
* Integration Tests
* REST Spec
* REST Spec tests
* Some documentation
* Update docs/changelog/129474.yaml
* Fix failing security test
* PR Fixes
* PR Fixes - Add missing feature flag name to YAML spec
* PR Fixes - Fix support for timeout and master_timeout parameters
* PR Fixes - Make the REST handler validation happy with the new params
* Delete docs/changelog/129474.yaml
* PR Fixes - Switch to local metadata action type and improve request handling
* PR Fixes - Make enable / disable endpoint cancellable
* PR Fixes - Switch timeout param name for status endpoint
* PR Fixes - Switch timeout param name for status endpoint in spec
* PR Fixes - Enforce local only use for status action
* PR Fixes - Refactor StreamsMetadata into server
* PR Fixes - Add streams module to multi project YAML test suite
* PR Fixes - Add streams cluster module to multi project YAML test suite
The RunningSnapshotIT upgrade test adds shutdown markers to all nodes
and removes them once all nodes are upgraded. If an index gets created
in a mixed cluster, for example by ILM or deprecation messages, the
index cannot be allocated because all nodes are shutting down. Since the
cluster ready check between node upgrades expects a yellow cluster, the
unassigned index prevents the ready check to succeed and eventually
timeout. This PR fixes it by removing shutdown marker for the 1st
upgrade node to allow it hosting new indices.
Resolves: #129644Resolves: #129645Resolves: #129646
Our hypothesis is that the path is not round-tripping on Windows. This
fixes the test by always using the canonical string for the path, when
inserting and when purging.
Fixes#129635
Chris Hegarty