When return an error from Elasticsearch exceptions may contain values
written as http response headers. ElasticsearchException contains a map
of headers that are added to the response. But these values are also
written to a special "header" section of the response body.
This commit renames the existing "headers" in ElasticsearchException to
"body headers", which are both http headers and written to the response
body. A new "http headers" is added for headers that should only be
written as response headers.
This is three fixes:
- We should be doing actual sampling when doing kmeans clustering, taking the first N vectors creates some weird edge cases
- Having assignments initialized as `0` means that if a vector gets assigned to cluster ord `0`, that cluster centroid actually isn't updated later in the lloyd steps. So, this initializes assignments to -1
- If we actually don't sample the vectors for lloyd, don't bother with final pass to potentially update the centroids
this is a small refactor, laying ground work for more generalized bulk
writing.
I did some benchmarking and there was no significant performance
difference (as expected).
There was a bug in previous version where flattened fields would produce incorrect synthetic source with too few opening braces. This bug was fixed as a side effect of #129600. Adding this test to confirm. See #129600 for a full explanation.
* Add age_in_millis to ILM Explain Response
* Fix: Declare age_in_millis in ILM Explain Response Parser
* Add unit tests for age_in_millis field in IndexLifecycleExplainResponse
* Update docs/changelog/128866.yaml
* Update docs/changelog/128866.yaml
* Update IndexLifecycleExplainResponse.java
Fix comments; follow DRY for getAge(nowSupplier) call in IndexLifecycleExplainResponse
---------
Co-authored-by: Joe Gallo <joe.gallo@elastic.co>
Updates the majority of the plugin to be aware of multiple projects.
The deprecation plugin is excluded in serverless, so in
`DeprecationIndexingComponent` we hard-code the default project ID to
avoid an unworthy investment of namespacing effort.
Add Cluster Allocation Explain API Tests
Adds tests for the `cluster/allocation/explain` API for when non-integer values are passed as the integer expected shard parameter. This change does not modify the `cluster/allocation/explain` API itself.
Updates the rollup APIs to work in a multi-project context (even though
that will never actually happen, as rollup is deprecated and the plugin
is excluded in serverless).
Passes the minimum transport version down to expressions when
we convert them into queries that we'll use for can_match.
Right now all this is used for is skipping the can_match from
the wildcard like queries. The queries we make there aren't
serializable. We'll fix that - but this should give us the
levers that we need to do it in a backwards incompatible way.
Updates the APIs to be able to handle multiple projects. This commit
does not guarantee that every sub-API returns sensible results in a
multi-project context. Some APIs/features potentially require further
work to return sensible data for multiple projects.
Since this API is for internal use only in serverless, we can check at a
later time whether any features require additional work.
When running under security manager an assumptionw as made that failing
to create a hard link due to security exception implied hard links were
supported. Now that security manager is gone, the code to create a hard
link in StoreRecoveryTests executes. But in the case of windows,
BasicFileAttributes.fileKey does not return a unique object that can be
used to verify a link exists. Yet the fact createLink returned is enough
to trust the jdk was able to create a link.
closes#124104
For dense vector fields using the `flat` index, we already know a brute-force search will be used—so there’s no need to go through the codec’s approximate KNN logic. This change skips that step and builds the brute-force query directly, making things faster and simpler.
I tested this on a setup with **10 million random vectors**, each with **1596 dimensions** and **17,500 partitions**, using the `random_vector` track.
The results:
### Performance Comparison
| Metric | Before | After | Change |
| ----------------- | --------- | ---------- | --------- |
| **Throughput** | 221 ops/s | 2762 ops/s | 🟢 +1149% |
| **Latency (p50)** | 29.2 ms | 1.6 ms | 🔻 -94.4% |
| **Latency (p99)** | 81.6 ms | 3.5 ms | 🔻 -95.7% |
Filtered KNN queries on flat vectors are now over 10x faster on my laptop!
This PR adds unit tests and docs for first_over_time, last_over_time,
and rate. For the rate function, the tests currently only verify that
the output is a double, not the actual value.
Newlines are system dependent. The release notes generator uses groovy's
template engine, which produces system dependent newlines. This commit
adjusts the test to account for newlines on both windows and nix
systems.
The enqueued task can run before the code reaches the status update
line. When it happens, the status update can set the status backwards.
This PR fixes it by moving the status update before enqueuing the task.
Resolves: #129752
Mark Vieira