mirrors
/
redis
mirror of https://mirror.osredm.com/root/redis.git
2
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

README: suggest using FP32 vs VALUES.

This commit is contained in:
antirez 2025-03-26 11:28:05 +01:00
parent 16e3c5a8f9
commit 23b3e21817
1 changed files with 11 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

11
README.md
View File

@ -537,6 +537,8 @@ Vector Sets, or better, HNSWs, the underlying data structure used by Vector Sets
# Vector Sets troubleshooting and understandability
## Debugging poor recall or unexpected results
Vector graphs and similarity queries pose many challenges mainly due to the following three problems:
1. The error due to the approximated nature of Vector Sets is hard to evaluate.
@ -549,6 +551,15 @@ The only way to debug such problems, is the ability to inspect step by step what
2. The `VLINKS` command allows to explore the graph to see if the connections among nodes make sense, and to investigate why a given node may be more isolated than expected. Such command can also be used in a different way, when we want very fast "similar items" without paying the HNSW traversal time. It exploits the fact that we have a direct reference from each element in our vector set to each node in our HNSW graph.
3. The `WITHSCORES` option, in the supported commands, return a value that is directly related to the *cosine similarity* between the query and the items vectors, the interval of the similarity is simply rescaled from the -1, 1 original range to 0, 1, otherwise the metric is identical.
## Clients, latency and bandwidth usage
During Vector Sets testing, we discovered that often clients introduce considerable latecy and CPU usage (in the client side, not in Redis) for two main reasons:
1. Often the serialization to `VALUES ... list of floats ...` can be very slow.
2. The vector payload of floats represented as strings is very large, resulting in high bandwidth usage and latency, compared to other Redis commands.
Switching from `VALUES` to `FP32` as a method for transmitting vectors may easily provide 10-20x speedups.
# Known bugs
* When `VADD` with `REDUCE` is replicated, we should probably send the replicas the random matrix, in order for `VEMB` to read the same things. This is not critical, because the behavior of `VADD` / `VSIM` should be transparent if you don't look at the transformed vectors, but still probably worth doing.