Commit Graph

10 Commits

Author SHA1 Message Date
lerman25 94aebb7324
Add config base to vector-sets and hnsw thread config (#14082)
This PR introduces the initial configuration infrastructure for
vector-sets, along with a new option:
`vset-force-single-threaded-execution`. When enabled, it applies the
`NOTHREAD` flag to VSIM and disables the `CAS` option for VADD, thereby
enforcing single-threaded execution.
Note: This mode is not optimized for single-threaded performance.

---------

Co-authored-by: GuyAv46 <47632673+GuyAv46@users.noreply.github.com>
Co-authored-by: debing.sun <debing.sun@redis.com>
2025-06-16 10:06:43 +08:00
Salvatore Sanfilippo 27dd3b71ce
Vector Sets fixes against corrupted data in absence of checksum verification (#14102)
Vector Sets deserialization was not designed to resist corrupted data,
assuming that a good checksum would mean everything is fine. However
Redis allows the user to specify extra protection via a specific
configuration option.

This commit makes the implementation more resistant, at the cost of some
slowdown. This also fixes a serialization bug that is unrelated (and has
no memory corruption effects) about the lack of the worst index /
distance serialization, that could lower the quality of a graph after
links are replaced. I'll address the serialization issues in a new PR
that will focus on that aspect alone (already work in progress).

The net result is that loading vector sets is, when the serialization of
worst index/distance is missing (always, for now) 100% slower, that is 2
times the loading time we had before. Instead when the info will be
added it will be just 10/15% slower, that is, just making the new sanity
checks.

It may be worth to export to modules if advanced sanity check if needed
or not. Anyway most of the slowdown in this patch comes from having to
recompute the worst neighbor, since duplicated and non reciprocal links
detection was heavy optimized with probabilistic algorithms.

---------

Co-authored-by: debing.sun <debing.sun@redis.com>
2025-06-10 21:55:09 +08:00
Salvatore Sanfilippo 0ac822e154
Implement WITHATTRIBS for VSIM. (#14065)
Hi, as described, this implements WITHATTRIBS, a feature requested by a
few users, and indeed needed.
This was requested the first time by @rowantrollope but I was not sure
how to make it work with RESP2 and RESP3 in a clean way, hopefully
that's it.

The patch includes tests and documentation updates.
2025-05-27 22:12:48 +08:00
Salvatore Sanfilippo 65e164caff
[Vector sets] More rdb loading fixes (#14032)
Hi all, this PR fixes two things:

1. An assertion, that prevented the RDB loading from recovery if there
was a quantization type mismatch (with regression test).
2. Two code paths that just returned NULL without proper cleanup during
RDB loading.
2025-05-12 21:57:38 +03:00
Salvatore Sanfilippo a46624e10e
[Vector sets] RDB IO errors handling (#13978)
This PR adds support for REDISMODULE_OPTIONS_HANDLE_IO_ERRORS.
and tests for short read and corrupted RESTORE payload.

Please: note that I also removed the comment about async loading support
since we should be already covered. No manipulation of global data
structures in Vector Sets, if not for the unique ID used to create new
vector sets with different IDs.
2025-05-07 21:49:00 +03:00
Pieter Cailliau d65102861f
Adding AGPLv3 as a license option to Redis! (#13997)
Read more about [the new license option](http://redis.io/blog/agplv3/)
and [the Redis 8 release](http://redis.io/blog/redis-8-ga/).
2025-05-01 14:04:22 +01:00
Salvatore Sanfilippo 41ecf7323e
Vector Sets: VISMEMBER and other fixes (#13941)
CI / test-ubuntu-latest (push) Failing after 31s Details
CI / test-sanitizer-address (push) Failing after 31s Details
CI / build-debian-old (push) Failing after 31s Details
CI / build-32bit (push) Failing after 31s Details
CI / build-libc-malloc (push) Failing after 31s Details
CI / build-centos-jemalloc (push) Failing after 31s Details
CI / build-old-chain-jemalloc (push) Failing after 31s Details
Codecov / code-coverage (push) Failing after 31s Details
Spellcheck / Spellcheck (push) Failing after 31s Details
CI / build-macos-latest (push) Has been cancelled Details
CodeQL / Analyze (cpp) (push) Failing after 31s Details
External Server Tests / test-external-standalone (push) Failing after 31s Details
External Server Tests / test-external-nodebug (push) Failing after 32s Details
Coverity Scan / coverity (push) Has been skipped Details
External Server Tests / test-external-cluster (push) Failing after 1m50s Details
In this PR there is also a VADD leak fixed (when wrong arity is
reported) and improvements to the test.
2025-04-15 22:58:57 +03:00
Salvatore Sanfilippo 96a0cfdea2
Vectror Sets: build fixes for the w2v test (#13919)
CI / test-ubuntu-latest (push) Failing after 32s Details
CI / build-debian-old (push) Failing after 32s Details
CI / build-32bit (push) Failing after 32s Details
CI / build-libc-malloc (push) Failing after 32s Details
CI / build-centos-jemalloc (push) Failing after 32s Details
CI / build-old-chain-jemalloc (push) Failing after 32s Details
Codecov / code-coverage (push) Failing after 33s Details
CI / test-sanitizer-address (push) Failing after 1m16s Details
Spellcheck / Spellcheck (push) Failing after 32s Details
External Server Tests / test-external-standalone (push) Failing after 31s Details
Coverity Scan / coverity (push) Has been skipped Details
External Server Tests / test-external-cluster (push) Failing after 32s Details
External Server Tests / test-external-nodebug (push) Failing after 32s Details
CI / build-macos-latest (push) Has been cancelled Details
Hi, this fixes building Vector Sets as modules. Right now the module
builds but there are issues with w2v. This PR should fix the problem.
Thanks.
2025-04-09 14:39:33 +03:00
YaacovHazan 41b1b5df18 Add vector-sets module
The vector-sets module is a part of Redis Core and is available by default,
just like any other data type in Redis.

As a result, when building Redis from the source, the vector-sets module
is also compiled as part of the Redis binary and loaded at server start-up.

This new data type added as a preview currently doesn't support
all the capabilities in Redis like:
32-bit OS
C99
Short-read that might end with memory leak
AOF rewirte
defrag
2025-04-02 15:06:24 +00:00
YaacovHazan 78e0d87177 Add 'modules/vector-sets/' from commit 'c6db0a7c20ff5638f3a0c9ce9c106303daeb2f67'
git-subtree-dir: modules/vector-sets
git-subtree-mainline: 8ea8f4220c
git-subtree-split: c6db0a7c20
2025-04-02 16:34:28 +03:00