redis/tests/unit
uriyage 46393f9819
Optimization: sdsRemoveFreeSpace to avoid realloc on noop (#11766)
In #7875 (Redis 6.2), we changed the sds alloc to be the usable allocation
size in order to:

> reduce the need for realloc calls by making the sds implicitly take over
the internal fragmentation

This change was done most sds functions, excluding `sdsRemoveFreeSpace` and
`sdsResize`, the reason is that in some places (e.g. clientsCronResizeQueryBuffer)
we call sdsRemoveFreeSpace when we see excessive free space and want to trim it.
so if we don't trim it exactly to size, the caller may still see excessive free space and
call it again and again.

However, this resulted in some excessive calls to realloc, even when there's no need
and it's gonna be a no-op (e.g. when reducing 15 bytes allocation to 13).

It turns out that a call for realloc with jemalloc can be expensive even if it ends up
doing nothing, so this PR adds a check using `je_nallocx`, which is cheap to avoid
the call for realloc.

in addition to that this PR unifies sdsResize and sdsRemoveFreeSpace into common
code. the difference between them was that sdsResize would avoid using SDS_TYPE_5,
since it want to keep the string ready to be resized again, while sdsRemoveFreeSpace
would permit using SDS_TYPE_5 and get an optimal memory consumption.
now both methods take a `would_regrow` argument that makes it more explicit.

the only actual impact of that is that in clientsCronResizeQueryBuffer we call both sdsResize
and sdsRemoveFreeSpace for in different cases, and we now prevent the use of SDS_TYPE_5 in both.

The new test that was added to cover this concern used to pass before this PR as well,
this PR is just a performance optimization and cleanup.

Benchmark:
`redis-benchmark -c 100 -t set  -d 512 -P 10  -n  100000000`
on i7-9850H with jemalloc, shows improvement from 1021k ops/sec to 1067k (average of 3 runs).
some 4.5% improvement.

Co-authored-by: Oran Agra <oran@redislabs.com>
2023-01-31 17:26:35 +02:00
..
cluster Removed unecessary conversion of a dict to a dict (#11546) 2022-11-27 09:16:16 -08:00
moduleapi Fix potential issue with Lua argv caching, module command filter and libc realloc (#11652) 2023-01-04 11:03:55 +02:00
type Optimization: sdsRemoveFreeSpace to avoid realloc on noop (#11766) 2023-01-31 17:26:35 +02:00
acl-v2.tcl Unify ACL failure error messaging. (#11160) 2022-10-16 09:01:37 +03:00
acl.tcl Retain ACL categories used to generate ACL for displaying them later (#11224) 2022-11-03 10:14:56 -07:00
aofrw.tcl Add listpack encoding for list (#11303) 2022-11-16 20:29:46 +02:00
auth.tcl Fix incorrect error code for eval scripts and fix test error checking (#10575) 2022-04-14 11:18:32 +03:00
bitfield.tcl Fix command BITFIELD_RO and BITFIELD argument json file, add some test cases for them (#11445) 2022-11-02 15:15:12 +02:00
bitops.tcl Run large-memory tests as solo. (#10626) 2022-04-24 17:29:35 +03:00
client-eviction.tcl Optimize client memory usage tracking operation while client eviction is disabled (#11348) 2022-12-07 08:26:56 +02:00
dump.tcl Bump codespell from 2.1.0 to 2.2.1 in /.codespell (#11184) 2022-08-24 15:07:43 +03:00
expire.tcl Fix flaky PTTL time to live in milliseconds test on slow machines (#11651) 2022-12-22 10:51:43 +02:00
functions.tcl Tests: Add missing key declaration in scripts (#11134) 2022-08-16 22:04:22 +03:00
geo.tcl Fixed small distance replies on GEODIST and GEO commands WITHDIST (#11631) 2022-12-15 22:25:38 +02:00
hyperloglog.tcl Hyperloglog avoid allocate more than 'server.hll_sparse_max_bytes' bytes of memory for sparse representation (#11438) 2022-11-28 17:35:31 +02:00
info-command.tcl Make INFO command variadic (#6891) 2022-02-08 13:14:42 +02:00
info.tcl Fix error stats and failed command stats for blocked clients (#10309) 2022-02-21 11:20:41 +02:00
introspection-2.tcl Fix TIME command microseconds overflow under 32-bits (#11368) 2022-10-09 18:02:37 +03:00
introspection.tcl reprocess command when client is unblocked on keys (#11012) 2023-01-01 23:35:42 +02:00
keyspace.tcl Add listpack encoding for list (#11303) 2022-11-16 20:29:46 +02:00
latency-monitor.tcl fix flaky latency test (#11636) 2022-12-18 17:07:46 +02:00
lazyfree.tcl attempt to fix tracking test issue with external tests due to lazy free (#9722) 2021-11-02 16:42:53 +02:00
limits.tcl Improve test suite to handle external servers better. (#9033) 2021-06-09 15:13:24 +03:00
maxmemory.tcl Make assert_refcount skip the OBJECT REFCOUNT check with needs:debug tag (#11487) 2022-11-22 16:38:27 +02:00
memefficiency.tcl Solve issues with active defrag test failing on fast machines (#11598) 2022-12-09 13:33:38 +02:00
multi.tcl Fix replication inconsistency on modules that uses key space notifications (#10969) 2022-08-18 10:16:32 +03:00
networking.tcl Protected configs and sensitive commands (#9920) 2021-12-19 10:46:16 +02:00
obuf-limits.tcl Obuf limit, exit during loop in *RAND* commands and KEYS (#11676) 2023-01-16 13:51:18 +02:00
oom-score-adj.tcl Return 0 when config set out-of-range oom-score-adj-values (#10601) 2022-04-19 11:31:15 +03:00
other.tcl FLUSHDB and FLUSHALL add call forceCommandPropagation / FLUSHALL reset dirty counter to 0 if we enable save (#10691) 2022-05-11 11:21:16 +03:00
pause.tcl Refactor and (internally) rebrand from pause-clients to pause-actions (#11098) 2022-10-27 11:57:04 +03:00
printver.tcl Print version info before running the test 2011-05-20 11:44:54 +02:00
protocol.tcl add test suite infra to test RESP3 attributes (#10247) 2022-02-07 00:10:05 +02:00
pubsub.tcl Keyspace event for new keys (#10512) 2022-04-13 11:36:38 +03:00
pubsubshard.tcl Sharded pubsub publish messagebulk as smessage (#10792) 2022-05-31 08:03:59 +03:00
querybuf.tcl Ignore resize threshold on idle qbuf resizing (#9322) 2021-08-06 20:50:34 +03:00
quit.tcl Add tests for OK on QUIT 2010-10-15 12:54:53 +02:00
replybufsize.tcl Introduce debug command to disable reply buffer resizing (#10360) 2022-03-01 14:40:29 +02:00
scan.tcl Listpack encoding for sets (#11290) 2022-11-09 19:50:07 +02:00
scripting.tcl Fix potential issue with Lua argv caching, module command filter and libc realloc (#11652) 2023-01-04 11:03:55 +02:00
shutdown.tcl unblockClient: avoid to reset client when the client was shutdown-blocked (#10440) 2022-03-20 15:18:53 +02:00
slowlog.tcl reprocess command when client is unblocked on keys (#11012) 2023-01-01 23:35:42 +02:00
sort.tcl Avoid integer overflows in SETRANGE and SORT (CVE-2022-35977) (#11720) 2023-01-16 13:49:30 +02:00
tls.tcl Add support for reading encrypted keyfiles. (#8644) 2021-03-22 13:27:46 +02:00
tracking.tcl fix the client type in trackingInvalidateKey() (#11052) 2022-08-10 11:58:54 +03:00
violations.tcl Run large-memory tests as solo. (#10626) 2022-04-24 17:29:35 +03:00
wait.tcl Improve BLMPOP/BZMPOP/WAIT timeout overflow handling and error messages (#11338) 2022-10-06 12:12:05 +03:00