Commit Graph

4306 Commits

Author SHA1 Message Date
antirez e0582b3598 Fix maxmemory shared integer check bug introduced with LFU. 2016-07-21 11:14:18 +02:00
antirez 2d5eb1f1a0 Volatile-ttl eviction policy implemented in terms of the pool.
Precision of the eviction improved sensibly. Also this allows us to have
a single code path for most eviction types.
2016-07-20 19:54:12 +02:00
antirez 6854c7b9ee LFU: make counter log factor and decay time configurable. 2016-07-20 15:00:35 +02:00
antirez 6416ab19d0 LFU: Use the LRU pool for the LFU algorithm.
Verified to have better real world performances with power-law access
patterns because of the data accumulated across calls.
2016-07-18 18:17:59 +02:00
antirez dbce190ad0 LFU: Fix bugs in frequency decay code. 2016-07-18 14:19:38 +02:00
antirez a8e2d0849e LFU: Initial naive eviction cycle.
It is possible to get better results by using the pool like in the LRU
case. Also from tests during the morning I believe the current
implementation has issues in the frequency decay function that should
decrease the counter at periodic intervals.
2016-07-18 13:50:19 +02:00
antirez 24dd4a8f04 redis-cli LRU test mode: randomize value of key when setting.
This way it is possible from an observer to tell when the key is
replaced with a new one having the same name.
2016-07-18 10:56:47 +02:00
antirez b8450d7cc5 redis-cli LRU test mode: remove newline from key names. 2016-07-18 10:53:02 +02:00
antirez 5d07984c5d LFU: Redis object level implementation.
Implementation of LFU maxmemory policy for anything related to Redis
objects. Still no actual eviction implemented.
2016-07-15 12:12:58 +02:00
antirez e423f76e75 LRU: Make cross-database choices for eviction.
The LRU eviction code used to make local choices: for each DB visited it
selected the best key to evict. This was repeated for each DB. However
this means that there could be DBs with very frequently accessed keys
that are targeted by the LRU algorithm while there were other DBs with
many better candidates to expire.

This commit attempts to fix this problem for the LRU policy. However the
TTL policy is still not fixed by this commit. The TTL policy will be
fixed in a successive commit.

This is an initial (partial because of TTL policy) fix for issue #2647.
2016-07-13 13:12:30 +02:00
antirez e64bf05f43 LRU: cache SDS strings in the eviction pool.
To destroy and recreate the pool[].key element is slow, so we allocate
in pool[].cached SDS strings that can account up to 255 chars keys and
try to reuse them. This provides a solid 20% performance improvement
in real world workload alike benchmarks.
2016-07-12 12:31:37 +02:00
antirez 965905c9f2 Move the struct evictionPoolEntry() into only file using it.
Local scope is always better when possible.
2016-07-12 12:22:38 +02:00
antirez d8e92a8207 Move prototype of evictionPoolAlloc() in server.h. 2016-07-12 12:22:35 +02:00
antirez 3b9495d20d LRU: use C99 variable len stack array in evictionPoolPopulate(). 2016-07-12 12:05:45 +02:00
antirez 2a1247309a redis-benchmark: new option to show server errors on stdout.
Disabled by default, can be activated with -e. Maybe the reverse was
more safe but departs from the past behavior.
2016-07-12 11:23:31 +02:00
antirez 382991f82e Remove useless memmove() from freeMemoryIfNeeded().
We start from the end of the pool to the initial item, zero-ing
every entry we use or every ghost entry, there is nothing to memmove
since to the right everything should be already set to NULL.
2016-07-11 19:18:17 +02:00
antirez 51c1d40d08 redis_check_rdb(): the rio structure must be global.
The rio structure is referenced in the global 'riostate' structure
in order for the logging functions to be always able to access the state
of the "pseudo-loading" of the RDB, needed for the check.

Courtesy of Valgrind.
2016-07-06 19:12:24 +02:00
antirez eee878cbc5 redis_check_rdb_main(): create shared objects only if needed.
Otherwise Valgrind will complain a memory leak under certain tests where
RDB checking is invoked from within Redis.
2016-07-06 19:06:22 +02:00
antirez 24882e31f2 Fix redis_check_rdb() return value. 2016-07-06 19:00:43 +02:00
antirez 1e6bb9ef0c Remove dead code from geohash_helper.c.
The function removed also had potential bugs related to signess of the
expression, and is not used anyway.
2016-07-06 16:39:23 +02:00
antirez 3961071b20 Fix signess issue in geohashEstimateStepsByRadius(). 2016-07-06 16:38:05 +02:00
antirez 504ccad1fa Fix definition of M_PI in geohash_helper.c.
Without the right feature macros M_PI is not defined in math.h.
2016-07-06 16:31:11 +02:00
antirez eaa713e93b geohash.c and geohash_helper.c are part of Redis.
They were under /deps since they originate from a different source tree,
however at this point they are very modified and we took ownership of
both the files making changes, fixing bugs, so there is no upgrade path
from the original code tree.

Given that, better to move the code under /src with proper dependencies
and with a more simpler editing experience.
2016-07-06 16:02:38 +02:00
antirez 4a140d320f Add expire.c and evict.c. 2016-07-06 15:28:18 +02:00
antirez b46239e58b Expire and LRU related code moved into different files. 2016-07-06 15:24:06 +02:00
antirez 0610683d5e Makefile: don't build dependencies file for clean, distclean. 2016-07-06 12:56:43 +02:00
antirez 10361829f9 Generate Makefile.dep at every build.
Normally we used to update it from time to time. Too fragile... better
to generate dependencies at every run and delete them on 'make clean'.
2016-07-06 12:24:48 +02:00
antirez 23791828f1 getLongLongFromObject: use string2ll() instead of strict_strtoll().
strict_strtoll() has a bug that reports the empty string as ok and
parses it as zero.

Apparently nobody ever replaced this old call with the faster/saner
string2ll() which is used otherwise in the rest of the Redis core.

This commit close #3333.
2016-07-06 11:43:33 +02:00
antirez ef6a4df29c redis-cli: check SELECT reply type just in state updated.
In issues #3361 / #3365 a problem was reported / fixed with redis-cli
not updating correctly the current DB on error after SELECT.

In theory this bug was fixed in 0042fb0e, but actually the commit only
fixed the prompt updating, not the fact the state was set in a wrong
way.

This commit removes the check in the prompt update, now that hopefully
it is the state that is correct, there is no longer need for this check.
2016-07-05 15:18:40 +02:00
Salvatore Sanfilippo 0df286596d Merge pull request #3365 from sskorgal/unstable
Fix for redis_cli printing default DB when select command fails. #3361
2016-07-05 15:12:06 +02:00
antirez c383be3b0f Sentinel: fix cross-master Sentinel address update.
This commit both fixes the crash reported with issue #3364 and
also properly closes the old links after the Sentinel address for the
other masters gets updated.

The two problems where:

1. The Sentinel that switched address may not monitor all the masters,
   it is possible that there is no match, and the 'match' variable is
   NULL. Now we check for no match and 'continue' to the next master.

2. By ispecting the code because of issue "1" I noticed that there was a
   problem in the code that disconnects the link of the Sentinel that
   needs the address update. Basically link->disconnected is non-zero
   even if just *a single link* (cc -- command link or pc -- pubsub
   link) are disconnected, so to check with if (link->disconnected)
   in order to close the links risks to leave one link connected.

I was able to manually reproduce the crash at "1" and verify that the
commit resolves the issue.

Close #3364.
2016-07-04 18:45:24 +02:00
antirez b2cc8bccdb CONFIG GET is now no longer case sensitive.
Like CONFIG SET always was. Close #3369.
2016-07-04 16:09:24 +02:00
antirez b99ad1bd80 Make tcp-keepalive default to 300 in internal conf.
We already changed the default in the redis.conf template, but I forgot
to change the internal config as well.
2016-07-04 12:08:42 +02:00
antirez 7e220a964a In Redis RDB check: more details in error reportings. 2016-07-01 15:26:55 +02:00
antirez e697153d18 In Redis RDB check: log decompression errors. 2016-07-01 11:59:25 +02:00
antirez df3c69e89e In Redis RDB check: log object type on error. 2016-07-01 11:40:40 +02:00
antirez 2ab7097178 In Redis RDB check: minor output message changes. 2016-07-01 09:52:35 +02:00
antirez e9f31ba9c2 In Redis RDB check: better error reporting. 2016-07-01 09:36:52 +02:00
sskorgal 9dfd9d1412 Fix for redis_cli printing default DB when select command fails. 2016-07-01 10:42:22 +05:30
antirez e97fadb045 In Redis RDB check: initial POC.
So far we used an external program (later executed within Redis) and
parser in order to check RDB files for correctness. This forces, at each
RDB format update, to have two copies of the same format implementation
that are hard to keep in sync. Morover the former RDB checker only
checked the very high-level format of the file, without actually trying
to load things in memory. Certain corruptions can only be handled by
really loading key-value pairs.

This first commit attempts to unify the Redis RDB loadig code with the
task of checking the RDB file for correctness. More work is needed but
it looks like a sounding direction so far.
2016-06-30 23:44:44 +02:00
Rojin George d0f53079e3 Merge remote-tracking branch 'refs/remotes/antirez/unstable' into unstable 2016-06-30 16:34:01 +05:30
antirez dc18a6a6f8 Merge branch 'unstable' of github.com:/antirez/redis into unstable 2016-06-27 18:12:46 +02:00
antirez 5e176e1af5 Fix quicklistReplaceAtIndex() by updating the quicklist ziplist size.
The quicklist takes a cached version of the ziplist representation size
in bytes. The implementation must update this length every time the
underlying ziplist changes. However quicklistReplaceAtIndex() failed to
fix the length.

During LSET calls, the size of the ziplist blob and the cached size
inside the quicklist diverged. Later, when this size is used in an
authoritative way, for example during nodes splitting in order to copy
the nodes, we end with a duplicated node that may contain random
garbage.

This commit should fix issue #3343, however several problems were found
reviewing the quicklist.c code in search of this bug that should be
addressed soon or later.

For example:

1. To take a cached ziplist length is fragile since failing to update it
leads to this kind of issues.

2. The node splitting code needs auditing. For example it works just for
a side effect of ziplistDeleteRange() to be able to cope with a wrong
count of elements to remove. The code inside quicklist.c assumes that
-1 means "delete till the end" while actually it's just a count of how
many elements to delete, and is an unsigned count. So -1 gets converted
into the maximum integer, and just by chance the ziplist code stops
deleting elements after there are no more to delete.

3. Node splitting is extremely inefficient, it copies the node and
removes elements from both nodes even when actually there is to move a
single entry from one node to the other, or when the new resulting node
is empty at all so there is nothing to copy but just to create a new
node.

However at least for Redis 3.2 to introduce fresh code inside
quicklist.c may be even more risky, so instead I'm writing a better
fuzzy tester to stress the internals a bit more in order to anticipate
other possible bugs.

This bug was found using a fuzzy tester written after having some clue
about where the bug could be. The tester eventually created a ~2000
commands sequence able to always crash Redis. I wrote a better version
of the tester that searched for the smallest sequence that could crash
Redis automatically. Later this smaller sequence was minimized by
removing random commands till it still crashed the server. This resulted
into a sequence of 7 commands. With this small sequence it was just a
matter of filling the code with enough printf() to understand enough
state to fix the bug.
2016-06-27 18:02:33 +02:00
Yossi Gottlieb 19c401d81d Fix RedisModule_Calloc() definition typo. 2016-06-23 22:30:32 +03:00
rojingeorge 4242fdf45c Display the nodes summary once the cluster is established using redis-trib.rb
Display the nodes summary once the cluster is established using
redis-trib.rb
After the cluster meet and join was done, when the summary was shown, it
was giving info regarding the nodes. This fix ensures that confusion
where the slaves were shown as masters.
Fix would be to reset the nodes and reload the cluster information
before checking the cluster status after creating it.
2016-06-23 21:58:03 +05:30
Salvatore Sanfilippo f60aa4de30 Merge pull request #3324 from mishan/fix-wrong-comment-about-sentinel-mode
Fix incorrect comment for checkForSentinelMode function
2016-06-23 16:39:51 +02:00
antirez 18983113c5 Modules: mention RedisModule_Calloc() in the doc. 2016-06-23 16:20:48 +02:00
Salvatore Sanfilippo 3a0b776b94 Merge pull request #3335 from dvirsky/rm_calloc
added RM_Calloc implementation
2016-06-23 16:19:14 +02:00
antirez c026b5cd3e Merge branch 'unstable' of github.com:/antirez/redis into unstable 2016-06-23 16:18:57 +02:00
antirez 0f484d8312 Actually remove static from #3331.
I forgot -a when amending in the previous commit.
2016-06-23 16:18:30 +02:00
Salvatore Sanfilippo 28ea585fce Merge pull request #3336 from yossigo/create_string_from_string
Add RedisModule_CreateStringFromString().
2016-06-23 16:16:28 +02:00
antirez c0ca87dcc0 Minor change to conform PR #3331 to Redis code base style.
Also avoid "static" in order to have symbols during crashes.
2016-06-23 16:14:16 +02:00
Salvatore Sanfilippo a66dd43331 Merge pull request #3331 from yossigo/fix_openkey_crash
Fix occasional RM_OpenKey() crashes.
2016-06-23 16:12:07 +02:00
tielei f648c5a70c A string with 21 chars is not representable as a 64-bit integer. 2016-06-23 19:53:56 +08:00
Salvatore Sanfilippo 3d48c93122 Merge pull request #3330 from yossigo/fix_const
Use const in Redis Module API where possible.
2016-06-23 12:29:52 +02:00
antirez 4b12c6a360 Modules: changes to logging function.
This commit changes what provided by PR #3315 (merged) in order to
let the user specify the log level as a string.

The define could be also used, but when this happens, they must be
decoupled from the defines in the Redis core, like in the other part of
the Redis modules implementations, so that a switch statement (or a
function) remaps between the two, otherwise we are no longer free to
change the internal Redis defines.
2016-06-23 12:11:30 +02:00
Yossi Gottlieb 715794b829 Add RedisModule_Log() logging API function. 2016-06-23 12:01:44 +02:00
antirez b507289750 Commit change in autoMemoryFreed(): first -> last.
It's more natural to call the last entry added as "last", the original
commet got me confused until I actually read the code.
2016-06-23 09:38:30 +02:00
antirez f2dbc02f65 Modules: implement zig-zag scanning in autoMemoryFreed().
Most of the time to check the last element is the way to go, however
there are patterns where the contrary is the best choice. Zig-zag
scanning implemented in this commmit always checks the obvious element
first (the last added -- think at a loop where the last element
allocated gets freed again and again), and continues checking one
element in the head and one in the tail.

Thanks to @dvisrky that fixed the original implementation of the
function and proposed zig zag scanning.
2016-06-23 09:09:51 +02:00
Salvatore Sanfilippo 2fe9b79897 Merge pull request #3244 from dvirsky/optimize_autoMemoryFreed
Optimized autoMemoryFreed loop
2016-06-23 08:59:38 +02:00
Yossi Gottlieb 61172ed01e Add RedisModule_CreateStringFromString(). 2016-06-22 21:02:40 +03:00
Dvir Volk dc7f3fefad added RM_Calloc implementation 2016-06-22 17:32:41 +03:00
antirez 4e10b08fb3 Modules doc: hint about replacing libc malloc calls. 2016-06-22 15:24:51 +02:00
Yossi Gottlieb e22f3e40d5 Cleanup: remove zset reset function from RM_ZsetRangeStop(). 2016-06-22 07:34:14 +03:00
Yossi Gottlieb a8e2034548 Fix occasional RM_OpenKey() crashes. 2016-06-21 10:22:19 +03:00
Yossi Gottlieb 8f3a4df775 Use const in Redis Module API where possible. 2016-06-20 23:08:06 +03:00
Salvatore Sanfilippo 0b4b7ebd95 Merge pull request #3252 from oranagra/config_fix
fix: config set list-max-ziplist-size didn't support negative values
2016-06-17 14:48:41 +02:00
Misha Nasledov 7a5538d3a9 Fix incorrect comment for checkForSentinelMode function 2016-06-16 16:50:53 -07:00
antirez f7351f4c07 Fix Sentinel pending commands counting.
This bug most experienced effect was an inability of Redis to
reconfigure back old masters to slaves after they are reachable again
after a failover. This was due to failing to reset the count of the
pending commands properly, so the master appeared fovever down.

Was introduced in Redis 3.2 new Sentinel connection sharing feature
which is a lot more complex than the 3.0 code, but more scalable.

Many thanks to people reporting the issue, and especially to
@sskorgal for investigating the issue in depth.

Hopefully closes #3285.
2016-06-16 19:27:24 +02:00
antirez 7c8f275a24 redis-cli: really connect to the right server.
I recently introduced populating the autocomplete help array with the
COMMAND command if available. However this was performed before parsing
the arguments, defaulting to instance 6379. After the connection is
performed it remains stable.

The effect is that if there is an instance running on port 6339,
whatever port you specify is ignored and 6379 is connected to instead.
The right port will be selected only after a reconnection.

Close #3314.
2016-06-16 17:23:31 +02:00
antirez a3f893b800 RESTORE: accept RDB dumps with older versions.
Reference issue #3218.

Checking the code I can't find a reason why the original RESTORE
code was so opinionated about restoring only the current version. The
code in to `rdb.c` appears to be capable as always to restore data from
older versions of Redis, and the only places where it is needed the
current version in order to correctly restore data, is while loading the
opcodes, not the values itself as it happens in the case of RESTORE.

For the above reasons, this commit enables RESTORE to accept older
versions of values payloads.
2016-06-16 15:53:57 +02:00
Salvatore Sanfilippo 8272ceadaa Merge pull request #3255 from oranagra/error_string
CLIENT error message was out of date
2016-06-16 12:59:05 +02:00
Salvatore Sanfilippo 64b834b8b2 Merge pull request #3256 from oranagra/georasius_neg
fix georadius returns multiple replies
2016-06-16 12:57:59 +02:00
antirez 2f2fd64c0d Minor aesthetic fixes to PR #3264.
Comment format fixed + local var modified from camel case to underscore
separators as Redis code base normally does (camel case is mostly used
for global symbols like structure names, function names, global vars,
...).
2016-06-16 12:54:33 +02:00
Salvatore Sanfilippo 33a9836fe3 Merge pull request #3264 from oranagra/bitfield_fix2
fix crash in BITFIELD GET on non existing key or wrong type see #3259
2016-06-16 12:52:36 +02:00
Salvatore Sanfilippo 5d83f6cfde Merge pull request #3274 from MOON-CLJ/fix_promoted_slave
Sentinel: fix check when can't send the command to the promoted slave
2016-06-15 17:24:11 +02:00
Salvatore Sanfilippo 226f679651 Merge pull request #3283 from ideal/unstable
fix mistake in comment in object.c
2016-06-15 15:53:19 +02:00
Salvatore Sanfilippo bd2cd7059a Merge pull request #3281 from jamespedwards42/unstable
Fix modules intro typos.
2016-06-15 12:51:15 +02:00
antirez 2d86995273 GETRANGE: return empty string with negative, inverted start/end. 2016-06-15 12:48:58 +02:00
antirez eb45e11496 Remove additional round brackets from fix for #3282. 2016-06-15 12:16:39 +02:00
Salvatore Sanfilippo 001cadc854 Merge pull request #3282 from wenduo/unstable
bitcount bug:return non-zero value when start > end (both negative)
2016-06-15 12:15:34 +02:00
zach shipko b7b9aa6d9b BSDs don't have -ldl 2016-06-14 13:46:42 +00:00
antirez 41d804d9dc TTL and TYPE LRU access fixed. TOUCH implemented. 2016-06-14 15:33:59 +02:00
antirez cd8e688226 redis-cli help.h updated. 2016-06-14 14:45:28 +02:00
antirez 5ba9bdec75 Modules: document how to pass config params to modules.
Related to #3293.
2016-06-13 10:05:28 +02:00
antirez 5831dd860a Fix example modules to have the right OnLoad() prototype.
Related to #3293.
2016-06-13 09:57:19 +02:00
antirez a4bce77e92 Don't assume no padding or specific ordering in moduleLoadQueueEntry structure.
We need to be free to shuffle fields or add more fields in a structure
without breaking code.

Related to issue #3293.
2016-06-13 09:51:06 +02:00
antirez 9a02dac2e8 Free module context after loading.
Now that modules receive RedisModuleString objects on loading, they are
allowed to call the String API, so the context must be released
correctly.

Related to #3293.
2016-06-13 09:45:53 +02:00
antirez b6cd008508 Make sure modules arguments are raw strings.
Related to PR #3293.
2016-06-13 09:40:28 +02:00
antirez 1ad5c22763 Minor changes to unifor C style to Redis code base for PR #3293. 2016-06-13 09:39:44 +02:00
Salvatore Sanfilippo e8d5387a44 Merge pull request #3293 from yossigo/module_config
Allow passing arguments to modules on load.
2016-06-13 09:31:59 +02:00
antirez e71f22f5f2 Fix typo: after -> before. 2016-06-10 10:39:38 +02:00
antirez e4567f243b Explain why module type names are 9 chars. 2016-06-10 10:36:09 +02:00
Salvatore Sanfilippo 688996f415 Merge pull request #3295 from catwell/pr-1-warnings
fix some compiler warnings
2016-06-10 10:20:28 +02:00
Salvatore Sanfilippo b4327ae5fe Merge pull request #3294 from yossigo/fix_unload
Fix MODULE UNLOAD crash and/or wrong error message.
2016-06-10 10:18:49 +02:00
antirez a1684ff1bb Remove tryObjectEncoding() calls from list type.
All lists are now represented via quicklists.
Quicklists are never represented referencing robj structures, so trying
to compress their representation does not make sense. That the new way
is faster was experimentally verified with micro benchmarks in order to
prove that the intuition was correct.
2016-06-10 10:15:37 +02:00
antirez 5beec9743e Merge branch 'unstable' of github.com:/antirez/redis into unstable 2016-06-10 10:12:11 +02:00
Salvatore Sanfilippo ab73544396 Merge pull request #3296 from catwell/pr-2-variadic-pushx
make RPUSHX and LPUSHX variadic
2016-06-10 10:10:08 +02:00
andyli 93a09877fe fix comment "b>a" to "a > b" 2016-06-10 09:15:26 +02:00
antirez 2a57ad5d90 Fixed typo in Sentinel compareSlavesForPromotion() comment. 2016-06-10 09:15:01 +02:00
antirez 5481336872 Merge branch 'unstable' of github.com:/antirez/redis into unstable 2016-06-10 09:00:02 +02:00
antirez b664aebda1 Improve timer callback creation comment. 2016-06-08 16:07:32 +02:00
jspraul 4df95e8419 Include 'fd_set' type name
Fix an MSYS2-build-breaking error: unknown type name ‘fd_set’
2016-06-07 16:46:00 -04:00
Yossi Gottlieb 2fd6ca3cf8 Remove gcc warning when redismodule.h is included by a multi-file
module.
2016-06-07 13:31:33 +03:00
Pierre Chapuis d88c3c77be make RPUSHX and LPUSHX variadic 2016-06-05 16:50:24 +02:00
Pierre Chapuis b670a16282 remove unused variable 2016-06-05 16:50:17 +02:00
Pierre Chapuis 3e9c20f63b untangle LINSERT and {L,R}PUSHX implementations 2016-06-05 16:50:10 +02:00
Pierre Chapuis 188d90fc87 fix some compiler warnings 2016-06-05 16:48:45 +02:00
Yossi Gottlieb 87312ff781 Fix MODULE UNLOAD crash and/or wrong error message. 2016-06-05 13:27:38 +03:00
Yossi Gottlieb cc58f11ccc Use RedisModuleString for OnLoad argv. 2016-06-05 13:18:24 +03:00
Yossi Gottlieb 2bd13cf0eb Allow passing arguments to modules on load. 2016-06-05 11:37:24 +03:00
antirez 550fa7e14f modules API.md updated. 2016-06-04 12:55:39 +02:00
antirez c3f5b6ebf9 Modules: native types doc, 70% done. 2016-06-04 12:54:18 +02:00
antirez 5830d8821b Modules: pool allocator doc. 2016-06-03 18:32:35 +02:00
antirez 31eb8eccef Modules: top comments in helloworld.c and hellotype.c. 2016-06-03 18:19:31 +02:00
antirez 8ec28002be Modules: support for modules native data types. 2016-06-03 18:14:04 +02:00
antirez 27e5f385c1 RDB v8: fix rdbLoadLen() return value. 2016-06-01 20:18:28 +02:00
antirez e6554bed92 RDB v8: new ZSET storage format with binary doubles. 2016-06-01 12:12:26 +02:00
antirez 4aae4f7d35 RDB v8: ability to save uint64_t lengths. 2016-06-01 11:35:47 +02:00
antirez 2503acfc83 Avoid undefined behavior in BITFIELD implementation.
Probably there is no compiler that will actaully break the code or raise
a signal for unsigned -> signed overflowing conversion, still it was
apparently possible to write it in a more correct way.

All tests passing.
2016-05-31 11:52:07 +02:00
ideal 4067132092 fix mistake comment in object.c 2016-05-30 16:57:36 +08:00
wenduo 41dacdbcbe bitcount bug:return non-zero value when start > end (both negative) 2016-05-30 16:21:08 +08:00
jamespedwards42 3432061cbb
Fix modules intro typos. 2016-05-29 15:53:24 -07:00
Itamar Haber 2866e023f8 Allow SPOP from Lua scripts
The existing `R` flag appears to be sufficient and there's no apparent reason why the command should be blocked.
2016-05-28 20:01:46 +03:00
MOON_CLJ aa578446ba fix check when can't send the command to the promoted slave 2016-05-26 13:10:12 +08:00
oranagra 5d96b7ed4f check WRONGTYPE in BITFIELD before looping on the operations.
optimization: lookup key only once, and grow at once to the max need
fixes #3259 and #3221, and also an early return if wrongtype is discovered by SET
2016-05-24 23:31:36 +03:00
oranagra c4433d2a6a fix crash in BITFIELD GET on non existing key or wrong type see #3259
this was a bug in the recent refactoring: bee963c445
2016-05-24 14:52:43 +03:00
oranagra f3e81de176 fix georadius returns multiple replies 2016-05-23 13:58:50 +03:00
oranagra 8d9d8d16e4 CLIENT error message was out of date 2016-05-23 11:42:21 +03:00
oranagra 5fa711fa37 config set list-max-ziplist-size didn't support negative values, unlike config file 2016-05-22 20:35:14 +03:00
Jan-Erik Rediger 892565f924 Remove debug printing 2016-05-21 13:50:01 +02:00
Dvir Volk 137fd86a61 optimized amFree even further 2016-05-19 13:51:55 +03:00
Dvir Volk 46b07cbb5c Optimized autoMemoryFreed loop 2016-05-19 12:16:14 +03:00
antirez b09a6b6a5d Fix modules compilation when libc malloc is used.
Compiling Redis worked as a side effect of jemalloc target specifying
-ldl as needed linker options, otherwise it is not provided during
linking and dlopen() API will remain unresolved symbols.
2016-05-18 17:48:06 +02:00
Salvatore Sanfilippo bafed3ddd6 Merge pull request #3222 from oranagra/more_minir_fixes
minor fixes - mainly signalModifiedKey, and GEORADIUS
2016-05-18 07:50:53 -07:00
antirez e3edae957b Modules: RM_HashSet() SDS ownership business clarified in comments.
Related to #3239.
2016-05-18 16:30:20 +02:00
Salvatore Sanfilippo 01a83d0a2a Merge pull request #3239 from dvirsky/fix_hashset_crash
fixed bad transfer of ownership in HashSet causing a potential crash
2016-05-18 07:15:15 -07:00
antirez bee963c445 Code to access object string bytes repeated 3x refactored into 1 function. 2016-05-18 15:35:19 +02:00
antirez ffd1600ccf Clarify that the LOG_STR_SIZE includes null term. 2016-05-18 15:23:35 +02:00
Salvatore Sanfilippo 4c9c9d726a Merge pull request #3221 from oranagra/bitfield_fix
fix crash in BITFIELD GET when key is integer encoded
2016-05-18 05:53:53 -07:00
whatacold 0f36e5e5d8 fix the wrong description of intsetGet(). 2016-05-18 16:39:52 +08:00
Dvir Volk cfaef8d5d4 fixed bad transfer of ownership in HashSet causing a potential crash 2016-05-17 16:47:36 +03:00
oranagra 283a8125cb reduce struct padding by reordering members 2016-05-16 20:12:11 +03:00
antirez 9aff564045 Modules: initial pool allocator and a LEFTPAD usage example. 2016-05-14 19:42:31 +02:00
antirez 646c958bbd Modules: doc layout improved. 2016-05-10 18:54:58 +02:00
antirez 745845df0a Modules doc: mention the functions not yet documented. 2016-05-10 15:50:21 +02:00
oranagra 77a9144245 fix crash in BITFIELD GET when key is integer encoded 2016-05-10 11:19:45 +03:00
antirez 5daece2ec6 RM_ZsetRangeNext()/Prev() typo in define name leading to crash fixed. 2016-05-10 06:40:11 +02:00
antirez 5f977c3f8b Modules: commandFlagsFromString() top comment back to 80 cols max. 2016-05-10 06:40:11 +02:00
antirez ef2b4f6496 Trailing spaces removed from moduleCreateArgvFromUserFormat(). 2016-05-10 06:40:11 +02:00
antirez 6250a6b11f Modules: RM_GetClientId() implemented. 2016-05-10 06:40:11 +02:00
Dvir Volk 9a71df505c fixed crash when calling CreateStringFromCallReply on array elements 2016-05-10 06:40:11 +02:00
Itamar Haber 3816f16a53 Avoids reallocating and double String on truncate 2016-05-10 06:40:11 +02:00
Dvir Volk d41bd233d5 fixed comment formatting in RM_CreateCommand 2016-05-10 06:40:10 +02:00
Ramon Snir 909a707b72 vector of strings is implemented now 2016-05-10 06:40:10 +02:00
Dvir Volk a00e872560 another small comment fix 2016-05-10 06:40:10 +02:00
Dvir Volk 87de31f968 fixed comment 2016-05-10 06:40:10 +02:00
Dvir Volk 8fc67080a2 second attempt at vector formtting 2016-05-10 06:40:10 +02:00
antirez 7f5e1340f9 Modules: add ZADD_INCR flag to zset increment API. 2016-05-10 06:40:10 +02:00
Itamar Haber 1186f92b1b typo: %s/Emtpy/Empty/g 2016-05-10 06:40:10 +02:00
antirez 227d68094b Modules: command <-> core interface modified to get flags & keys. 2016-05-10 06:40:09 +02:00
Ramon Snir 676a6a4d19 tiny typo in Redis Modules API documentation 2016-05-10 06:40:09 +02:00
Dvir Volk a1f8e22b79 fixed return value of HashGet (and a slight error in the documentation) 2016-05-10 06:40:09 +02:00
antirez 02c4a6c7f9 Modules: REDISMODULE_POSTPONED_ARRAY_LEN doc. 2016-05-10 06:40:09 +02:00
antirez 42f72210fd Modules: Hash API defines made more uniform. 2016-05-10 06:40:09 +02:00
antirez 9b0556cf10 Modules: Hash type API WIP #2. 2016-05-10 06:40:09 +02:00
antirez 10993ca0d5 Modules: Hash type API WIP #1. 2016-05-10 06:40:09 +02:00
antirez 5bf5fd24c6 Modules: a few fixes for the zset iterator. 2016-05-10 06:40:09 +02:00
antirez 33e1231e53 Modules: postponed array lengths. 2016-05-10 06:40:09 +02:00
antirez 00109e1113 Modules: zset lex iterator #3. 2016-05-10 06:40:09 +02:00
antirez db3ade22eb Modules: zset lex iterator #2. 2016-05-10 06:40:09 +02:00
antirez 2b04f86ae5 Modules: zset lex iterator #1. 2016-05-10 06:40:08 +02:00
antirez 083f5277c5 Modules: zset iterator redesign #1. 2016-05-10 06:40:08 +02:00
antirez d998170809 Simple Ruby script to generate reference doc added. 2016-05-10 06:40:08 +02:00
antirez 0fd6d548ca Modules: fix top comments to be user-facing doc quality. About 33% done. 2016-05-10 06:40:08 +02:00
antirez f362f7a18a Modules: sorted set iterators WIP #3. 2016-05-10 06:40:08 +02:00
antirez bdbb5a0253 Modules: put zset iterator current element in auto memory pool. 2016-05-10 06:40:08 +02:00
antirez 6eeeda39e9 Modules: sorted set iterators WIP #2. 2016-05-10 06:40:08 +02:00
antirez eac5a13cb7 Modules: sorted set iterators WIP. 2016-05-10 06:40:08 +02:00
antirez 556d593d37 Remove useless space. 2016-05-10 06:40:08 +02:00
antirez d5ecedd185 Modules: ZSET API WIP #4. 2016-05-10 06:40:07 +02:00
antirez e1b34ecf98 Modules: ZSET API WIP #3. 2016-05-10 06:40:07 +02:00
antirez 4457e4acbe Modules: ZSET API WIP #2. 2016-05-10 06:40:07 +02:00
antirez f199504ac9 Modules: ZSET API WIP. 2016-05-10 06:40:07 +02:00
antirez 11b3df24cb Modules: expire API and documentation. 2016-05-10 06:40:07 +02:00
antirez f4e0129fa9 Modules: RedisModule_ReplyWithCallReply(). 2016-05-10 06:40:07 +02:00
Itamar Haber 6054089fa8 Stops SPLICE from accepting negative counts 2016-05-10 06:40:07 +02:00
Sun He 2e464bf0b1 modules/RM_StringTruncate: correct reallocate condition 2016-05-10 06:40:06 +02:00
Sun He 3a7b170205 modules/RM_OpenKey: avoid decrRefCount obj twice 2016-05-10 06:40:06 +02:00
Sun He 1868dee6f2 modules/RM_StringSet: set key->value 2016-05-10 06:40:06 +02:00
Sun He fded8aa4e5 modules: correct protolen 2016-05-10 06:40:06 +02:00
antirez 4efe9e19c1 Add the last break for consistency in moduleCreateCallReplyFromProto. 2016-05-10 06:40:06 +02:00
Dvir Volk e453d36e1e fixed case in moduleCreateCallReplyFromProto 2016-05-10 06:40:06 +02:00
Dvir Volk ae5cb3f7d5 renamed RedisModule_ReplyWithNull to RM_ReplyWithNull to fix compilation 2016-05-10 06:40:06 +02:00
antirez 2967c00c2c Modules: RedisModule_ReplyWithNull() implemented. 2016-05-10 06:40:06 +02:00
Dvir Volk e711a9ac45 fixed the doc with a right function name 2016-05-10 06:40:06 +02:00
Yossi Gottlieb e443ad9c29 Log loadmodule dlopen() errors. 2016-05-10 06:40:06 +02:00
antirez 85919f80ed Modules: avoid conflict between modules func pointers and dynamic symbols.
In modules we fill a set of function pointers defined in redismodule.h,
populating a set of APIs that are callable from the module. We use this
manual process instead of resorting to dynamic linking so that we have
exact control on how we pass the API to the module, and we can even pass
different functions for the same name, depending on the API version
declared by the module.

However if the function pointers in redismodule.h and the functions
defined in module.c have the same name, they conflict since the core
exports the symbols to the module.

There is probably some compiler flags trick to avoid this, but in order
to be safer in the future and be more easily compatible with different
builidng systems, this commit changes the internal function prefix from
RedisModule_ to RM_, so for example:

    RM_StringSet() will be exported as RedisModule_StringSet()
2016-05-10 06:40:05 +02:00
antirez 6020469452 Modules: remove warnings due to void/function pointer conversion. 2016-05-10 06:40:05 +02:00
Dvir Volk 34f2fb7d8c fixed makefile for linux 2016-05-10 06:40:05 +02:00
antirez 6dead2cff5 Modules: first preview 31 March 2016. 2016-05-10 06:40:05 +02:00
oranagra 9682b616a2 minor fixes - mainly signalModifiedKey, and GEORADIUS 2016-05-09 12:05:33 +03:00
antirez 3b644e82b0 Merge branch 'unstable' of github.com:/antirez/redis into unstable 2016-05-08 18:04:53 +02:00
Salvatore Sanfilippo b44ad302d2 Merge pull request #732 from evilpacket/remove_dofile
Removes dofile() from Lua
2016-05-08 18:04:41 +02:00
antirez 8eb43bf72c redis-cli: integrate help.h with COMMAND output.
Use the COMMAND output to fill with partial information the built-in
help. This makes redis-cli able to at least complete commands that are
exported by the Redis server it is connected to, but were not available
in the help.h file when the redis-cli binary was compiled.
2016-05-07 13:03:25 +02:00
antirez 2205c465ca Cluster: don't check scripts key slots during AOF loading. 2016-05-05 23:37:08 +02:00
antirez 94dc71ff08 redis-cli: remove debugging message. 2016-05-05 18:05:37 +02:00
antirez bdbeb07a9e Merge branch 'unstable' of github.com:/antirez/redis into unstable 2016-05-05 17:39:40 +02:00
antirez 68dd1c9155 Revert "Fix commandCommand arity"
This reverts commit 1189a4eae6.

Actually this is wrong, the command can be called without args at all.
2016-05-05 17:36:05 +02:00
Salvatore Sanfilippo f0fcc36ce1 Merge pull request #2956 from pkulchenko/global-protection-msg-typo
Update global protection error message
2016-05-05 17:26:35 +02:00
Ruben Bridgewater efa0840707 Fix commandCommand arity 2016-05-05 17:20:14 +02:00
Salvatore Sanfilippo f5ff91f675 Merge pull request #2998 from danielhtshih/unstable
Fix a possible race condition of sdown event detection if sentinel's connection to master/slave/sentinel became disconnected just after the last PONG and before the next PING.
2016-05-05 17:16:58 +02:00
Salvatore Sanfilippo 0c1f84f0ea Merge pull request #3006 from baishaofei/unstable
fix linux compile Error zmalloc.c:109: error: invalid lvalue in unary `&'
2016-05-05 16:23:47 +02:00
Salvatore Sanfilippo 7b90815fc3 Merge pull request #3008 from badboy/fix-2911
Fix nanosecond conversion
2016-05-05 16:21:21 +02:00
antirez 02db338a82 redis-cli: don't free historyfile, is used later. 2016-05-05 13:58:06 +02:00
antirez 971e3c51b6 Cluster: make getNodeByQuery() responsible of -CLUSTERDOWN errors.
This fixes a bug introduced by d827dbf, and makes the code consistent
with the logic of always allowing, while the cluster is down, commands
that don't target any key.

As a side effect the code is also simpler now.
2016-05-05 11:33:43 +02:00
Salvatore Sanfilippo 330715afd8 Merge pull request #3039 from itamarhaber/patch-3
Fixes a typo in the comments
2016-05-05 10:15:17 +02:00
Salvatore Sanfilippo a4df156872 Merge pull request #3077 from Palethorn/ipv6-redirect-parse
Fix for #3076 Reverse redirect address parse
2016-05-05 10:13:00 +02:00
antirez c77b95fd01 Bind both IPv4 and IPv6 or exit with an error by default.
Thanks to @tushar2708 for the PR. I applied a slightly different fix.
Thanks to @cespare for reporting.

Close #3024
Close #3020
2016-05-05 10:02:42 +02:00
Chris Thunes d827dbfda7 Ensure slots are rechecked on EXEC.
Fixes #2515.
2016-05-05 09:35:20 +02:00
Salvatore Sanfilippo 91b4966783 Merge pull request #3188 from therealbill/unstable
maxmemory_policy fix for #3187
2016-05-05 09:02:25 +02:00
Salvatore Sanfilippo 09153b9127 Merge pull request #3152 from be-hase/fix/check_open_slots
Fix redis-trib.rb
2016-05-05 08:56:28 +02:00
antirez b632f7846b Minor redis-cli wording change in --help output. 2016-05-04 22:34:26 +02:00
antirez 995b9ffe07 Allow CONFIG GET during loading.
Thanks to @oranagra for the idea of allowing CONFIG GET during loading.
2016-05-04 15:45:45 +02:00
antirez 5500c51083 Command "r" flag removed from commands not accessing the key space.
Thanks to @oranagra for the hint about misplaced 'r' flags.
2016-05-04 15:42:33 +02:00
antirez 840ac20855 DEBUG command self documentation. 2016-05-04 12:45:55 +02:00
Salvatore Sanfilippo b5352eea51 Merge pull request #3191 from oranagra/minor_fix
Minor fixes found during merge and code review
2016-05-04 09:11:36 +02:00
antirez 2c22f59c3f Reply with error on negative geo radius.
Thanks to @tidwall for reporting.
Close #3194.
2016-05-04 09:00:32 +02:00
antirez 4fdde78c72 New masters with slots are now targets of migration if others are.
This fixes issue #3043.

Before this fix, after a complete resharding of a master slots
to other nodes, the master remains empty and the slaves migrate away
to other masters with non-zero nodes. However the old master now empty,
is no longer considered a target for migration, because the system has
no way to tell it had slaves in the past.

This fix leaves the algorithm used in the past untouched, but adds a
new rule. When a new or old master which is empty and without slaves,
are assigend with their first slot, if other masters in the cluster have
slaves, they are automatically considered to be targets for replicas
migration.
2016-05-02 18:37:30 +02:00
Oran Agra 5e3880a492 various cleanups and minor fixes 2016-04-25 16:49:57 +03:00
Oran Agra 6ed8c28230 dict.c minor optimization 2016-04-25 16:48:25 +03:00
Oran Agra 7b52ef1da2 networking.c minor optimization 2016-04-25 16:48:09 +03:00
Oran Agra f8909a2579 add DEBUG JEMALLC PURGE and JEMALLOC INFO cleanup 2016-04-25 16:47:42 +03:00
Oran Agra 7ba90225a0 fix small issues in redis 3.2 2016-04-25 14:19:28 +03:00
Oran Agra b554895715 additional fix to issue #2948 2016-04-25 14:18:40 +03:00
therealbill 14086a46ca fix for #3187
I've renamed maxmemoryToString to evictPolicyToString since that is
more accurate (and easier to mentally connect with the correct data), as
well as updated the function to user server.maxmemory_policy rather than
server.maxmemory. Now with a default config it is actually returning
the correct policy rather than volatile-lru.
2016-04-22 10:43:48 -05:00
Damian Janowski 0b4bb502a2 Fix ZINCRBY return value. 2016-04-18 00:35:54 -03:00
antirez dda0f37f11 ZREM refactored into proper API. 2016-04-15 15:20:57 +02:00
antirez 6f926c3e83 ZRANK refactored into proper API. 2016-04-15 12:47:03 +02:00
antirez b73c7af0f4 zsetAdd() API exposed into server.h. 2016-04-14 16:00:08 +02:00
antirez b1f181a736 ZADD refactored into a proper API. 2016-04-14 12:49:40 +02:00
antirez e0eb5f6bbf redis-cli preferences and rc file support. 2016-04-13 14:20:41 +02:00
antirez 70b3314141 redis-cli help.h updated. 2016-04-13 12:35:18 +02:00
antirez d1ddf7e958 redis-cli hints. 2016-04-13 12:34:14 +02:00
antirez b9feef9ae8 ae.c: Fix delay until next timer event.
This fix was written by Anthony LaTorre.
The old code mis-calculated the amount of time to wait till next event.
2016-04-04 14:13:46 +02:00
antirez ace780c002 ae.c: comment to explain why we have a useless maxId check. 2016-04-04 12:24:13 +02:00
antirez 67b70a1813 Fix ae.c to avoid timers infinite loop.
This fix was suggested by Anthony LaTorre, that provided also a good
test case that was used to verify the fix.

The problem with the old implementation is that, the time returned by
a timer event (that is the time after it want to run again) is added
to the event *start time*. So if the event takes, in order to run, more
than the time it says it want to be scheduled again for running, an
infinite loop is triggered.
2016-04-04 08:50:58 +02:00
Ryosuke Hasebe cad9ea5c68 fix variable 2016-03-30 23:09:36 +09:00
Ryosuke Hasebe d5aa7e2abe fix check_open_slots 2016-03-30 21:56:22 +09:00
antirez fc843784c3 BITFIELD: Farest bit set is offset+bits-1. Off by one error fixed. 2016-03-02 16:20:28 +01:00
antirez fe64960ad5 More BITFIELD fixes. Overflow conditional simplified.
See issue #3114.
2016-03-02 15:13:45 +01:00
Salvatore Sanfilippo 235f55344b Merge pull request #3118 from sunheehnus/bitfield-fix-minor-bug
bitops/bitfield: fix length, overflow condition and *sign
2016-03-02 15:12:38 +01:00
Sun He 93cc8baf1a bitops/bitfield: fix length, overflow condition and *sign 2016-03-02 18:11:30 +08:00
antirez e85d6f22cf Fix INFO commandstats reporting when argv is rewritten.
We want to report the original command in the stats, for example GEOADD,
even when what is actually executed is the ZADD implementation.
2016-03-02 08:56:50 +01:00
antirez 32289d5719 BITFIELD: refactoring & fix of retval on FAIL. 2016-02-29 09:08:46 +01:00
antirez 11745e0981 BITFIELD: Fix #<index> form parsing. 2016-02-26 15:53:29 +01:00
antirez 2800d090a7 BITFIELD: Support #<index> offsets form. 2016-02-26 15:16:24 +01:00
antirez 70af626d61 BITFIELD command initial implementation.
The new bitfield command is an extension to the Redis bit operations,
where not just single bit operations are performed, but the array of
bits composing a string, can be addressed at random, not aligned
offsets, with any width unsigned and signed integers like u8, s5, u10
(up to 64 bit signed integers and 63 bit unsigned integers).

The BITFIELD command supports subcommands that can SET, GET, or INCRBY
those arbitrary bit counters, with multiple overflow semantics.

Trivial and credits:

A similar command was imagined a few times in the past, but for
some reason looked a bit far fetched or not well specified.
Finally the command was proposed again in a clear form by
Yoav Steinberg from Redis Labs, that proposed a set of commands on
arbitrary sized integers stored at bit offsets.

Starting from this proposal I wrote an initial specification of a single
command with sub-commands similar to what Yoav envisioned, using short
names for types definitions, and adding control on the overflow.

This commit is the resulting implementation.

Examples:

    BITFIELD mykey OVERFLOW wrap INCRBY i2 10 -1 GET i2 10
2016-02-26 15:00:19 +01:00
Itamar Haber b5149f0868 Eliminates engineers near the equator & prime meridian 2016-02-18 15:11:30 -08:00
Itamar Haber 41030ae2de Fixes a typo in a comment 2016-02-18 15:01:34 -08:00
Itamar Haber 4e9c3027b1 Adjusts accuracy for GEODIST 2016-02-18 15:00:39 -08:00
antirez cf42c48adc addReplyHumanLongDouble() API added.
Send a long double or double as a bulk reply, in a human friendly
format.
2016-02-18 22:08:50 +01:00
antirez bb75ecddfd New options for GEORADIUS: STORE and STOREDIST.
Related to issue #3019.
2016-02-18 10:24:16 +01:00
antirez b0ec22f948 Include full paths on RDB/AOF files errors.
Close #3086.
2016-02-15 16:15:01 +01:00
David Cavar c30ffaab05 Reverse redirect address parse
Fix issue in case the redirect address is in ipv6 format. Parse from behind to extract last part of the response which represents actual port.
2016-02-09 15:04:42 +01:00
Itamar Haber 31a70a8bcf Fixes a typo 2016-02-03 11:04:09 -08:00
Itamar Haber 68e779f34b Adds keyspace notifications for lrem 2016-02-02 09:58:19 -08:00
antirez cbcffed907 Cluster: redis-trib: support @busport format in ClusterNode. 2016-02-02 08:27:36 +01:00
antirez b841f3ad1a Cluster: store busport with different separator in CLUSTER NODES.
We need to be able to correctly parse the node address in the case of
IPv6 addresses.
2016-02-02 08:20:04 +01:00
antirez 92b9de2417 Cluster announce: WIP, allow building again. 2016-02-01 18:16:25 +01:00
antirez e27b9b1cec Merge branch 'cluster-docker' into unstable 2016-02-01 18:01:22 +01:00
antirez cdbe8a6ae1 Typo ASII -> ASCII fixed in comment. 2016-01-29 12:08:10 +01:00
antirez c285862621 Cluster: include node IDs in SLOTS output.
CLUSTER SLOTS now includes IDs in the nodes description associated with
a given slot range. Certain client libraries implementations need a way
to reference a node in an unique way, so they were relying on CLUSTER
NODES, that is not a stable API and may change frequently depending on
Redis Cluster future requirements.
2016-01-29 12:00:40 +01:00
antirez d0a8512eda Cluster anounce-ip/port WIP. 2016-01-29 09:06:37 +01:00
antirez 4abf486ca3 Cluster announce port: set port/bport for myself at startup. 2016-01-29 09:06:37 +01:00
antirez 1c038379f7 Cluster: persist bus port in nodes.conf. 2016-01-29 09:06:37 +01:00
antirez dc98907e50 Cluster announce ip: take myself->ip always in sync. 2016-01-29 09:06:37 +01:00
antirez 11436b1449 Cluster announce ip / port initial implementation. 2016-01-29 09:06:37 +01:00
antirez b0939303e6 Cluster announce ip / port configuration handling. 2016-01-29 09:06:37 +01:00
antirez a455e4b948 Cluster: add announce ip field in messages header. 2016-01-29 09:06:36 +01:00
Itamar Haber 9e46bf22ed Fixes a typo 2016-01-28 21:47:18 +02:00
antirez 5bbb09ed2c Cluster: check packets length before accessing far fields. 2016-01-27 16:35:21 +01:00
antirez 751b5666fb Sentinel: improve handling of known Sentinel instances.
1. Bug #3035 is fixed (NULL pointer access). This was happening with the
   folling set of conditions:

* For some reason one of the Sentinels, let's call it Sentinel_A, changed ID (reconfigured from scratch), but is as the same address at which it used to be.

* Sentinel_A performs a failover and/or has a newer configuration compared to another Sentinel, that we call, Sentinel_B.

* Sentinel_B receives an HELLO message from Sentinel_A, where the address and/or ID is mismatched, but it is reporting a newer configuration for the master they are both monitoring.

2. Sentinels now must have an ID otherwise they are not loaded nor persisted in the configuration. This allows to have conflicting Sentinels with the same address since now the master->sentinels dictionary is indexed by Sentinel ID.

3. The code now detects if a Sentinel is annoucing itself with an IP/port pair already busy (of another Sentinel). The old Sentinel that had the same port/pair is set as having port 0, that means, the address is invalid. We may discover the right address later via HELLO messages.
2016-01-27 16:27:49 +01:00
antirez 5bc7e019e1 Use a smoother running average for avg_ttl in INFO.
Reported here:
https://www.reddit.com/r/redis/comments/42r0i0/avg_ttl_varies_a_lot/
2016-01-26 15:29:30 +01:00
antirez fe44a7cb60 Cluster: mismatch sender ID log put back at DEBUG level. 2016-01-26 14:21:18 +01:00
antirez d6c5922f75 Cluster: fix missing ntohs() call to access gossip section port. 2016-01-26 14:18:13 +01:00
antirez 592419b4ca Better address udpate strategy when processing gossip sections.
The change covers the case where:

1. There is a node we can't reach (in fail or pfail state).
2. We see a different address for this node, in the gossip section sent
to us by a node that, instead, is able to talk with the node we cannot
talk to.

In this case it's a good bet to switch to the address reported by this
node, since there was an address switch and it is able to talk with the
node and we are not.

However previosuly this was done in a dangerous way, by initiating an
handshake. The handshake, using the MEET packet, forces the receiver to
join our cluster, and this is not a good idea. If the node in question
really just switched address, but is the same node, it already knows about
us, so we just need to perform an address update and a reconnection.

So with this commit instead we just update the address of the node,
release the node link if any, and attempt to reconnect in the next
clusterCron() cycle.

The commit also improves debugging messages printed by Cluster during
address or ID switches.
2016-01-26 12:32:53 +01:00
antirez 22892ce043 Merge branch 'unstable' of github.com:/antirez/redis into unstable 2016-01-25 15:24:55 +01:00
antirez bc1558622a Fix memory leak in masterauth config option loading. 2016-01-25 15:24:16 +01:00
Itamar Haber 57f8230234 Removes an extra space in protected mode message 2016-01-20 17:08:28 +02:00
antirez 83b862a30e Minor MIGRATE refactoring.
Centralize cleanup of newargv in a single place.
Add more comments to help a bit following a complex function.

Related to issue #3016.
2016-01-19 09:53:04 +01:00
antirez f5a1e608cc More variadic MIGRATE fixes.
Another leak was fixed in the case of syntax error by restructuring the
allocation strategy for the two dynamic vectors.

We also make sure to always close the cached socket on I/O errors so that
all the I/O errors are handled the same, even if we had a previously
queued error of a different kind from the destination server.

Thanks to Kevin McGehee. Related to issue #3016.
2016-01-19 09:28:43 +01:00
antirez 00d3a40f82 Various fixes to MIGRATE with multiple keys.
In issue #3016 Kevin McGehee identified multiple very serious issues in
the new implementation of MIGRATE. This commit attempts to restructure
the code in oder to avoid mistakes, an analysis of the new
implementation is in progress in order to check for possible edge cases.
2016-01-18 16:49:21 +01:00
antirez fc3ca8ff87 Cluster: fix setting nodes slaveof pointer to NULL on node release.
With this commit we preserve the list of nodes that have .slaveof set
to the node, even when the node is turned into a slave, and make sure to
fix the .slaveof pointers to NULL when a node is freed from memory,
regardless of the fact it's a slave or a master.

Basically we try to remember the logical master in the current
configuration even if the logical master advertised it as a slave
already. However we still remember the associations, so that when a node
is freed we can fix them.

This should fix issue #3002.
2016-01-14 17:34:49 +01:00
antirez a411d557d4 Cluster: clarify node->slave may be NULL. 2016-01-14 11:58:31 +01:00
Jan-Erik Rediger 15dacfec6f Fix nanosecond conversion
1 microsecond = 1000 nanoseconds
1e3 = 1000
10e3 = 10000
2016-01-13 10:22:29 -07:00
antirez f984cef217 Cluster: fix rebalancing to always empty nodes.
Because of rounding error even with weight=0 sometimes a node was left
with an assigned slot.

Close #3001.
2016-01-13 17:30:53 +01:00
root 28e80bf96d fix linux compile bug 2016-01-13 00:49:28 -08:00
antirez 152e9f67f8 Cluster: redis-trib move_to_slot: don't send SETSLOT to slaves. 2016-01-12 12:16:00 +01:00
Daniel Shih e6d970534b Fix a possible race condition of sdown detection if the
connection to master/slave/sentinel decames disconnected just after the last PONG and before the next PING.
2016-01-12 17:06:47 +08:00
antirez c6e508856a Cluster: fix redis-trib reference of variable in warning. 2016-01-11 16:37:51 +01:00
antirez 02c40c9dc2 CLUSTER BUMPEPOCH initial implementation fixed. 2016-01-11 15:39:11 +01:00
antirez e4eb6c7a5d Cluster: implement redis-trib fix when slot is open without owners.
Still work to do.
2016-01-11 15:05:19 +01:00
antirez 04ae459bc2 Cluster: implement redis-trib fix for uncovered slots. 2016-01-11 15:04:35 +01:00
antirez b58796f520 Cluster: CLUSTER BUMPEPOCH introduced to help redis-trib fix.
Sometimes during "fixes" we have to setup a new configuration and assign
slots to nodes. With BUMPEPOCH we can make sure the new configuration of
the node will win if there are conflicting configurations (for example
another node is *also* claiming the same slot because the cluster is
totally messed up).
2016-01-11 15:01:14 +01:00
antirez 524be1e465 Cluster: don't allow CLUSTER SETSLOT with slaves. 2016-01-11 15:00:45 +01:00
antirez f43c794b0b Scripting: handle trailing comments.
This fix, provided by Paul Kulchenko (@pkulchenko), allows the Lua
scripting engine to evaluate statements with a trailing comment like the
following one:

    EVAL "print() --comment" 0

Lua can't parse the above if the string does not end with a newline, so
now a final newline is always added automatically. This does not change
the SHA1 of scripts since the SHA1 is computed on the body we pass to
EVAL, without the other code we add to register the function.

Close #2951.
2016-01-08 15:44:21 +01:00
antirez e15e518a67 Allow MIGRATE to always be called on local keys for open slots.
Extend the MIGRATE extra freedom to be able to be called in the context
of the local slot, anytime there is a slot open in one or the other
direction (importing or migrating). This is useful for redis-trib to fix
the cluster when it has in an odd state.

Thix fix allows "redis-trib fix" to make its work in certain cases where
previously an error was reported.
2016-01-08 15:04:16 +01:00
antirez 36704d653b Fix typos & grammar in clusterBumpConfigEpochWithoutConsensus() comment. 2016-01-08 12:07:54 +01:00
antirez 7c1a5ff3ce Lua debugger: support direct calls to SCRIPT DEBUG in redis-cli.
Previously it was possible to activate a debugging session only using
the --ldb option in redis-cli. Now calling SCRIPT DEBUG can also
activate the debugging mode without putting the redis-cli in a
desynchronized state.

Related to #2952.
2016-01-08 09:43:16 +01:00
antirez a75aa4bf92 Lua debugger: fix crash printing nested or deep objects.
Example of offending code:

> script debug yes
OK
> eval "local a = {1} a[1] = a\nprint(a)" 0
1) * Stopped at 1, stop reason = step over
2) -> 1   local a = {1} a[1] = a
> next
1) * Stopped at 2, stop reason = step over
2) -> 2   print(a)
> print

... server crash ...

Close #2955.
2016-01-08 09:14:13 +01:00
antirez 1e7a8f8221 Another typo in protected mode error message. 2016-01-07 22:42:43 +01:00
antirez 08c7bba32a Fix protected mode error message typo. 2016-01-07 14:35:10 +01:00
antirez edd4d555df New security feature: Redis protected mode.
An exposed Redis instance on the internet can be cause of serious
issues. Since Redis, by default, binds to all the interfaces, it is easy
to forget an instance without any protection layer, for error.

Protected mode try to address this feature in a soft way, providing a
layer of protection, but giving clues to Redis users about why the
server is not accepting connections.

When protected mode is enabeld (the default), and if there are no
minumum hints about the fact the server is properly configured (no
"bind" directive is used in order to restrict the server to certain
interfaces, nor a password is set), clients connecting from external
intefaces are refused with an error explaining what to do in order to
fix the issue.

Clients connecting from the IPv4 and IPv6 lookback interfaces are still
accepted normally, similarly Unix domain socket connections are not
restricted in any way.
2016-01-07 13:00:14 +01:00
antirez 00d637f2cc Cluster: don't send -ASK to MIGRATE.
For non existing keys, we don't want to send -ASK redirections to
MIGRATE, since when moving slots from the migrating node to the
importing node, we want just to ignore keys that are no longer there.
They may be expired or deleted between the GETKEYSINSLOT call and the
MIGRATE call. Otherwise this causes an error during migrations with
redis-trib (or equivalent cluster management tools).
2016-01-06 12:14:49 +01:00
antirez 190babe2df redis-trib: Remove duplicated key in hash initialization. 2016-01-02 13:13:32 +01:00
Salvatore Sanfilippo 075ea1646f Merge pull request #2954 from pkulchenko/debug-table-pretty-printing
Update pretty printing during debugging to generate valid Lua code for tables
2015-12-22 09:00:36 +01:00
Salvatore Sanfilippo f054b4ac47 Merge pull request #2957 from pkulchenko/debug-userdata-pretty-printing
Update pretty printing in debugging to generate valid Lua code for userdata-like types.
2015-12-22 08:59:48 +01:00
antirez 80b70371e8 Cluster: rebalance now supports --threshold option. 2015-12-18 15:51:39 +01:00
antirez 628af70214 Cluster: redis-trib reshard / rebalance --pipeline support. 2015-12-18 12:27:14 +01:00
antirez bb21537596 Fix a race that may lead to the active (slave) client to be freed.
In issue #2948 a crash was reported in processCommand(). Later Oran Agra
(@oranagra) traced the bug (in private chat) in the following sequence
of events:

1. Some maxmemory is set.
2. The slave is the currently active client and is executing PING or
   REPLCONF or whatever a slave can send to its master.
3. freeMemoryIfNeeded() is called since maxmemory is set.
4. flushSlavesOutputBuffers() is called by freeMemoryIfNeeded().
5. During slaves buffers flush, a write error could be encoutered in
   writeToClient() or sendReplyToClient() depending on the version of
   Redis. This will trigger freeClient() against the currently active
   client, so a segmentation fault will likely happen in
   processCommand() immediately after the call to freeMemoryIfNeeded().

There are different possible fixes:

1. Add flags to writeToClient() (recent versions code base) so that
   we can ignore the write errors, and use this flag in
   flushSlavesOutputBuffers(). However this is not simple to do in older
   versions of Redis.
2. Use freeClientAsync() during write errors. This works but changes the
   current behavior of releasing clients ASAP when possible. Normally
   we write to clients during the normal event loop processing, in the
   writable client, where there is no active client, so no care must be
   taken.
3. The fix of this commit: to detect that the current client is no
   longer valid. This fix is a bit "ad-hoc", but works across all the
   versions and has the advantage of not changing the remaining
   behavior. Only alters what happens during this race condition,
   hopefully.
2015-12-17 09:39:43 +01:00
antirez 218e522c82 Fix processCommand() comment about return value. 2015-12-17 09:22:16 +01:00
antirez a1c9c05e17 Hopefully better memory test on crash.
The old test, designed to do a transformation on the bits that was
invertible, in order to avoid touching the original memory content, was
not effective as it was redis-server --test-memory. The former often
reported OK while the latter was able to spot the error.

So the test was substituted with one that may perform better, however
the new one must backup the memory tested, so it tests memory in small
pieces. This limits the effectiveness because of the CPU caches. However
some attempt is made in order to trash the CPU cache between the fill
and the check stages, but not for the addressing test unfortunately.

We'll see if this test will be able to find errors where the old failed.
2015-12-16 17:41:22 +01:00
antirez b9aeb98156 Suppress harmless warnings. 2015-12-16 12:36:32 +01:00
antirez ac8f4a6af9 memtest.c now can be called as API in non interactive mode. 2015-12-16 12:31:42 +01:00
antirez 30f057d88f Crash report format improvements. 2015-12-16 12:14:55 +01:00
Paul Kulchenko b754c8e18b Update pretty printing in debugging to generate valid Lua code for userdata-like types. 2015-12-15 20:24:41 -08:00
Paul Kulchenko 3969e9d3df Update pretty printing in debugging to generate valid Lua code for tables. 2015-12-15 18:15:39 -08:00
Paul Kulchenko 2f3f3fd5e0 Update global protection error message to fix a typo. 2015-12-15 18:13:09 -08:00
antirez 6db8e8569d Log address causing SIGSEGV. 2015-12-15 18:00:29 +01:00
antirez 8f8c3992ac Cluster: allows abbreviated node IDs with rebalance --weight option. 2015-12-15 16:08:00 +01:00
antirez 9df1ae8808 Cluster: rebalancing option --simulate, and a fix. 2015-12-15 15:48:49 +01:00
antirez cba1c29580 Cluster: redis-trib rebalance initial implementation. 2015-12-15 12:54:40 +01:00
antirez 3782902bec Initial implementation of redis-trib info subcommand. 2015-12-14 18:14:52 +01:00
antirez f0b168e894 Cluster: redis-trib: use variadic MIGRATE.
We use the new variadic/pipelined MIGRATE for faster migration.
Testing is not easy because to see the time it takes for a slot to be
migrated requires a very large data set, but even with all the overhead
of migrating multiple slots and to setup them properly, what used to
take 4 seconds (1 million keys, 200 slots migrated) is now 1.6 which is
a good improvement. However the improvement can be a lot larger if:

1. We use large datasets where a single slot has many keys.
2. By moving more than 10 keys per iteration, making this configurable,
   which is planned.

Close #2710
Close #2711
2015-12-11 18:12:56 +01:00
antirez 4e252e4c09 MIGRATE: Fix key extraction for new form. 2015-12-11 18:09:01 +01:00
antirez ac0a731057 MIGRATE: Fix new argument rewriting refcount handling. 2015-12-11 14:26:41 +01:00
antirez d85fc1e9cf MIGRATE: fix replies processing and argument rewriting.
We need to process replies after errors in order to delete keys
successfully transferred. Also argument rewriting was fixed since
it was broken in several ways. Now a fresh argument vector is created
and set if we are acknowledged of at least one key.
2015-12-11 14:04:47 +01:00
antirez 9ebf7a6776 Pipelined multiple keys MIGRATE. 2015-12-11 13:38:26 +01:00
antirez e7945cf839 Cluster: redis-trib migrate default timeout set to 60 sec. 2015-12-11 11:00:27 +01:00
daniele 3d254e05f4 redis-trib.rb: --timeout XXXXX option added to fix and reshard commands. Defaults to 15000 milliseconds 2015-12-11 10:59:08 +01:00
antirez adc2fe6993 Cluster: replica migration with delay.
We wait a fixed amount of time (5 seconds currently) much greater than
the usual Cluster node to node communication latency, before migrating.
This way when a failover occurs, before detecting the new master as a
target for migration, we give the time to its natural slaves (the slaves
of the failed over master) to announce they switched to the new master,
preventing an useless migration operation.
2015-12-11 09:19:06 +01:00
antirez 4159055f83 Remove debugging message left there for error. 2015-12-10 08:56:33 +01:00
antirez 69897f5f30 unlinkClient(): clear flags according to ops performed. 2015-12-09 23:06:44 +01:00
antirez e0f22df995 Fix replicas migration by adding a new flag.
Some time ago I broken replicas migration (reported in #2924).
The idea was to prevent masters without replicas from getting replicas
because of replica migration, I remember it to create issues with tests,
but there is no clue in the commit message about why it was so
undesirable.

However my patch as a side effect totally ruined the concept of replicas
migration since we want it to work also for instances that, technically,
never had slaves in the past: promoted slaves.

So now instead the ability to be targeted by replicas migration, is a
new flag "migrate-to". It only applies to masters, and is set in the
following two cases:

1. When a master gets a slave, it is set.
2. When a slave turns into a master because of fail over, it is set.

This way replicas migration targets are only masters that used to have
slaves, and slaves of masters (that used to have slaves... obviously)
and are promoted.

The new flag is only internal, and is never exposed in the output nor
persisted in the nodes configuration, since all the information to
handle it are implicit in the cluster configuration we already have.
2015-12-09 23:03:18 +01:00
antirez acc2336fd1 Centralize slave replication handshake aborting.
Now we have a single function to call in any state of the slave
handshake, instead of using different functions for different states
which is error prone. Change performed in the context of issue #2479 but
does not fix it, since should be functionally identical to the past.
Just an attempt to make replication.c simpler to follow.
2015-12-03 10:38:56 +01:00
antirez 96628cc40d fix sprintf and snprintf format string
There are some cases of printing unsigned integer with %d conversion
specificator and vice versa (signed integer with %u specificator).

Patch by Sergey Polovko. Backported to Redis from Disque.
2015-11-28 09:05:41 +01:00
antirez e6a5117426 Fix typo in prepareClientToWrite() comment. 2015-11-27 16:17:21 +01:00
antirez c2c68c50ef Merge branch 'unstable' of github.com:/antirez/redis into unstable 2015-11-27 16:12:23 +01:00
antirez da82723858 Handle wait3() errors.
My guess was that wait3() with WNOHANG could never return -1 and an
error. However issue #2897 may possibly indicate that this could happen
under non clear conditions. While we try to understand this better,
better to handle a return value of -1 explicitly, otherwise in the
case a BGREWRITE is in progress but wait3() returns -1, the effect is to
match the first branch of the if/else block since server.rdb_child_pid
is -1, and call backgroundSaveDoneHandler() without a good reason, that
will, in turn, crash the Redis server with an assertion.
2015-11-27 16:11:05 +01:00
Itamar Haber 36801f7e43 Revert Lua's `redis.LOG_<level>` to original
Fixes #2898
2015-11-27 15:55:38 +02:00
antirez a0d41e51c2 Redis Cluster: hint about validity factor when slave can't failover. 2015-11-27 08:59:17 +01:00
antirez 4b0b28b469 Lua debugger: infinite loop detection. 2015-11-18 10:23:49 +01:00
antirez 1f35f2dd5a Lua debugger: fix trace command infinite loop.
Thanks to Itamar Haber for bug report and test case to reproduce.
2015-11-17 16:24:27 +01:00
antirez 6604e04260 Lua debugger: redis-cli: allow restart after end of session. 2015-11-17 15:43:24 +01:00
antirez 58573f1dd2 Lua debugger: redis-cli can restart Lua debugging sessions. 2015-11-17 15:43:24 +01:00
antirez 0cc19174f9 Lua debugger: maxlen command implemented.
Let the user control the replies truncation.
2015-11-17 15:43:24 +01:00
antirez c560c645e9 Lua debugger: trace command implemented. 2015-11-17 15:43:24 +01:00
antirez 22959e07dc Lua debugger: redis-cli: show compile errors in LDB mode. 2015-11-17 15:43:24 +01:00
antirez 70a51694de Lua debugger: print without args show all local vars. 2015-11-17 15:43:23 +01:00
antirez fb53459ce8 Lua debugger: default behavior of "list" command changed.
Now it lists code around the current position by default. Can list any
other part using other arguments, but a new "whole" command was added in
order to show the whole source code easily.
2015-11-17 15:43:23 +01:00
antirez 6de2306add Lua debugger: redis-cli error when --ldb is without --eval.
Otherwise the result is a messed CLI without prompt.
Thanks to Itamar Haber for reporting this issue.
2015-11-17 15:43:23 +01:00
antirez e57cccdefb Lua debugger: use sds_malloc() to allocate eval cli array.
Redis-cli handles the debugger "eval" command in a special way since
sdssplitargs() would not be ok: we need to send the Redis debugger the
whole Lua script without any parsing. However in order to later free the
argument vector inside redis-cli using just sdsfreesplitres(), we need
to allocate the array of SDS pointers using the same allocator SDS is
using, that may differ to what Redis is using.

So now a newer version of SDS exports sds_malloc() and other allocator
functions to give access, to the program it is linked to, the allocator
used internally by SDS.
2015-11-17 15:43:23 +01:00
antirez 34aadf79c3 Lua debugging: fix error message for SCRIPT DEBUG.
"async" -> "sync".

Thanks to Itamar Haber for reporting.
2015-11-17 15:43:23 +01:00
antirez 3d24cd6bf8 Lua debugger: reply +OK to SCRIPT DEBUG no.
Thanks to Itamar Haber for reporting.
2015-11-17 15:43:23 +01:00
antirez 333547dab6 Lua debugger: call wait3() if there are pending forked debugging sessions. 2015-11-17 15:43:23 +01:00
antirez 01636435fa Lua debugger: abort implemented. 2015-11-17 15:43:23 +01:00
antirez d99ce09343 Lua debugger: ldbSendLogs() memory leak fixed. 2015-11-17 15:43:23 +01:00
antirez 87672adee2 Lua debugger: better support for synchronous mode. 2015-11-17 15:43:22 +01:00
antirez 7be9170585 Lua debugger: handle forked sessions children during shutdown. 2015-11-17 15:43:22 +01:00
antirez 3ab0b4d6d0 Lua debugger: inform user changes are rolled back.
When redis-cli will support the synchronous mode, this will not be
printed.
2015-11-17 15:43:22 +01:00
antirez 56d9bb8c55 Lua debugger: fix help typo, beark -> break. 2015-11-17 15:43:22 +01:00
antirez e386cd8ccf Lua debugger: clear end of session protocol.
When the debugger exits now it produces an <endsession> tag that informs
redis-cli (or other debugging clients) that the session terminated.
This way the client knows there is yet another reply to read (the one of
the EVAL script itself), and can switch to non-debugging mode ASAP.
2015-11-17 15:43:22 +01:00
antirez 7492237c3c Lua debugger: redis.debug() implemented. 2015-11-17 15:43:22 +01:00
antirez 23a4d70e56 Lua debugger: redis-cli, mark end of debugging session. 2015-11-17 15:43:22 +01:00
antirez cd112db0ae Lua debugger: removing breakpoints now works. 2015-11-17 15:43:22 +01:00
antirez a076e421e9 Lua debugger: redis command implemented. 2015-11-17 15:43:22 +01:00
antirez e6eb6eadec Lua debugger: try to eval as expression first.
It's handly to just eval "5+5" without the return and see it printed on
the screen as result. However prepending "return" does not always result
into valid Lua code. So what we do is to exploit a common Lua community
trick of trying to compile with return prepended, and if compilation
fails then it's not an expression that can be returned, so we try again
without prepending "return". Works great apparently.
2015-11-17 15:43:21 +01:00
antirez 1f8fdafe65 Lua debugger: much better Lua values pretty printer. 2015-11-17 15:43:21 +01:00
antirez f4805800dc Lua debugger: print now handles ARGV and KEYS. 2015-11-17 15:43:21 +01:00
antirez 36392dd867 Lua debugger: added comment about helper functions. 2015-11-17 15:43:21 +01:00
antirez 3a04cb05ee Lua debugger: redis.breakpoint() implemented. 2015-11-17 15:43:21 +01:00
antirez cf4700bda4 Lua debugger: output improvements, eval command. 2015-11-17 15:43:21 +01:00
antirez 1f8d614423 Lua debugger: breakpoints. 2015-11-17 15:43:21 +01:00
antirez 5c4f492844 Lua debugger: ability to show local vars content. 2015-11-17 15:43:21 +01:00
antirez 5417217c87 Lua debugger: log Redis commands. List command. 2015-11-17 15:43:21 +01:00
antirez d3d1fa9437 Lua debugger: initial REPL. 2015-11-17 15:43:20 +01:00
antirez c494db89b5 Lua debugger: foundations implemented. 2015-11-17 15:43:20 +01:00
antirez 7cfdccd94e Remove "s" flag for MIGRATE in command table.
Maybe there are legitimate use cases for MIGRATE inside Lua scripts, at
least for now. When the command will be executed in an asynchronous
fashion (planned) it is possible we'll no longer be able to permit it
from within Lua scripts.
2015-11-17 15:43:13 +01:00
antirez f3dd472e97 Update redis-cli help and the script to generate it. 2015-11-17 15:38:34 +01:00
antirez 0cb66fa156 Fix MIGRATE entry in command table.
Thanks to Oran Agra (@oranagra) for reporting. Key extraction would not
work otherwise and it does not make sense to take wrong data in the
command table.
2015-11-17 15:33:09 +01:00
antirez 8a0258aa43 AOF: rewriting child killed by SIGUSR1 is not an error. 2015-11-13 09:32:20 +01:00
antirez 54f5ecfd49 call() deserves a good top-comment. 2015-11-10 13:31:25 +01:00
Salvatore Sanfilippo c950facf43 Merge pull request #2848 from badboy/removed-printf
Remove printf
2015-11-09 18:06:41 +01:00
antirez 87a12a6085 Best effort flush of slave buffers before SHUTDOWN. 2015-11-09 17:26:58 +01:00
antirez b719eedfc6 Use clientHasPendingReplies() in flushSlavesOutputBuffers()
The old version only flushed data to slaves if there were strings
pending in the client->reply list. Now also static buffers are flushed.
Does not help to free memory (which is the only use we have right now in
the fuction), but is more correct conceptually, and may be used in
other contexts.
2015-11-09 17:07:46 +01:00
antirez 1b5d24eb2b Scripting: fix redis.call() error reporting.
Arguments arity and arguments type error of redis.call() were not
reported correctly to Lua, so the command acted in this regard like
redis.pcall(), but just for two commands. Redis.call() should always
raise errors instead.
2015-11-09 11:43:51 +01:00
antirez 111d24f246 Fix error reply in subscribed Pub/Sub mode.
PING is now a valid command to issue in this context.
2015-11-09 11:10:53 +01:00
antirez cd8f19e9ca Initialize all Lua scripting related things into scripting.c 2015-11-05 11:37:39 +01:00
antirez 9aa1f94449 scripting.c source code better organized into sections. 2015-11-05 10:37:10 +01:00
antirez 71aa9b75f2 Fix HINCRBYFLOAT to work with long doubles.
During the refactoring needed for lazy free, specifically the conversion
of t_hash from struct robj to plain SDS strings, HINCRBFLOAT was
accidentally moved away from long doubles to doubles for internal
processing of increments and formatting.

The diminished precision created more obvious artifacts in the way small
numbers are formatted once we convert from decimal number in radix 10 to
double and back to its string in radix 10.

By using more precision, we now have less surprising results at least
with small numbers like "1.23", exactly like in the previous versions of
Redis.

See issue #2846.
2015-11-04 17:16:34 +01:00
Jan-Erik Rediger 35afefc7cb Remove printf 2015-11-04 11:48:14 +01:00
antirez 77362b9027 Dependencies updated. 2015-10-30 12:15:07 +01:00
antirez f26072eb66 More reliable DEBUG loadaof.
Make sure to flush the AOF output buffers before reloading.
Result: less false timing related false positives on AOF tests.
2015-10-30 12:06:09 +01:00
antirez ff6d296000 Scripting: ability to turn on Lua commands style replication globally.
Currently this feature is only accessible via DEBUG for testing, since
otherwise depending on the instance configuration a given script works
or is broken, which is against the Redis philosophy.
2015-10-30 12:06:09 +01:00
antirez ebaa9226ee Scripting: fix error reporting of many Redis provided functions. 2015-10-30 12:06:09 +01:00
antirez 2dabf82d5f Fix call() FORCE_REPL/AOF flags setting.
This commit also inverts two stanzas of the code just becuase they are
more logical like that, not because currently it makes any difference.
2015-10-30 12:06:08 +01:00
antirez 514a234722 Lua script selective replication fixes. 2015-10-30 12:06:08 +01:00
antirez a3e8de0430 Lua script selective replication WIP. 2015-10-30 12:06:08 +01:00
antirez fc38235664 Scripting: single commands replication mode implemented.
By calling redis.replicate_commands(), the scripting engine of Redis
switches to commands replication instead of replicating whole scripts.
This is useful when the script execution is costly but only results in a
few writes performed to the dataset.

Morover, in this mode, it is possible to call functions with side
effects freely, since the script execution does not need to be
deterministic: anyway we'll capture the outcome from the point of view
of changes to the dataset.

In this mode math.random() returns different sequences at every call.

If redis.replicate_commnads() is not called before any other write, the
command returns false and sticks to whole scripts replication instead.
2015-10-30 12:06:08 +01:00
antirez cdda6748c2 call(): selective ability to prevent propagation on AOF / slaves. 2015-10-30 12:06:08 +01:00
antirez 9dd3d2e9bd call(): don't inherit CLIENT_PREVENT_PROP + minor refactoring. 2015-10-30 12:06:08 +01:00
antirez 86f0a2ee87 CLIENT REPLY command implemented: ON, OFF and SKIP modes.
Sometimes it can be useful for clients to completely disable replies
from the Redis server. For example when the client sends fire and forget
commands or performs a mass loading of data, or in caching contexts
where new data is streamed constantly. In such contexts to use server
time and bandwidth in order to send back replies to clients, which are
going to be ignored, is a shame.

Multiple mechanisms are possible to implement such a feature. For
example it could be a feature of MULTI/EXEC, or a command prefix
such as "NOREPLY SADD myset foo", or a different mechanism that allows
to switch on/off requests using the CLIENT command.

The MULTI/EXEC approach has the problem that transactions are not
strictly part of the no-reply semantics, and if we want to insert a lot
of data in a bulk way, creating a huge MULTI/EXEC transaction in the
server memory is bad.

The prefix is the best in this specific use case since it does not allow
desynchronizations, and is pretty clear semantically. However Redis
internals and client libraries are not prepared to handle this
currently.

So the implementation uses the CLIENT command, providing a new REPLY
subcommand with three options:

    CLIENT REPLY OFF disables the replies, and does not reply itself.
    CLIENT REPLY ON re-enables the replies, replying +OK.
    CLIENT REPLY SKIP only discards the reply of the next command, and
                      like OFF does not reply anything itself.

The reason to add the SKIP command is that it allows to have an easy
way to send conceptually "single" commands that don't need a reply
as the sum of two pipelined commands:

    CLIENT REPLY SKIP
    SET key value

Note that CLIENT REPLY ON replies with +OK so it should be used when
sending multiple commands that don't need a reply. However since it
replies with +OK the client can check that the connection is still
active and all the previous commands were received.

This is currently just into Redis "unstable" so the proposal can be
modified or abandoned based on users inputs.
2015-10-21 20:43:37 +02:00
Salvatore Sanfilippo bdcb145566 Merge pull request #2810 from dwlt/add-copy-replace-parameters-to-redis-trib-import
Add copy & replace parameters to redis-trib import
2015-10-15 13:02:35 +02:00
antirez ed6228851c PR 2813 fix ported to unstable. 2015-10-15 10:20:09 +02:00
David Thomson 8a85ad2d1c Add back blank line 2015-10-14 06:58:36 +01:00
David Thomson 03d0de7cdf Update import command to optionally use copy and replace parameters 2015-10-14 06:56:14 +01:00
antirez 35a0c772b5 DEBUG RESTART/CRASH-AND-RECOVER [delay] implemented. 2015-10-13 11:12:25 +02:00
antirez 73427462ed Server: restartServer() API.
This new function is able to restart the server "in place". The current
Redis process executes the same executable it was executed with, using
the same arguments and configuration file.
2015-10-13 11:02:35 +02:00
antirez c372a59520 Cluster: redis-trib fix, coverage for migrating=1 case.
Kinda related to #2770.
2015-10-09 16:15:58 +02:00
antirez f29e384853 Fix extractLongLatOrReply() sanity check conditionals.
the check for lat/long valid ranges were performed inside the for loop,
two times instead of one, and the first time when the second element of
the array, xy[1], was yet not populated. This resulted into issue #2799.

Close issue #2799.
2015-10-07 22:27:18 +02:00
antirez 319d180136 Fix GEORADIUS COUNT option arity checks. 2015-10-06 09:25:28 +02:00
antirez 880c606a7c Lazyfree options implemented in the configuration. 2015-10-05 12:11:27 +02:00
antirez 252cfa0a39 Lazyfree: cond vars to enabled/disable it based on DEL context. 2015-10-02 15:27:57 +02:00
antirez 5359696796 Fixed a bug in the emptyDb() new implementation. 2015-10-01 13:02:26 +02:00
antirez ecdbc333a3 FLUSHDB and FLUSHALL ASYNC option implemented. 2015-10-01 13:02:26 +02:00
antirez 1f26a9468f Lazyfree: pending objects count in INFO output. 2015-10-01 13:02:26 +02:00
antirez c69c6c80fb Lazyfree: ability to free whole DBs in background. 2015-10-01 13:02:26 +02:00
antirez b08c36c5f2 Lazyfree: keep count of objects to free. 2015-10-01 13:02:25 +02:00
antirez c7b46a4719 zmalloc.c converted to use atomicvar.h. 2015-10-01 13:02:25 +02:00
antirez 7e5d69019e Atomic vars implemented in a more general way.
We have them into zmalloc.c, but this is going to replace that
implementation, so that it's possible to use the same idea everywhere
inside the code base.
2015-10-01 13:02:25 +02:00
antirez 7af4eeb745 Lazyfree: incremental removed, only threaded survived. 2015-10-01 13:02:25 +02:00
antirez 9253d85073 Threaded lazyfree WIP #1. 2015-10-01 13:02:25 +02:00
antirez 4d50d691e3 bio.c: new API bioWaitStepOfType(). 2015-10-01 13:02:25 +02:00
antirez 1dab60df81 Hash new implementation memleaks fixed. 2015-10-01 13:02:25 +02:00
antirez 97ba4e3886 Lazyfree: Hash converted to use plain SDS WIP 5. 2015-10-01 13:02:25 +02:00
antirez 974514b936 Lazyfree: Hash converted to use plain SDS WIP 4. 2015-10-01 13:02:25 +02:00
antirez 4a18352877 Lazyfree: Hash converted to use plain SDS WIP 3. 2015-10-01 13:02:24 +02:00
antirez 777396aeba Lazyfree: Hash converted to use plain SDS WIP 2. 2015-10-01 13:02:24 +02:00
antirez 1c247556c6 Lazyfree: Hash converted to use plain SDS WIP 1. 2015-10-01 13:02:24 +02:00
antirez afc4b9241c DEBUG DIGEST Set type memory leak fixed. 2015-10-01 13:02:24 +02:00
antirez 34e489cb8c SORT memory leak fixed. 2015-10-01 13:02:24 +02:00
antirez a7c5be18a8 Lazyfree: Sorted sets convereted to plain SDS. (several commits squashed) 2015-10-01 13:02:24 +02:00
antirez 86d48efbfd Lazyfree: Convert Sets to use plains SDS (several commits squashed). 2015-10-01 13:02:24 +02:00
antirez 4ff3c17a20 Lazyfree: client output buffers no longer use Redis Objects. 2015-10-01 13:02:24 +02:00
antirez 0c05436cef Lazyfree: a first implementation of non blocking DEL. 2015-10-01 13:00:19 +02:00
antirez 712ea7296d Call writeToClient() directly instead of the write handler. 2015-09-30 17:41:52 +02:00
antirez 01c08b5089 Fix processEventsWhileBlocked() to handle PENDING_WRITE clients.
After the introduction of the list with clients with pending writes, to
process clients incrementally outside of the event loop we also need to
process the pending writes list.
2015-09-30 17:23:44 +02:00
antirez 1e7153831d Refactoring: unlinkClient() added to lower freeClient() complexity. 2015-09-30 17:10:03 +02:00
antirez fdb3be939e Refactoring: new function to test if client has pending output. 2015-09-30 16:41:48 +02:00
antirez 825f65d2bd Reverse list of clients with pending writes.
May potentially improve locality... not exactly clear if this makes a
difference or not. But for sure is harmless.
2015-09-30 16:29:42 +02:00
antirez 063ecbd5e5 writeToClient(): don't remove write handler if not needed. 2015-09-30 16:29:42 +02:00
antirez b741a90ce9 handleClientsWithPendingWrites(): detect dead clients. 2015-09-30 16:29:42 +02:00
antirez 481a0db315 Move handleClientsWithPendingWrites() in networking.c. 2015-09-30 16:29:42 +02:00
antirez 1c7d87df0c Avoid installing the client write handler when possible. 2015-09-30 16:29:41 +02:00
antirez d1b6a17d1e redis-cli pipe mode: don't stay in the write loop forever.
The code was broken and resulted in redis-cli --pipe to, most of the
times, writing everything received in the standard input to the Redis
connection socket without ever reading back the replies, until all the
content to write was written.

This means that Redis had to accumulate all the output in the output
buffers of the client, consuming a lot of memory.

Fixed thanks to the original report of anomalies in the behavior
provided by Twitter user @fsaintjacques.
2015-09-30 16:24:21 +02:00
antirez 622366aa74 Mark version of unstable branch in an unique way. 2015-09-29 17:30:24 +02:00
antirez 3c23b5ffd0 GEORADIUS: Don't report duplicates when radius is huge.
Georadius works by computing the center + neighbors squares covering all
the area of the specified position and radius. Then a distance filter is
used to remove elements which are actually outside the range.

When a huge radius is used, like 5000 km or more, adjacent neighbors may
collide and be the same, leading to the reporting of the same element
multiple times. This only happens in the edge case of huge radius but is
not ideal.

A robust but slow solution would involve qsorting the range to remove
all the duplicates. However since the collisions are only in adjacent
boxes, for the way they are ordered in the code, it is much faster to
just check if the current box is the same as the previous one processed.

This commit adds a regression test for the bug.

Fixes #2767.
2015-09-14 23:10:50 +02:00
antirez 4fec5ee165 MOVE re-add TTL check fixed.
getExpire() returns -1 when no expire exists.

Related to #2765.
2015-09-14 12:34:17 +02:00
antirez f529a01c1b MOVE now can move TTL metadata as well.
MOVE was not able to move the TTL: when a key was moved into a different
database number, it became persistent like if PERSIST was used.

In some incredible way (I guess almost nobody uses Redis MOVE) this bug
remained unnoticed inside Redis internals for many years.
Finally Andy Grunwald discovered it and opened an issue.

This commit fixes the bug and adds a regression test.

Close #2765.
2015-09-14 12:30:00 +02:00
antirez 33769f840c Sentinel: command arity check added where missing. 2015-09-08 09:27:43 +02:00
Salvatore Sanfilippo 0c62d95538 Merge pull request #2695 from rogerlz/unstable
redis-sentinel crash if ckquorum command is executed without args
2015-09-08 09:24:45 +02:00
antirez 8e55537459 Undo slaves state change on failed rdbSaveToSlavesSockets().
As Oran Agra suggested, in startBgsaveForReplication() when the BGSAVE
attempt returns an error, we scan the list of slaves in order to remove
them since there is no way to serve them currently.

However we check for the replication state BGSAVE_START, which was
modified by rdbSaveToSlaveSockets() before forking(). So when fork fails
the state of slaves remain BGSAVE_END and no cleanup is performed.

This commit fixes the problem by making rdbSaveToSlavesSockets() able to
undo the state change on fork failure.
2015-09-07 16:09:23 +02:00
ubuntu 11381b09d9 SCAN iter parsing changed from atoi to chartoull 2015-09-07 11:20:52 +00:00
kmiku7 413d8239df fix boundary case for _dictNextPower 2015-08-23 16:47:42 +08:00
antirez d036abe27d Log client details on SLAVEOF command having an effect. 2015-08-21 15:29:07 +02:00
antirez f18e5b634d startBgsaveForReplication(): handle waiting slaves state change.
Before this commit, after triggering a BGSAVE it was up to the caller of
startBgsavForReplication() to handle slaves in WAIT_BGSAVE_START in
order to update them accordingly. However when the replication target is
the socket, this is not possible since the process of updating the
slaves and sending the FULLRESYNC reply must be coupled with the process
of starting an RDB save (the reason is, we need to send the FULLSYNC
command and spawn a child that will start to send RDB data to the slaves
ASAP).

This commit moves the responsibility of handling slaves in
WAIT_BGSAVE_START to startBgsavForReplication() so that for both
diskless and disk-based replication we have the same chain of
responsiblity. In order accomodate such change, the syncCommand() also
needs to put the client in the slave list ASAP (just after the initial
checks) and not at the end, so that startBgsavForReplication() can find
the new slave alrady in the list.

Another related change is what happens if the BGSAVE fails because of
fork() or other errors: we now remove the slave from the list of slaves
and send an error, scheduling the slave connection to be terminated.

As a side effect of this change the following errors found by
Oran Agra are fixed (thanks!):

1. rdbSaveToSlavesSockets() on failed fork will get the slaves cleaned
up, otherwise they remain in a wrong state forever since we setup them
for full resync before actually trying to fork.

2. updateSlavesWaitingBgsave() with replication target set as "socket"
was broken since the function changed the slaves state from
WAIT_BGSAVE_START to WAIT_BGSAVE_END via
replicationSetupSlaveForFullResync(), so later rdbSaveToSlavesSockets()
will not find any slave in the right state (WAIT_BGSAVE_START) to feed.
2015-08-20 17:39:48 +02:00
antirez bea1259190 slaveTryPartialResynchronization and syncWithMaster: better synergy.
It is simpler if removing the read event handler from the FD is up to
slaveTryPartialResynchronization, after all it is only called in the
context of syncWithMaster.

This commit also makes sure that on error all the event handlers are
removed from the socket before closing it.
2015-08-07 12:04:37 +02:00
antirez 88c716a0f5 syncWithMaster(): non blocking state machine. 2015-08-06 18:12:20 +02:00
antirez 55cb64bbfb flushSlavesOutputBuffers(): details clarified via comments.
Talking with @oranagra we had to reason a little bit to understand if
this function could ever flush the output buffers of the wrong slaves,
having online state but actually not being ready to receive writes
before the first ACK is received from them (this happens with diskless
replication).

Next time we'll just read this comment.
2015-08-06 15:08:54 +02:00
antirez ce5761e061 startBgsaveForReplication(): log what you really do. 2015-08-06 09:49:38 +02:00
antirez fd08839a3a Client structure comments improved. 2015-08-06 09:41:11 +02:00
antirez 3e6d4d599a Replication: add REPLCONF CAPA EOF support.
Add the concept of slaves capabilities to Redis, the slave now presents
to the Redis master with a set of capabilities in the form:

    REPLCONF capa SOMECAPA capa OTHERCAPA ...

This has the effect of setting slave->slave_capa with the corresponding
SLAVE_CAPA macros that the master can test later to understand if it
the slave will understand certain formats and protocols of the
replication process. This makes it much simpler to introduce new
replication capabilities in the future in a way that don't break old
slaves or masters.

This patch was designed and implemented together with Oran Agra
(@oranagra).
2015-08-06 09:23:23 +02:00
antirez 7ab3af0edc Fix synchronous readline "\n" handling.
Our function to read a line with a timeout handles newlines as requests
to refresh the timeout, however the code kept subtracting the buffer
size left every time a newline was received, for a bug in the loop
logic. Fixed by this commit.
2015-08-05 16:51:50 +02:00
antirez 55ba772703 Fix replication slave pings period.
For PINGs we use the period configured by the user, but for the newlines
of slaves waiting for an RDB to be created (including slaves waiting for
the FULLRESYNC reply) we need to ping with frequency of 1 second, since
the timeout is fixed and needs to be refreshed.
2015-08-05 16:49:16 +02:00
antirez 54ece2c583 Remove slave state change handled by replicationSetupSlaveForFullResync(). 2015-08-05 13:58:56 +02:00
antirez 15de6b108b Make sure we re-emit SELECT after each new slave full sync setup.
In previous commits we moved the FULLRESYNC to the moment we start the
BGSAVE, so that the offset we provide is the right one. However this
also means that we need to re-emit the SELECT statement every time a new
slave starts to accumulate the changes.

To obtian this effect in a more clean way, the function that sends the
FULLRESYNC reply was overloaded with a more important role of also doing
this and chanigng the slave state. So it was renamed to
replicationSetupSlaveForFullResync() to better reflect what it does now.
2015-08-05 13:34:46 +02:00
antirez a5a06a8ecd Don't send SELECT to slaves in WAIT_BGSAVE_START state. 2015-08-05 11:23:22 +02:00
antirez 62b5c60ead syncCommand() comments improved. 2015-08-05 08:41:57 +02:00
antirez 292fec058a PSYNC initial offset fix.
This commit attempts to fix a bug involving PSYNC and diskless
replication (currently experimental) found by Yuval Inbar from Redis Labs
and that was later found to have even more far reaching effects (the bug also
exists when diskstore is off).

The gist of the bug is that, a Redis master replies with +FULLRESYNC to
a PSYNC attempt that fails and requires a full resynchronization.
However, the baseline offset sent along with FULLRESYNC was always the
current master replication offset. This is not ok, because there are
many reasosn that may delay the RDB file creation. And... guess what,
the master offset we communicate must be the one of the time the RDB
was created. So for example:

1) When the BGSAVE for replication is delayed since there is one
   already but is not good for replication.
2) When the BGSAVE is not needed as we attach one currently ongoing.
3) When because of diskless replication the BGSAVE is delayed.

In all the above cases the PSYNC reply is wrong and the slave may
reconnect later claiming to need a wrong offset: this may cause
data curruption later.
2015-08-04 17:06:10 +02:00
antirez 6233d210cd Sentinel: add more commonly useful sections to INFO.
Debugging is hard without those when there are problems like the one
investigated in issue #2700.
2015-07-29 12:29:12 +02:00
antirez 4bc42ca7f6 checkTcpBacklogSetting() now called in Sentinel mode too. 2015-07-29 12:13:20 +02:00
antirez 3c8861a73a Support for CLIENT KILL TYPE MASTER. 2015-07-28 17:01:19 +02:00
antirez e6f39338e6 CLIENT_MASTER introduced. 2015-07-28 16:58:35 +02:00
antirez c1e94b6b9c Force slaves to resync after unsuccessful PSYNC.
Using chained replication where C is slave of B which is in turn slave of
A, if B reconnects the replication link with A but discovers it is no
longer possible to PSYNC, slaves of B must be disconnected and PSYNC
not allowed, since the new B dataset may be completely different after
the synchronization with the master.

Note that there are varius semantical differences in the way this is
handled now compared to the past. In the past the semantics was:

1. When a slave lost connection with its master, disconnected the chained
slaves ASAP. Which is not needed since after a successful PSYNC with the
master, the slaves can continue and don't need to resync in turn.

2. However after a failed PSYNC the replication backlog was not reset, so a
slave was able to PSYNC successfully even if the instance did a full
sync with its master, containing now an entirely different data set.

Now instead chained slaves are not disconnected when the slave lose the
connection with its master, but only when it is forced to full SYNC with
its master. This means that if the slave having chained slaves does a
successful PSYNC all its slaves can continue without troubles.

See issue #2694 for more details.
2015-07-28 16:35:02 +02:00
antirez 278ea9d16b replicationHandleMasterDisconnection() belongs to replication.c. 2015-07-28 14:36:50 +02:00
antirez 54c71f2d96 RDMF: Redis -> Server in adjustOpenFilesLimit(). 2015-07-28 11:19:20 +02:00
antirez 813ff7fdde Avoid magic "0" argument to prepareForShutdown().
Backported from Disque.
2015-07-28 11:10:42 +02:00
antirez 5cfb792777 RDMF: dictRedisObjectDestructor -> dictObjectDestructor." 2015-07-28 11:03:01 +02:00
antirez a83e79b176 Use mstime_t as return value of mstime(). 2015-07-28 10:14:33 +02:00
antirez 02b1d5213d RDMF: use representClusterNodeFlags() generic name. 2015-07-27 15:08:58 +02:00
antirez 3325a9b11f RDMF: more names updated. 2015-07-27 15:03:10 +02:00
antirez 32f80e2f1b RDMF: More consistent define names. 2015-07-27 14:37:58 +02:00
antirez 40eb548a80 RDMF: REDIS_OK REDIS_ERR -> C_OK C_ERR. 2015-07-26 23:17:55 +02:00
antirez 2d9e3eb107 RDMF: redisAssert -> serverAssert. 2015-07-26 15:29:53 +02:00
antirez 14ff572482 RDMF: OBJ_ macros for object related stuff. 2015-07-26 15:28:00 +02:00
antirez 554bd0e7bd RDMF: use client instead of redisClient, like Disque. 2015-07-26 15:20:52 +02:00
antirez 424fe9afd9 RDMF: redisLog -> serverLog. 2015-07-26 15:17:43 +02:00
antirez cef054e868 RDMF (Redis/Disque merge friendlyness) refactoring WIP 1. 2015-07-26 15:17:18 +02:00
antirez c6333def13 SDS: Copyright updated further. 2015-07-25 17:41:56 +02:00
antirez cb2782c314 SDS: changes to unify Redis SDS with antirez/sds repo. 2015-07-25 17:25:44 +02:00
antirez 9894495c5a SDS: Copyright notice updated. 2015-07-25 17:08:44 +02:00
antirez 11425c89cf SDS: sdsjoinsds() call ported from antirez/sds fork. 2015-07-25 17:05:20 +02:00
Rogerio Goncalves ef29748d0d Check args before run ckquorum. Fix issue #2635 2015-07-24 14:08:50 +02:00
antirez 64fcd0e6ff SDS: avoid compiler warning in sdsIncrLen(). 2015-07-24 09:39:12 +02:00
antirez 935251259f Merge branch 'sds' into unstable 2015-07-24 08:49:23 +02:00
antirez ea9bd243ec SDS: use type 8 if we are likely to append to the string.
When empty strings are created, or when sdsMakeRoomFor() is called, we
are likely into an appending pattern. Use at least type 8 SDS strings
since TYPE 5 does not remember the free allocation size and requires to
call sdsMakeRoomFor() at every new piece appended.
2015-07-23 16:10:08 +02:00
antirez cf68f4ee6a Fix SDS type 5 sdsIncrLen() bug and added test.
Thanks to @oranagra for spotting this error.
2015-07-20 16:18:08 +02:00
Salvatore Sanfilippo bcb4d09123 Merge pull request #2636 from badboy/cluster-lock-fix
Cluster lock fix
2015-07-17 11:00:44 +02:00
Salvatore Sanfilippo 29391002f6 Merge pull request #2644 from MOON-CLJ/command_info_fix
pfcount support multi keys
2015-07-17 10:55:58 +02:00
Yongyue Sun 427794d845 bugfix: errno might change before logging
Signed-off-by: Yongyue Sun <abioy.sun@gmail.com>
2015-07-17 10:47:32 +02:00
Tom Kiemes 6142ddc6eb Fix: aof_delayed_fsync is not reset
aof_delayed_fsync was not set to 0 when calling CONFIG RESETSTAT
2015-07-17 10:39:36 +02:00
Salvatore Sanfilippo f049cfdb0d Merge pull request #2676 from july2993/unstable
config tcp-keepalive should be numerical field not bool
2015-07-17 10:34:43 +02:00
antirez 25e1cb3f04 Client timeout handling improved.
The previos attempt to process each client at least once every ten
seconds was not a good idea, because:

1. Usually because of the past min iterations set to 50, you get much
better processing period most of the times.

2. However when there are many clients and a normal setting for
server.hz, the edge case is triggered, and waiting 10 seconds for a
BLPOP that asked for 1 second is not ok.

3. Moreover, because of the high min-itereations limit of 50, when HZ
was set to an high value, the actual behavior was to process a lot of
clients per second.

Also the function checking for timeouts called gettimeofday() at each
iteration which can be costly.

The new implementation will try to process each client once per second,
gets the current time as argument, and does not attempt to process more
than 5 clients per iteration if not needed.

So now:

1. The CPU usage of an idle Redis process is the same or better.
2. The CPU usage of a busy Redis process is the same or better.
3. However a non trivial amount of work may be performed per iteration
when there are many many clients. In this particular case the user may
want to raise the "HZ" value if needed.

Btw with 4000 clients it was still not possible to noticy any actual
latency created by processing 400 clients per second, since the work
performed for each client is pretty small.
2015-07-16 10:54:18 +02:00
Jiahao Huang 92c146dfd3 config tcp-keepalive should be numerical field not bool 2015-07-16 15:53:44 +08:00
antirez e0bb454a16 Clarify a comment in clientsCron(). 2015-07-16 09:26:36 +02:00
antirez 3da97ea67f Add sdshdr5 to DEBUG structsize. 2015-07-16 09:14:39 +02:00
antirez 0ab27a4594 SDS: New sds type 5 implemented.
This is an attempt to use the refcount feature of the sds.c fork
provided in the Pull Request #2509. A new type, SDS_TYPE_5 is introduced
having a one byte header with just the string length, without
information about the available additional length at the end of the
string (this means that sdsMakeRoomFor() will be required each time
we want to append something, since the string will always report to have
0 bytes available).

More work needed in order to avoid common SDS functions will pay the
cost of this type. For example both sdscatprintf() and sdscatfmt()
should try to upgrade to SDS_TYPE_8 ASAP when appending chars.
2015-07-15 12:24:49 +02:00
antirez 056a0ca199 Fix redis-benchmark sds binding.
Same as redis-cli, now redis-benchmark requires to use hiredis sds copy
since it is different compared to the memory optimized fork of Redis
sds.
2015-07-14 17:33:30 +02:00
antirez a76b380e06 Fix DEBUG structsize output. 2015-07-14 17:17:06 +02:00
Oran Agra f15df8ba5d sds size classes - memory optimization 2015-07-14 17:17:06 +02:00
antirez 0f64080dcb DEBUG HTSTATS <dbid> added.
The command reports information about the hash table internal state
representing the specified database ID.

This can be used in order to investigate rehashings, memory usage issues
and for other debugging purposes.
2015-07-14 17:15:37 +02:00
antirez 4c7ee0d584 EXISTS is now variadic.
The new return value is the number of keys existing, among the ones
specified in the command line, counting the same key multiple times if
given multiple times (and if it exists).

See PR #2667.
2015-07-13 18:09:41 +02:00
antirez 5c4fcaf3fe Geo: fix command table keys position indexes for three commands.
GEOHASH, GEOPOS and GEODIST where declared as commands not accepting
keys, so the Redis Cluster redirection did not worked.

Close #2671.
2015-07-13 15:30:11 +02:00
antirez b96af595a5 GEOENCODE / GEODECODE commands removed.
Rationale:

1. The commands look like internals exposed without a real strong use
case.
2. Whatever there is an use case, the client would implement the
commands client side instead of paying RTT just to use a simple to
reimplement library.
3. They add complexity to an otherwise quite straightforward API.

So for now KILLED ;-)
2015-07-09 17:42:59 +02:00
antirez 1e12784259 Geo: -Ofast breaks builds on older GCCs. 2015-07-09 11:25:29 +02:00
antirez 5e04189887 Geo: validate long,lat passed by user via API 2015-07-06 18:39:25 +02:00
antirez 5254c2d3c3 Removed useless tryObjectEncoding() call from ZRANK. 2015-07-03 09:47:08 +02:00
antirez 4160bf0448 Geo: sync faster decoding from krtm that synched from Ardb.
Instead of successive divisions in iteration the new code uses bitwise
magic to interleave / deinterleave two 32bit values into a 64bit one.
All tests still passing and is measurably faster, so worth it.
2015-07-01 16:12:08 +02:00
antirez d308cadc8a Geo: added my copyright notice in modified files. 2015-06-29 16:34:02 +02:00
antirez 69c5b27273 Geo: support units only in abbreviated form.
I'm not a strong believer in multiple syntax for the same stuff, so
now units can be specified only as m, km, ft, mi.
2015-06-29 16:02:33 +02:00
antirez 083acbebc8 Geo: remove static declarations.
Stack traces produced by Redis on crash are the most useful tool we
have to fix non easily reproducible crashes, or even easily reproducible
ones where the user just posts a bug report and does not collaborate
furhter.

By declaring functions "static" they no longer show up in the stack
trace.
2015-06-29 15:57:17 +02:00
antirez f108c687ad Geo: GEODIST and tests. 2015-06-29 12:44:34 +02:00
antirez a12192f5ff Geo: command function names converted to lowercase, as elsewhere.
In Redis MULTIWORDCOMMANDNAME are mapped to functions where the command
name is all lowercase: multiwordcommandnameCommand().
2015-06-29 12:07:18 +02:00
antirez aae0a1f9cc Geo: GEOPOS command and tests. 2015-06-29 10:47:07 +02:00
antirez ddc7b85c5f Geo: GEOENCODE: fix command arity check. 2015-06-29 09:39:34 +02:00
antirez 6a8e108e2d Geo: GEOENCODE now returns score ranges.
If GEOENCODE must be our door to enter the Geocoding implementation
details and do fancy things client side, than return the scores as well
so that we can query the sorted sets directly if we wish to do the same
search multiple times, or want to compute the boxes in the client side
to refine our search needs.
2015-06-29 09:34:05 +02:00
antirez 1884bff12d Geo: fix comment indentation. 2015-06-29 09:24:22 +02:00
antirez db3df44184 Geo: debugging printf calls removed. 2015-06-29 09:21:31 +02:00
antirez 6d21027a23 Geo: GEOADD form using radius removed.
Can't immagine how this is useful in the context of the API exported by
Redis, and we are always in time to add more bloat if needed, later.
2015-06-29 09:20:07 +02:00
antirez 7d59e0a8c3 Geo: commands top comment as in other Redis code. 2015-06-29 09:16:27 +02:00
antirez a3b07b1718 Geo: COUNT option for GEORADIUS. 2015-06-27 10:23:58 +02:00
antirez cd91beea1c Geo: only one way to specify any given option. 2015-06-27 09:43:47 +02:00
antirez 710c05ac2a Geo: remove useless variable. geoRadiusGeneric() top comment improved. 2015-06-27 09:38:56 +02:00
MOON_CLJ c232235734 pfcount support multi keys 2015-06-26 17:58:45 +08:00
antirez fa9d62d34f Geo: from lat,lon API to lon,lat API according to GIS standard
The GIS standard and all the major DBs implementing GIS related
functions take coordinates as x,y that is longitude,latitude.
It was a bad start for Redis to do things differently, so even if this
means that existing users of the Geo module will be required to change
their code, Redis now conforms to the standard.

Usually Redis is very backward compatible, but this is not an exception
to this rule, since this is the first Geo implementation entering the
official Redis source code. It is not wise to try to be backward
compatible with code forks... :-)

Close #2637.
2015-06-26 10:58:27 +02:00
antirez 03ce189628 Geo: explain increment magic in membersOfGeoHashBox(). 2015-06-24 17:37:20 +02:00
antirez 87521f4455 Geo: GEOHASH command added, returning standard geohash strings. 2015-06-24 16:34:07 +02:00
Jan-Erik Rediger c7462ca9ff Don't include sysctl header
It's not needed (anymore) and is not available on Solaris.
2015-06-24 14:57:20 +02:00
Jan-Erik Rediger d28c51d166 Do not attempt to lock on Solaris 2015-06-24 14:57:15 +02:00