mirrors
/
redis
mirror of https://mirror.osredm.com/root/redis.git
2
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
8,599 Commits 92 Branches 328 Tags 473 MiB
Commit Graph

6100 Commits

Author SHA1 Message Date
Oran Agra d1a005ab39 fix issues found by a static analyzer 
cluster.c - stack buffer memory alignment
    The pointer 'buf' is cast to a more strictly aligned pointer type
evict.c - lazyfree_lazy_eviction, lazyfree_lazy_eviction always called
defrag.c - bug in dead code
server.c - casting was missing parenthesis
rax.c - indentation / newline suggested an 'else if' was intended
 2019-10-07 09:09:32 +03:00
antirez bd60447066 LOLWUT: ability to specify VERSION <ver> option. 2019-10-04 19:19:54 +02:00
antirez 9073d56eec LOLWUT: refactoring + skeleton of LOLWUT 6. 2019-10-04 19:19:48 +02:00
Oran Agra 0f1969f16f trim the double implementation of jemalloc purge 2019-10-04 14:22:13 +03:00
Oran Agra bf759cc9c3 Merge remote-tracking branch 'antirez/unstable' into jemalloc_purge_bg 2019-10-04 13:53:40 +03:00
antirez ee1cef189f Minor aesthetic changes to #6419. 2019-10-04 12:00:41 +02:00
Salvatore Sanfilippo 264778af0a
Merge pull request #6419 from oranagra/lua_timeout_sha 
On LUA script timeout, print the script SHA to the log
 2019-10-04 11:58:05 +02:00
Salvatore Sanfilippo c326f28833
Merge pull request #6308 from oranagra/repl-diskless-load-config 
fix error handling on config parsing of repl-diskless-load
 2019-10-04 11:56:08 +02:00
antirez 3eaff2941e Modules: RM_Call(): give pointer to documentation. 2019-10-04 11:46:53 +02:00
antirez 2a81e49dde Modules: RM_Call/Replicate() ability to exclude AOF/replicas. 2019-10-04 11:44:53 +02:00
antirez 1bca62c4b7 Modules: RM_Replicate() in thread safe contexts. 2019-10-03 11:03:46 +02:00
antirez e938bbc543 Modules: implement RM_Replicate() from async callbacks. 2019-10-03 10:56:37 +02:00
antirez c549513acd Modules: handle propagation when ctx is freed. Flag modules commands ctx. 2019-10-02 18:33:40 +02:00
Oran Agra 2e19b94113 RED-31295 - redis: avoid race between dlopen and thread creation 
It seeems that since I added the creation of the jemalloc thread redis
sometimes fails to start with the following error:

Inconsistency detected by ld.so: dl-tls.c: 493: _dl_allocate_tls_init: Assertion `listp->slotinfo[cnt].gen <= GL(dl_tls_generation)' failed!

This seems to be due to a race bug in ld.so, in which TLS creation on the
thread, collide with dlopen.

Move the creation of BIO and jemalloc threads to after modules are loaded.

plus small bugfix when trying to disable the jemalloc thread at runtime
 2019-10-02 15:39:44 +03:00
antirez 758b39be99 Speedup INFO server section. 2019-10-02 11:30:20 +02:00
antirez f49f0a6f72 SDS: make sdscatfmt() faster by pre-allocating a bit. 2019-10-02 11:30:20 +02:00
Oran Agra 98426e9886 On LUA script timeout, print the script SHA to the log 
since the slowlog and other means that can help you detect the bad script
are only exposed after the script is done. it might be a good idea to at least
print the script name (sha) to the log when it timeouts.
 2019-10-02 08:40:35 +03:00
antirez 40acb4412d GEORADIUS reply: fix of the previous fix about #6417. 2019-10-01 19:18:55 +02:00
Salvatore Sanfilippo 2e2fe98f9c
Merge pull request #6270 from oranagra/modules_info 
Extend modules API to allow modules report to redis INFO
 2019-10-01 18:02:33 +02:00
antirez 3281ebb495 Fix GEORADIUS replies broken after RESP3 introduction. 
This commit fixes #6417.
 2019-10-01 10:38:56 +02:00
antirez beebb3c013 Merge branch 'hashed-pass-opcode' into unstable 2019-10-01 10:33:04 +02:00
Oran Agra 1b4f888109 Use sdscatfmt instead of sdscatprintf in module info 
sdscatfmt is faster
 2019-09-30 21:13:13 +03:00
Oran Agra d5c14c70b7 Merge remote-tracking branch 'antirez/unstable' into modules_info 2019-09-30 20:47:35 +03:00
antirez a9628142fd Clarify a comment about memory total_frag field. 2019-09-30 18:37:59 +02:00
antirez 09041b9359 ACLs: change hashed passwords opcode to also remove them. 
Related to PR #6405
 2019-09-30 18:28:45 +02:00
Madelyn Olson ea7c3fe7fd Allowed passing in of password hash and fixed config rewrite 2019-09-30 17:57:49 +02:00
Salvatore Sanfilippo 2478b47b89
Merge pull request #6304 from dbenders/fix-issue-6141 
Fix issue #6141: cluster rebalance does not support binary key
 2019-09-30 17:43:58 +02:00
antirez 6a2831c24a Fix comments aesthetics. 2019-09-30 17:17:36 +02:00
antirez 23f5cb4233 Change a bit the style of #6385. 2019-09-30 10:58:15 +02:00
Salvatore Sanfilippo ad45d7e407
Merge pull request #6385 from filipecosta90/perf-reply-ss-error 
Improve performance of RM_ReplyWithSimpleString and RM_ReplyWi…
 2019-09-30 10:57:05 +02:00
antirez 916c4e5d86 moduleRDBLoadError(): io->ctx may be NULL. 
The correct way to access the module about a given IO context is to
deference io->type->module, since io->ctx is only populated if the user
requests an explicit context from an IO object.
 2019-09-27 18:42:38 +02:00
antirez 8b5848a4f5 Fix memory leak in moduleLoadFromQueue(). 2019-09-27 18:33:21 +02:00
antirez b394817754 Fix memory leak in RM_UnregisterCommandFilter(). 2019-09-27 18:33:05 +02:00
antirez 0a07f8ffee TerminateModuleForkChild(): fix function prototype. 2019-09-27 12:23:07 +02:00
antirez 721d3c9e0c TerminateModuleForkChild(): move safety checks there. 
We don't want that the API could be used directly in an unsafe way,
without checking if there is an active child. Now the safety checks are
moved directly in the function performing the operations.
 2019-09-27 12:17:47 +02:00
antirez de1f82aa33 Function renamed hasForkChild() -> hasActiveChildProcess(). 2019-09-27 12:03:09 +02:00
antirez 82845f8d04 TerminateModuleForkChild(): use wait4 for safety. 
In theory currently there is only one active child, but the API may
change or for bugs in the implementation we may have several (it was
like that for years because of a bug). Better to wait for a specific
pid and avoid consuing other pending children information.
 2019-09-27 11:59:58 +02:00
antirez beb1356b5b Improve error message in BGSAVE. 2019-09-27 11:59:37 +02:00
antirez ae3ef964c1 Modules fork: improve SIGUSR1 handling, fix include. 
We can't expect SIGUSR1 to have any specific value range, so let's
define an exit code that we can handle in a special way.
This also fixes an #include <wait.h> that is not standard.
 2019-09-27 11:39:45 +02:00
Salvatore Sanfilippo 6129758558
Merge branch 'unstable' into modules_fork 2019-09-27 11:24:06 +02:00
antirez fddc4757c8 BGREWRITEAOF: improve the generic error message. 2019-09-26 16:14:21 +02:00
Oran Agra 83e87bac76 Fix lastbgsave_status, when new child signal handler get intended kill 
And add a test for that.
 2019-09-26 15:16:34 +03:00
antirez b7b23bdfb8 INFO: more info about loaded modules. 
Related to #6024.
 2019-09-26 12:18:55 +02:00
Salvatore Sanfilippo c1ea6175c5
Merge pull request #6024 from itamarhaber/info_modules 
Adds a "Modules" section to `INFO`
 2019-09-26 11:58:52 +02:00
Salvatore Sanfilippo 959fb5cf68
Merge pull request #6235 from oranagra/module_rdb_load_errors 
Allow modules to handle RDB loading errors.
 2019-09-26 11:52:42 +02:00
Salvatore Sanfilippo b0a90d8fa8
Merge pull request #4980 from yossigo/rm-assert 
Add RedisModule_Assert() API call.
 2019-09-26 11:39:29 +02:00
filipecosta90 af15b285fa [fix] fixed the un-refactor bug. 2019-09-25 17:28:42 +01:00
Salvatore Sanfilippo 388bc55344
Merge pull request #6354 from jaredzhu/aarch64-devel 
Add aarch64 uc_mcontext debug dump info
 2019-09-25 18:14:24 +02:00
antirez b3d6cb268a Modify #6401 changes to fit 80 cols. 2019-09-25 18:08:11 +02:00
Salvatore Sanfilippo f3aaf2b4d8
Merge pull request #6401 from valentinogeron/fix-discard-during-oom 
DISCARD should not fail during OOM
 2019-09-25 18:07:25 +02:00
Salvatore Sanfilippo f6cf08e582
Merge pull request #6402 from mieko/unstable 
Seed SipHash with 128-bit key
 2019-09-25 18:03:39 +02:00
antirez 8a531cedb0 ACL: fix ##6408, default user state affecting all the connections. 2019-09-25 17:45:05 +02:00
Mike A. Owens 0a4d2bbd9c Seed SipHash with 128-bit key 
SipHash expects a 128-bit key, and we were indeed generating 128-bits,
but restricting them to hex characters 0-9a-f, effectively giving us
only 4 bits-per-byte of key material, and 64 bits overall.

Now, we skip the hex conversion and supply 128 bits of unfiltered
random data.
 2019-09-23 19:24:09 -04:00
filipecosta90 733280d9cb [fix] un-refactor the code. [perf] replyWithStatus now makes usage of addReplyProto 2019-09-23 23:45:31 +01:00
antirez eda703ab28 redis-cli: support for ACL style user/pass AUTH. 2019-09-23 19:57:13 +02:00
antirez b21dd082c3 redis-cli: CSV and RAW target for more RESP3 types. 2019-09-23 19:36:06 +02:00
antirez cc108057ff redis-cli: ability to start a session in RESP3 mode. 2019-09-23 17:55:05 +02:00
antirez 8ea185ea30 redis-cli: AUTH can now have 3 arguments as well. 2019-09-23 17:47:36 +02:00
antirez bb7546c913 RESP3: varbatim handling for other redis-cli outputs. 2019-09-23 17:42:57 +02:00
antirez 264708427b RESP3: implementation of verbatim output with TTY target. 2019-09-23 17:39:42 +02:00
valentino 7a73b7f168 DISCARD should not fail during OOM 
discard command should not fail during OOM, otherwise client MULTI state
will not be cleared.
 2019-09-22 09:22:53 +03:00
antirez 2e4fa7bb48 Make config.c always_replicate_commands more uniform. 
Better if it resembles the other similar code paths.
 2019-09-20 11:46:35 +02:00
antirez 1dd36347ed Merge branch 'unstable' of github.com:/antirez/redis into unstable 2019-09-20 11:45:02 +02:00
antirez 630638dcde Remove redundant statement in config.c. 
Thanks to @guybe7 for spotting the error in the original PR I merged.
 2019-09-20 11:44:32 +02:00
Salvatore Sanfilippo f135d6295b
Merge pull request #6367 from Dolondro/unstable 
Spelling: Typo fixes for API documentation
 2019-09-20 11:33:58 +02:00
Salvatore Sanfilippo 3976f4c88e
Merge pull request #6368 from suntw2015/unstable 
fix rdb function rdbLoadIntegerObject comment
 2019-09-20 11:33:27 +02:00
Salvatore Sanfilippo d0050db236
Merge pull request #6389 from ocadaruma/patch-1 
Fix typos in hyperloglog.c
 2019-09-20 11:24:07 +02:00
Salvatore Sanfilippo 975eb73145
Merge pull request #6400 from mors741/unstable 
Fix compilation error
 2019-09-20 11:19:56 +02:00
antirez 0a146a8be4 Add useless break for uniformity / future protection. 2019-09-20 11:19:07 +02:00
Salvatore Sanfilippo bd90c99f83
Merge pull request #6398 from trevor211/fixRESP3ToLuaBool 
RESP3: Fix function redisProtocolToLuaType about RESP3->Lua bools.
 2019-09-20 11:15:20 +02:00
Evgeny a1a65d238c
Fix compilation error 2019-09-20 01:11:20 -07:00
WuYunlong 30a3644e64 RESP3: Fix function redisProtocolToLuaType about RESP3->Lua bools. 2019-09-20 08:37:23 +08:00
WuYunlong 74e3a622a2 Fix bad handling of unexpected option while loading config "lua-replicate-commands". 2019-09-20 08:14:36 +08:00
antirez 474a923160 RESP3: Use verbatim in LATENCY subcommands. 2019-09-18 18:53:22 +02:00
antirez cb384127d1 RESP3: Use verbatim in CLIENT LIST. 2019-09-18 18:52:13 +02:00
antirez dd2f695d7e RESP3: Use verbatim in CLUSTER subcommands. 2019-09-18 18:51:15 +02:00
antirez 1b3cb3b0de RESP3: Use verbatim in MEMORY subcommands. 2019-09-18 18:48:14 +02:00
antirez ff9a5d231b RESP3: Use verbatim in DEBUG HTSTATS / HTSTATS-KEY. 2019-09-18 18:46:11 +02:00
antirez aca5482fbf RESP3: Use verbatim in INFO output. 2019-09-18 18:33:13 +02:00
antirez 68519b7034 RESP3: Verbatim conversion test in the LOLWUT command. 
redis-cli is currently not able to handle it after going in RESP3 mode,
because of hiredis limitations.
 2019-09-18 18:28:51 +02:00
antirez 8ea4bdd91d RESP3: Lua double -> RESP3 conversion. 2019-09-17 19:26:46 +02:00
antirez 89f929b12a RESP3: RESP3 double -> Lua conversion. 2019-09-17 19:20:30 +02:00
antirez 19aba4ac78 RESP3: double -> human readable conversion. 2019-09-17 19:08:33 +02:00
antirez e8e30bc402 RESP3: bool and null values in RESP -> human readable conversion. 2019-09-17 18:57:24 +02:00
Okada Haruki c1cc5ca767
Fix typo 2019-09-17 06:18:01 +09:00
antirez f01f0c02d1 RESP3: convert RESP3 null as Lua nil. Implement RESP3->Lua bools. 2019-09-16 18:36:16 +02:00
antirez 6931004969 RESP3: change behavior of Lua returning true/false for RESP3. 
Here we introduce a change in the way we convert values from Lua to
Redis when RESP3 is selected: this is possible without breaking the fact
we can return directly what a command returned, because there is no
Redis command in RESP2 that returns true or false to Lua, so the
conversion in the case of RESP2 is totally arbitrary. When a script is
written selecting RESP3 from Lua, it totally makes sense to change such
behavior and return RESP3 true/false when Lua true/false is returned.
 2019-09-16 18:18:17 +02:00
antirez 2cc4d0286c RESP3: implement new NULL representation parsing in Lua. 2019-09-16 17:49:47 +02:00
antirez ca81d49006 RESP3: handle set Lua -> Redis conversion. 2019-09-16 12:19:19 +02:00
antirez c792504fab RESP3: handle map Lua -> Redis conversion. 2019-09-16 12:15:39 +02:00
antirez dad38c19c8 RESP3: report set/map as nested tables to Lua. 2019-09-16 11:49:42 +02:00
filipecosta90 4a30a26f8f [add] improved performance of RM_ReplyWithSimpleString and RM_ReplyWithError by making usage addReplyProto instead of addReplySds 2019-09-15 21:16:30 +01:00
antirez 888efc1b36 RESP3: Lua parsing should depend on lua client, not lua caller. 
We want all the scripts to run in RESP2 mode by default. It's up to the
caller to switch to V3 using redis.setresp() if it is really needed.
This way most scripts written for past Redis versions will continue to
work with Redis >= 6 even if the client is in RESP3 mode.
 2019-09-13 19:38:39 +02:00
antirez 26b6c697d3 RESP3: Lua debugger support for printing sets and maps. 2019-09-13 19:19:10 +02:00
antirez 93c52ff5ff RESP3: implement lua.setresp(). 2019-09-13 19:01:39 +02:00
antirez b170a01fa7 ACL: store hashed passwords in memory. 
Note that this breaks API compatibility with Redis < 6:

    CONFIG GET requirepass

Will no longer return a cleartext password as well, but the SHA256 hash
of the password set.
 2019-09-12 12:54:57 +02:00
antirez ae5054b476 ACL: SHA256 based password hashing function implemented. 2019-09-12 12:33:22 +02:00
antirez 9d2ecf6be3 ACL: add slightly modified version of sha256.c for password hashing. 
memory.h include removed, types substituted with stdint types.
 2019-09-12 12:21:37 +02:00
antirez f4d37173fe ACL: protect MULTI/EXEC transactions after rules change. 2019-09-11 19:42:10 +02:00
antirez ae8cab3296 Merge branch 'unstable' of github.com:/antirez/redis into unstable 2019-09-06 12:24:40 +02:00
antirez a092f20d87 handleClientsBlockedOnKeys() refactoring. 2019-09-06 12:24:26 +02:00
suntiawnen 781f3fd8be fix rdb function rdbLoadIntegerObject comment 2019-09-06 12:01:44 +08:00
Doug Nelson cc8f06ece9 Typo fixes in API documentation 2019-09-05 16:25:06 +01:00
Salvatore Sanfilippo 86aade9a02
Merge pull request #6364 from oranagra/fix_module_aux_when 
Fix to module aux data rdb format for backwards compatibility with old check-rdb
 2019-09-05 13:30:26 +02:00
Oran Agra 3bfcae247a Fix to module aux data rdb format for backwards compatibility with old check-rdb 
When implementing the code that saves and loads these aux fields we used rdb
format that was added for that in redis 5.0, but then we added the 'when' field
which meant that the old redis-check-rdb won't be able to skip these.
this fix adds an opcode as if that 'when' is part of the module data.
 2019-09-05 14:11:37 +03:00
antirez 89ad0ca566 Fix handleClientsBlockedOnKeys() names in comments. 2019-09-05 13:05:57 +02:00
antirez 12c56a8e75 redis-cli: always report server errors on read errors. 
Before this commit we may have not consumer buffers when a read error is
encountered. Such buffers may contain errors that are important clues
for the user: for instance a protocol error in the payload we send in
pipe mode will cause the server to abort the connection. If the user
does not get the protocol error, debugging what is happening can be a
nightmare.

This commit fixes issue #3756.
 2019-09-04 17:55:46 +02:00
antirez f7090f43ad AOF: be future-proof and close the file pointer. 
Currently useless but we release the fake client, so better to do a full
cleanup. Thanks to @TomMD reporting this in #6353.
 2019-09-04 17:20:37 +02:00
zhudacai 00228490 66a5c1cd7e The aarch64 architecture is support normal memory unaligned accesses, 
so add the UNALIGNED_LE_CPU to the aarch64 .
 2019-09-04 12:14:25 +00:00
antirez b2e10131c0 Rio: fix flag name, function is never used btw. 
Thanks to @tnclong for reporting the problem.
 2019-09-04 13:01:07 +02:00
antirez 06d490342f RESP3: fix cases of NULL reported instead of empty aggregate. 2019-09-02 12:50:47 +02:00
antirez b12ccc840e Cluster: abort loading nodes data if vars arguments are unbalanced. 
See for reference PR #6337. Thanks to @git-hulk for spotting this.
 2019-09-02 11:41:20 +02:00
antirez e57067b677 More strict checks and better comments in flushSlaveOutputBuffers(). 
Related to #6296.
 2019-08-31 14:46:24 +02:00
antirez 2303ba1441 Merge branch 'unstable' of github.com:/antirez/redis into unstable 2019-08-31 14:41:28 +02:00
antirez ca5ccc58c5 Improve comment in flushSlavesOutputBuffers(). 2019-08-31 14:40:09 +02:00
Salvatore Sanfilippo 16435e02bf
Merge pull request #6296 from soloestoy/flushSlavesOutputBuffers-bugfix 
networking: flushSlavesOutputBuffers bugfix
 2019-08-31 14:39:33 +02:00
zhudacai 00228490 d7c25806c0 src/debug.c do not support aarch64 dump utcontext, so add the context 
of aarch64.
The content comes from the definition of the sigcontext and tested on
my aarch64 server.

sigcontext defined at the linux kernel code:
arch/arm64/include/uapi/asm/sigcontext.h
 2019-08-31 07:47:11 +00:00
Oran Agra 78bbb9b58d Modlue fork is killed when the parent exists 2019-08-25 10:11:48 +03:00
Oran Agra 61853ad8de Module INFO, support default section for simple modules 2019-08-18 10:01:57 +03:00
Oran Agra 1d6e5dc4dc Module INFO, add support for dict fields, rename API to have common prefix 2019-08-18 09:41:45 +03:00
Oran Agra 1026d2caf8 fix error handling on config parsing of repl-diskless-load 2019-08-08 14:53:36 +03:00
Oran Agra 0a97149dec RM_ReplyWithCString was missing registration 2019-08-07 13:06:11 +03:00
Diego Bendersky b27f388344 pass len to alloc in clusterManagerMigrateKeysInReply 2019-08-05 17:35:50 -03:00
antirez 0e9b5adbd3 Replication: clarify why repl_put_online_on_ack exists at all. 2019-08-05 17:38:15 +02:00
zhaozhao.zz d3f7eb0c87 networking: flushSlavesOutputBuffers bugfix 2019-08-02 17:17:19 +08:00
antirez 583933e2d6 Fix regression causing EXEC to appear in the slow log. 
This was recently introduced with PR #6266.
 2019-07-31 19:05:20 +02:00
antirez 2a8a63af44 Make EMBSTR case of #6261 more obvious. 2019-07-31 12:03:10 +02:00
Salvatore Sanfilippo f899c31fbc
Merge pull request #6261 from chendq8/robj 
make memory usage consistent of robj with OBJ_ENCODING_INT
 2019-07-31 11:54:11 +02:00
Salvatore Sanfilippo fc5c2052b5
Merge pull request #6266 from madolson/dev-unstable-hide-auth-and-hello 
Hide HELLO and AUTH from slowlog and monitor
 2019-07-31 11:12:46 +02:00
antirez e1b13b4d39 Merge branch 'unstable' of github.com:/antirez/redis into unstable 2019-07-31 10:37:17 +02:00
antirez b2079cb508 HyperLogLog: fix the fix of a corruption bug. 2019-07-31 10:36:23 +02:00
Salvatore Sanfilippo f0c10d4360
Merge pull request #6277 from devnexen/fbsd_get_rss 
Updating resident memory request impl on FreeBSD.
 2019-07-30 18:24:38 +02:00
Oran Agra 40c4183196 Log message when modules prevent diskless-load 2019-07-30 16:32:58 +03:00
Oran Agra e5187ad2ae Merge remote-tracking branch 'oss/unstable' into module_rdb_load_errors 2019-07-30 15:14:08 +03:00
Oran Agra 4339706e07 Avoid diskelss-load if modules did not declare they handle read errors 2019-07-30 15:11:57 +03:00
Salvatore Sanfilippo f42846e8c7
Merge pull request #6264 from oranagra/modules_api_aux_rdb 
Implement module api for aux data in rdb
 2019-07-30 11:50:44 +02:00
Salvatore Sanfilippo ce72cac0f8
Merge pull request #6269 from wubostc/patch-1 
Reduce the calling stack
 2019-07-30 11:41:48 +02:00
antirez e4b3c8bbc3 emptyDbGeneric(): call signalFlushDb() before deleting the keys. 
This was broken since a refactoring performed recently by myself.
 2019-07-30 11:20:54 +02:00
John Sully d659654f53 Fix HLL corruption bug 2019-07-29 18:11:52 -04:00
David Carlier f1c6c658ac Updating resident memory request impl on FreeBSD. 2019-07-28 14:33:57 +01:00
Salvatore Sanfilippo 5591a14c0a
Merge pull request #6248 from oranagra/module-log 
RM_Log - add support for logging without a context or context without module
 2019-07-25 18:48:46 +02:00
Oran Agra e91d9a6fff Extend modules API to allow modules report to redis INFO 
this implements #6012
 2019-07-24 12:58:15 +03:00
antirez fde5e737dc Merge branch 'unstable' of github.com:/antirez/redis into unstable 2019-07-24 11:45:19 +02:00
antirez 32efd9adf8 Client side caching: config option for table fill rate. 2019-07-24 11:35:01 +02:00
wubostc 48f4cfb601
Reduce the calling stack 2019-07-24 16:22:26 +08:00
Salvatore Sanfilippo dbee7ca007
Merge pull request #6265 from madolson/dev-unstable-remove-create-object 
Removed unnecessary creation of Redis objects
 2019-07-23 17:05:10 +02:00
antirez c98e7717bb Client side caching: show tracking slots usage in INFO. 2019-07-23 11:02:14 +02:00
antirez 9268493e8d Client side caching: implement full slot limit function. 2019-07-23 10:57:22 +02:00
zhaozhao.zz 9012e587ca Client side caching: do not reclaim tracking table if it's empty 2019-07-23 15:25:00 +08:00
Madelyn Olson 7d21754710 Hide HELLO and AUTH from slowlog and monitor 2019-07-22 22:53:15 -07:00
Madelyn Olson 8fe173452a Removed unecessary creation of Redis objects 2019-07-22 22:52:16 -07:00
Oran Agra 3b6aeea44c Implement module api for aux data in rdb 
Other changes:
* fix memory leak in error handling of rdb loading of type OBJ_MODULE
 2019-07-22 21:15:33 +03:00
antirez c41f94d2a3 Client side caching: split invalidation into key / slot. 2019-07-22 18:59:53 +02:00
antirez 09c06698e9 Client side caching: redis-cli ability to enable tracking. 
This is extremely useful in order to simulate an high load of requests
about different keys, and force Redis to track a lot of informations
about several clients, to simulate real world workloads.
 2019-07-22 18:45:47 +02:00
antirez 436d02dd7b Move signalFlushedDb() into a better place. 
Now that the call also invalidates client side caching slots, it is
important that after an internal flush operation we both send the
notifications to the clients and, at the same time, are able to reclaim
the memory of the tracking table. This may even fix a few edge cases
related to MULTI/EXEC + WATCH during resync, not sure, but in general
looks more correct.
 2019-07-22 12:37:47 +02:00
antirez 7d3992a80a Client side caching: reclaim the tracking table on FLUSHALL. 2019-07-22 12:33:49 +02:00
antirez 842b44dc46 Client side caching: call the invalidation functions always. 
Otherwise what happens is that the tracking table will never get garbage
collected if there are no longer clients with tracking enabled.
Now the invalidation function immediately checks if there is any table
allocated, otherwise it returns ASAP, so the overhead when the feature
is not used should be near zero.
 2019-07-22 12:29:54 +02:00
antirez f850201c64 Client side caching: don't hash the key if not needed. 2019-07-22 12:10:51 +02:00
antirez e00442eb13 Clinet side caching: take count of used caching slots. 2019-07-22 11:47:44 +02:00
chendianqiang 004c26ec28 make memory usage consistent of robj with OBJ_ENCODING_INT 2019-07-22 17:45:30 +08:00
Oran Agra d7d028a7a7 Allow modules to handle RDB loading errors. 
This is especially needed in diskless loading, were a short read could have
caused redis to exit. now the module can handle the error and return to the
caller gracefully.

this fixes #5326
 2019-07-21 18:19:32 +03:00
antirez bc5cb168f5 RDB: fix MODULE_AUX loading by continuing to next opcode. 
Thanks to @JohnSully for noticing this problem.
 2019-07-19 11:12:46 +02:00
antirez 5f450e4928 RDB: make sure to abort on LZF encoding error. 2019-07-18 18:59:38 +02:00
antirez bd0f06c18c RDB: handle encoding errors with rdbExitReportCorruptRDB(). 
Without such change, the diskless replicas, when loading RDB files from
the socket will not abort when a broken RDB file gets loaded. This is
potentially unsafe, because right now Redis is not able to guarantee
that encoding errors are safe from the POV of memory corruptions (for
instance the LZF library may not be safe against untrusted data?) so
better to abort when the RDB file we are going to load is corrupted.

Instead I/O errors are still returned to the caller without aborting,
so that in case of short read the diskless replica can try again.
 2019-07-18 18:51:45 +02:00
Salvatore Sanfilippo 41ed85bd26
Merge pull request #6252 from soloestoy/tracking-flushdb 
Tracking flushdb
 2019-07-18 16:41:37 +02:00
antirez 947319cadd RDB: update rdbLoadRio comment about EOF condition. 2019-07-18 12:37:55 +02:00
zhaozhao.zz f59119aad6 Client side caching: filter clients untracking 2019-07-18 16:46:19 +08:00
antirez 42b6305964 RDB: try to make error handling code more readable. 2019-07-17 17:30:02 +02:00
antirez 6b72b04a37 Rio: when in error condition avoid doing the operation. 2019-07-17 16:46:22 +02:00
antirez 5189db3d81 RDB: modify rdbReportError() var name for clarity. 2019-07-17 16:46:22 +02:00
antirez 48d91cf4cc Rio: remember read/write error conditions. 2019-07-17 16:46:22 +02:00
Oran Agra c56b4ddc6f prevent diskless replica from terminating on short read 
now that replica can read rdb directly from the socket, it should avoid exiting
on short read and instead try to re-sync.

this commit tries to have minimal effects on non-diskless rdb reading.
and includes a test that tries to trigger this scenario on various read cases.
 2019-07-17 16:46:22 +02:00
Oran Agra 56258c6b7d Module API for Forking 
* create module API for forking child processes.
* refactor duplicate code around creating and tracking forks by AOF and RDB.
* child processes listen to SIGUSR1 and dies exitFromChild in order to
  eliminate a valgrind warning of unhandled signal.
* note that BGSAVE error reply has changed.

valgrind error is:
  Process terminating with default action of signal 10 (SIGUSR1)
 2019-07-17 16:40:24 +03:00
zhaozhao.zz 6191ea90a1 Client side caching: implement trackingInvalidateKeysOnFlush() 2019-07-17 20:33:52 +08:00
Oran Agra 6e07fac405 RM_Log - add support for logging without a context or context without module 
for instance detached thread safe contexts, or various callbacks that don't
provide a context.
 2019-07-17 15:08:18 +03:00
antirez 241d18d954 Merge branch 'unstable' of github.com:/antirez/redis into unstable 2019-07-17 12:26:38 +02:00
Salvatore Sanfilippo 3f1c84751a
Merge pull request #6196 from AngusP/resp3 
[RESP3, Minor] in networking.c double representation for -infiinity leaves out comma
 2019-07-12 13:08:46 +02:00
Salvatore Sanfilippo 6100a82a3a
Merge pull request #5738 from bmerry/fix-swapdb-int-overflow 
Make dbSwapDatabases take args as long
 2019-07-12 12:36:33 +02:00
antirez 54c4e7f86c Vertically compact code in aofWrite(). 2019-07-12 12:18:33 +02:00
antirez 2a99f199bc Merge branch 'unstable' of github.com:/antirez/redis into unstable 2019-07-10 19:48:44 +02:00
antirez dad0e916ff Client side caching: filter clients no longer there. 2019-07-10 19:46:46 +02:00
Salvatore Sanfilippo 7567323071
Merge pull request #6198 from oranagra/skiplist-mem-usage 
missing per-skiplist overheads in MEMORY USAGE
 2019-07-10 19:43:24 +02:00
antirez c7aaf8db4d Client side caching: implement CLIENT GETREDIR. 
This subcommand may simplify the writing of Redis client libraries
using the tracking feature and/or improve observability and debugging
capabilities.
 2019-07-10 18:17:10 +02:00
antirez 923e4fb312 Client side caching: add tracking clients in INFO. 2019-07-10 18:08:31 +02:00
antirez 7c9f6ebc8f Diskless replica: fix disklessLoadRestoreBackups() bug. 2019-07-10 12:36:26 +02:00
antirez 3bbb9a1413 Diskless replica: refactoring of DBs backups. 2019-07-10 11:42:26 +02:00
antirez 3300e98ff0 Diskless replica: fix mispelled var name. 2019-07-10 09:34:21 +02:00
antirez dfcbeaf115 Diskless replica: a few aesthetic changes to rio.c 2019-07-08 18:39:59 +02:00
antirez 81b18fa3a0 Diskless replica: a few aesthetic changes to replication.c. 2019-07-08 18:32:47 +02:00
Oran Agra 2de544cfcc diskless replication on slave side (don't store rdb to file), plus some other related fixes 
The implementation of the diskless replication was currently diskless only on the master side.
The slave side was still storing the received rdb file to the disk before loading it back in and parsing it.

This commit adds two modes to load rdb directly from socket:
1) when-empty
2) using "swapdb"
the third mode of using diskless slave by flushdb is risky and currently not included.

other changes:
--------------
distinguish between aof configuration and state so that we can re-enable aof only when sync eventually
succeeds (and not when exiting from readSyncBulkPayload after a failed attempt)
also a CONFIG GET and INFO during rdb loading would have lied

When loading rdb from the network, don't kill the server on short read (that can be a network error)

Fix rdb check when performed on preamble AOF

tests:
run replication tests for diskless slave too
make replication test a bit more aggressive
Add test for diskless load swapdb
 2019-07-08 15:37:48 +03:00
Salvatore Sanfilippo 722446510f
Merge pull request #6116 from AngusP/scan-types 
SCAN: New Feature `SCAN cursor [TYPE type]` modifier suggested in issue #6107
 2019-07-08 12:53:34 +02:00
Angus Pearson 6eb52e200c Change typeNameCanonicalize -> getObjectTypeName, and other style changes 2019-07-08 11:04:37 +01:00
Guy Korland 2d07883cab
fix build tracking.c should be tracking.o 
thanks to @rafie
 2019-07-07 18:28:15 +03:00
Salvatore Sanfilippo e8ae07aabf
Merge pull request #6162 from madolson/dev-unstable-config-refactor 
Refactored yesno configs so there is less duplication
 2019-07-07 17:13:11 +02:00
Salvatore Sanfilippo 677e95e2f5
Merge pull request #6161 from swilly22/redismodule-loading-ctx-flag 
Extend REDISMODULE_CTX_FLAGS to indicate if redis is  loading
 2019-07-07 17:01:29 +02:00
Salvatore Sanfilippo a714004738
Merge pull request #6022 from itamarhaber/RedisModule_ReplyWithCString 
Adds RedisModule_ReplyWithCString
 2019-07-05 20:25:18 +02:00
antirez 6b29f2d83d Client side caching: RESP2 support. 2019-07-05 12:24:28 +02:00
antirez f099def733 Merge branch 'unstable' of github.com:/antirez/redis into unstable 2019-07-04 18:34:53 +02:00
Oran Agra 41c5ebcab6 missing per-skiplist overheads in MEMORY USAGE 
these had severe impact for small zsets, for instance ones with just one
element that is longer than 64 (causing it not to be ziplist encoded)
 2019-07-04 15:20:49 +03:00
antirez 3e43aed1ac Client side caching: fix invalidate message len and content. 2019-07-04 14:07:23 +02:00