Commit Graph

4910 Commits

Author SHA1 Message Date
Matt Stancliff 72b8574cca Fix sending uninitialized bytes
Fixes valgrind error:
Syscall param write(buf) points to uninitialised byte(s)
   at 0x514C35D: ??? (syscall-template.S:81)
   by 0x456B81: clusterWriteHandler (cluster.c:1907)
   by 0x41D596: aeProcessEvents (ae.c:416)
   by 0x41D8EA: aeMain (ae.c:455)
   by 0x41A84B: main (redis.c:3832)
 Address 0x5f268e2 is 2,274 bytes inside a block of size 8,192 alloc'd
   at 0x4932D1: je_realloc (jemalloc.c:1297)
   by 0x428185: zrealloc (zmalloc.c:162)
   by 0x4269E0: sdsMakeRoomFor.part.0 (sds.c:142)
   by 0x426CD7: sdscatlen (sds.c:251)
   by 0x4579E7: clusterSendMessage (cluster.c:1995)
   by 0x45805A: clusterSendPing (cluster.c:2140)
   by 0x45BB03: clusterCron (cluster.c:2944)
   by 0x423344: serverCron (redis.c:1239)
   by 0x41D6CD: aeProcessEvents (ae.c:311)
   by 0x41D8EA: aeMain (ae.c:455)
   by 0x41A84B: main (redis.c:3832)
 Uninitialised value was created by a stack allocation
   at 0x457810: nodeUpdateAddressIfNeeded (cluster.c:1236)
2015-01-21 17:50:17 +01:00
antirez acb1d8debf Cluster test: wait for port to unbound in kill_instance.
Otherwise kill_instance + restart_instance in short succession will
still find the port busy and will fail.
2015-01-21 16:47:36 +01:00
antirez b3bf7584b0 Cluster/Sentinel test: also pause on abort_sentinel_test call. 2015-01-21 16:18:34 +01:00
antirez a5bb0a0774 Cluster/Sentinel test: pause on exceptions as well. 2015-01-21 16:13:30 +01:00
antirez 2601e3e461 Cluster: node deletion cleanup / centralization. 2015-01-21 16:03:43 +01:00
antirez 59ad6ac5fe Cluster: set the slaves->slaveof filed to NULL when master is freed.
Related to issue #2289.
2015-01-21 15:55:53 +01:00
Matt Stancliff 491d57abaa Add --track-origins=yes to valgrind 2015-01-21 15:48:19 +01:00
Matt Stancliff b0146aafeb Tell sentinel/cluster tests to allow valgrind 2015-01-21 15:04:12 +01:00
antirez 92cfab44b2 Fix gcc warning for lack of casting to char pointer. 2015-01-21 14:51:42 +01:00
antirez e467cf5db3 luaRedisGenericCommand(): log error at WARNING level when re-entered.
Rationale is that when re-entering, it is likely due to Lua debugging
hooks. Returning an error will be ignored in most cases, going totally
unnoticed. With the log at least we leave a trace.

Related to issue #2302.
2015-01-20 23:21:21 +01:00
antirez bc8675612f luaRedisGenericCommand() recursion: just return an error.
Instead of calling redisPanic() to abort the server.

Related to issue #2302.
2015-01-20 23:16:19 +01:00
antirez 839767ad0b Panic on recursive calls to luaRedisGenericCommand().
Related to issue #2302.
2015-01-20 18:02:26 +01:00
antirez cf76af6b9f Cluster: fetch my IP even if msg is not MEET for the first time.
In order to avoid that misconfigured cluster nodes at some time may
force an IP update on other nodes, it is required that nodes update
their own address only on MEET messages. However it does not make sense
to do this the first time a node is contacted and yet does not have an
IP, we just risk that myself->ip remains not assigned if there are
messages lost or cluster creation procedures that don't make sure
everybody is targeted by at least one incoming MEET message.

Also fix the logging of the IP switch avoiding the :-1 tail.
2015-01-13 10:50:34 +01:00
antirez 5b0f4a83ac Cluster: clusterMsgDataGossip structure, explict padding + minor stuff.
Also explicitly set version to 0, add a protocol version define, improve
comments in the gossip structure.

Note that the structure layout is the same after the change, we are just
making the padding explicit with an additional not used 16 bits field.
So this commit is still able to talk with the previous versions of
cluster nodes.
2015-01-13 10:40:09 +01:00
antirez 237ab727b9 Suppress valgrind error about write sending uninitialized data.
Valgrind checks that the buffers we transfer via syscalls are all
composed of bytes actually initialized. This is useful, it makes we able
to avoid leaking informations in non initialized parts fo messages
transferred to other hosts. This commit fixes one of such issues.
2015-01-13 09:31:37 +01:00
antirez f08586347d Revert "Use REDIS_SUPERVISED_NONE instead of 0."
This reverts commit 2c925b0c30.

Nevermind.
2015-01-12 15:58:23 +01:00
antirez 2c925b0c30 Use REDIS_SUPERVISED_NONE instead of 0. 2015-01-12 15:57:50 +01:00
antirez 10007cb78c Merge branch 'unstable' of github.com:/antirez/redis into unstable 2015-01-12 15:56:46 +01:00
Salvatore Sanfilippo e6416ca71c Merge pull request #2266 from mattsta/improve/supervised/startup
Three fixes: explicit supervise, pidfile create, remove memory leaks.
2015-01-12 15:56:36 +01:00
antirez 6274a6789d Cluster: initialize mf_end.
Can't be initialized by resetManualFailover() since it's actual state
the function uses, so we need to initialize it at startup time. Not
really a bug in practical terms, but showed up into valgrind and is not
technically correct anyway.
2015-01-12 15:55:00 +01:00
Matt Stancliff eb7d67a3ab Remove RDB AUX memory leaks 2015-01-09 15:19:18 -05:00
Matt Stancliff 36a3b75355 Supervise redis processes only if configured
Adds configuration option 'supervised [no | upstart | systemd | auto]'

Also removed 'bzero' from the previous implementation because it's 2015.
(We could actually statically initialize those structs, but clang
throws an invalid warning when we try, so it looks bad even though it
isn't bad.)

Fixes #2264
2015-01-09 15:19:18 -05:00
Matt Stancliff 5e8b7e4f35 Define default pidfile when creating pid
We want pidfile to be NULL on startup so we can detect if the user
set an explicit value versus only using the default value.

Closes #1967
Fixes #2076
2015-01-09 15:19:18 -05:00
rebx e6d499a446 Create PID file even if in foreground
Previously, Redis only wrote the pid file if
it was daemonizing, but many times it's useful to have
the pid written out even if you're in the foreground.

Some background for this is:
I usually run redis via daemontools. That entails running
redis-server on the foreground. Given that, I'd also want
redis-server to create a pidfile so other processes (e.g. nagios)
can run checks for that.

Closes #463
2015-01-09 15:19:18 -05:00
antirez 0d22121c27 Add "-lrt" in Makefile for Solaris.
This fix is from @NanXiao, however I was not able to retain authorship
because the Pull Request original repository was removed.
2015-01-09 11:53:51 +01:00
antirez 792a94153a Check for __sun macro in solarisfixes.h, not in includers. 2015-01-09 11:23:22 +01:00
antirez f3fd58eb4a Cluster test: also write from Lua script in resharding test. 2015-01-09 11:23:22 +01:00
antirez da95d22ad2 Prevent Lua scripts from violating Redis Cluster keyspace access rules.
Before this commit scripts were able to access / create keys outside the
set of hash slots served by the local node.
2015-01-09 11:23:22 +01:00
Salvatore Sanfilippo 1019c72930 Merge pull request #2265 from mattsta/fix/trib/create
Fix redis-trib creation failure
2015-01-08 19:43:14 +01:00
Matt Stancliff bf58f8b513 Remove end of line whitespace from redis-trib 2015-01-08 13:31:03 -05:00
Matt Stancliff 1c477f62bc Fix redis-trib cluster create
Under certain conditions the node list wasn't being fully populated
and 'create' would fail trying to call methods on nil objects.
2015-01-08 13:28:35 -05:00
antirez 622c69e93d README section about make distclean reworded / extended. 2015-01-08 16:35:05 +01:00
antirez b45e16e7ee Merge branch 'unstable' of github.com:/antirez/redis into unstable 2015-01-08 16:30:04 +01:00
Salvatore Sanfilippo 161ff77572 Merge pull request #2262 from HeartSaVioR/explain-make-distclean
Explain make distclean which seems not well known
2015-01-08 16:29:29 +01:00
Jungtaek Lim 219ab66cc8 Explain make distclean which seems not well known 2015-01-09 00:07:25 +09:00
antirez 42357668e8 Advertise Redis Cluster as experimental in redis.conf. 2015-01-08 14:41:34 +01:00
antirez a7722dc31b Typo fixed: fiels -> fields in rdbSaveInfoAuxFields().
Thx to @badboy.
2015-01-08 12:06:22 +01:00
antirez 4c0e8923a6 A few more AUX info fields added to RDB. 2015-01-08 09:52:59 +01:00
antirez 206cd219b6 RDB AUX fields support.
This commit introduces a new RDB data type called 'aux'. It is used in
order to insert inside an RDB file key-value pairs that may serve
different needs, without breaking backward compatibility when new
informations are embedded inside an RDB file. The contract between Redis
versions is to ignore unknown aux fields when encountered.

Aux fields can be used in order to:

1. Augment the RDB file with info like version of Redis that created the
RDB file, creation time, used memory while the RDB was created, and so
forth.
2. Add state about Redis inside the RDB file that we need to reload
later: replication offset, previos master run ID, in order to improve
failovers safety and allow partial resynchronization after a slave
restart.
3. Anything that we may want to add to RDB files without breaking the
ability of past versions of Redis to load the file.
2015-01-08 09:52:55 +01:00
antirez 1a30e7ded1 rdbLoad() refactoring to make it simpler to follow. 2015-01-08 09:52:51 +01:00
antirez e8614a1a77 New RDB v7 opcode: RESIZEDB.
The new opcode is an hint about the size of the dataset (keys and number
of expires) we are going to load for a given Redis database inside the
RDB file. Since hash tables are resized accordingly ASAP, useless
rehashing is avoided, speeding up load times significantly, in the order
of ~ 20% or more for larger data sets.

Related issue: #1719
2015-01-08 09:52:47 +01:00
antirez 32b10004e2 sdsnative() removed: New rdb.c API can load native strings. 2015-01-08 09:52:44 +01:00
antirez f699b5e801 Use RDB_LOAD_PLAIN to load quicklists and encoded types.
Before we needed to create a string object with an embedded SDS, adn
basically duplicate the SDS part into a plain zmalloc() allocation.
2015-01-08 09:52:40 +01:00
antirez 68bc02c36c RDB refactored to load plain strings from RDB. 2015-01-08 09:52:36 +01:00
Salvatore Sanfilippo 05ba119fbb Merge pull request #2143 from mattsta/quicklist
Quicklist (linked list + ziplist)
2015-01-08 09:51:55 +01:00
Matt Stancliff 5870e22423 Upgrade LZF to 3.6 (2011) from 3.5 (2009)
This is lzf_c and lzf_d from
http://dist.schmorp.de/liblzf/liblzf-3.6.tar.gz
2015-01-02 11:16:10 -05:00
Matt Stancliff 25e12d10be Set optional 'static' for Quicklist+Redis
This also defines REDIS_STATIC='' for building everything
inside src/ and everything inside deps/lua/.
2015-01-02 11:16:10 -05:00
Matt Stancliff 9e11d07909 Add more quicklist info to DEBUG OBJECT
Adds: ql_compressed (boolean, 1 if compression enabled for list, 0
otherwise)
Adds: ql_uncompressed_size (actual uncompressed size of all quicklistNodes)
Adds: ql_ziplist_max (quicklist max ziplist fill factor)

Compression ratio of the list is then ql_uncompressed_size / serializedlength

We report ql_uncompressed_size for all quicklists because serializedlength
is a _compressed_ representation anyway.

Sample output from a large list:
127.0.0.1:6379> llen abc
(integer) 38370061
127.0.0.1:6379> debug object abc
Value at:0x7ff97b51d140 refcount:1 encoding:quicklist serializedlength:19878335 lru:9718164 lru_seconds_idle:5 ql_nodes:21945 ql_avg_node:1748.46 ql_ziplist_max:-2 ql_compressed:0 ql_uncompressed_size:1643187761
(1.36s)

The 1.36s result time is because rdbSavedObjectLen() is serializing the
object, not because of any new stats reporting.

If we run DEBUG OBJECT on a compressed list, DEBUG OBJECT takes almost *zero*
time because rdbSavedObjectLen() reuses already-compressed ziplists:
127.0.0.1:6379> debug object abc
Value at:0x7fe5c5800040 refcount:1 encoding:quicklist serializedlength:19878335 lru:9718109 lru_seconds_idle:5 ql_nodes:21945 ql_avg_node:1748.46 ql_ziplist_max:-2 ql_compressed:1 ql_uncompressed_size:1643187761
2015-01-02 11:16:10 -05:00
Matt Stancliff 02bb515a09 Config: Add quicklist, remove old list options
This removes:
  - list-max-ziplist-entries
  - list-max-ziplist-value

This adds:
  - list-max-ziplist-size
  - list-compress-depth

Also updates config file with new sections and updates
tests to use quicklist settings instead of old list settings.
2015-01-02 11:16:10 -05:00
Matt Stancliff bbbbfb1442 Add branch prediction hints to quicklist
Actually makes a noticeable difference.

Branch hints were selected based on profiler hotspots.
2015-01-02 11:16:10 -05:00