mirrors
/
redis
mirror of https://mirror.osredm.com/root/redis.git
2
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

keyspace - Unify key and value & use dict no_value=1 (#13806) 

The idea of packing the key (`sds`), value (`robj`) and optionally TTL
into a single struct in memory was mentioned a few times in the past by
the community in various flavors. This approach improves memory
efficiency, reduces pointer dereferences for faster lookups, and
simplifies expiration management by keeping all relevant data in one
place. This change goes along with setting keyspace's dict to
no_value=1, and saving considerable amount of memory.

Two more motivations that well aligned with this unification are:

- Prepare the groundwork for replacing EXPIRE scan based implementation
and evaluate instead new `ebuckets` data structure that was introduced
as part of [Hash Field Expiration
feature](https://redis.io/blog/hash-field-expiration-architecture-and-benchmarks/).
Using this data structure requires embedding the ExpireMeta structure
within each object.
- Consider replacing dict with a more space efficient open addressing
approach hash table that might rely on keeping a single pointer to
object.

Before this PR, I POC'ed on a variant of open addressing hash-table and
was surprised to find that dict with no_value actually could provide a
good balance between performance, memory efficiency, and simplicity.
This realization prompted the separation of the unification step from
the evaluation of a new hash table to avoid introducing too many changes
at once and to evaluate its impact independently before considering
replacement of existing hash-table. On an earlier
[commit](https://github.com/redis/redis/pull/13683) I extended dict
no_value optimization (which saves keeping dictEntry where possible) to
be relevant also for objects with even addresses in memory. Combining it
with this unification saves a considerable amount of memory for
keyspace.

# kvobj
This PR adopts Valkey’s
[packing](3eb8314be6)
layout and logic for key, value, and TTL. However, unlike Valkey
implementation, which retained a common `robj` throughout the project,
this PR distinguishes between the general-purpose, overused `robj`, and
the new `kvobj`, which embeds both the key and value and used by the
keyspace. Conceptually, `robj` serves as a base class, while `kvobj`
acts as a derived class.

Two new flags introduced into redis object, `iskvobj` and `expirable`:
```
struct redisObject {
    unsigned type:4;
    unsigned encoding:4;
    unsigned lru:LRU_BITS;
    unsigned iskvobj : 1;             /* new flag */
    unsigned expirable : 1;           /* new flag */
    unsigned refcount : 30;           /* modified: 32bits->30bits */
    void *ptr;
};

typedef struct redisObject robj;
typedef struct redisObject kvobj;
```
When the `iskvobj` flag is set, the object includes also the key and it
is appended to the end of the object. If the `expirable` flag is set, an
additional 8 bytes are added to the object. If the object is of type
string, and the string is rather short, then it will be embedded as
well.

As a result, all keys in the keyspace are promoted to be of type
`kvobj`. This term attempts to align with the existing Redis object,
robj, and the kvstore data structure.

# EXPIRE Implementation
As `kvobj` embeds expiration time as well, looking up expiration times
is now an O(1) operation. And the hash-table of EXPIRE is set now to be
`no_value` mode, directly referencing `kvobj` entries, and in turn,
saves memory.

Next, I plan to evaluate replacing the EXPIRE implementation with the
[ebuckets](https://github.com/redis/redis/blob/unstable/src/ebuckets.h)
data structure, which would eliminate keyspace scans for expired keys.
This requires embedding `ExpireMeta` within each `kvobj` of each key
with expiration. In such implementation, the `expirable` flag will be
shifted to indicate whether `ExpireMeta` is attached.


# Implementation notes

## Manipulating keyspace (find, modify, insert)
Initially, unifying the key and value into a single object and storing
it in dict with `no_value` optimization seemed like a quick win.
However, it (quickly) became clear that this change required deeper
modifications to how keys are manipulated. The challenge was handling
cases where a dictEntry is opt-out due to no_value optimization. In such
cases, many of the APIs that return the dictEntry from a lookup become
insufficient, as it just might be the key itself. To address this issue,
a new-old approach of returning a "link" to the looked-up key's
`dictEntry` instead of the `dictEntry` itself. The term `link` was
already somewhat available in dict API, and is well aligned with the new
dictEntLink declaration:
```
typedef dictEntry **dictEntLink;
```
This PR introduces two new function APIs to dict to leverage returned
link from the search:
```
dictEntLink dictFindLink(dict *d, const void *key, dictEntLink *bucket);
void dictSetKeyAtLink(dict *d, void *key, dictEntLink *link, int newItem);
```
After calling `link = dictFindLink(...)`, any necessary updates must be
performed immediately after by calling `dictSetKeyAtLink()` without any
intervening operations on given dict. Otherwise, `dictEntLink` may
become invalid. Example:
```
/* replace existing key */
link = dictFindLink(d, key, &bucket, 0);
// ... Do something, but don't modify the dict ...
// assert(link != NULL);
dictSetKeyAtLink(d, kv, &link, 0);
     
/* Add new value (If no space for the new key, dict will be expanded and 
   bucket will be looked up again.) */  
link = dictFindLink(d, key, &bucket);
// ... Do something, but don't modify the dict ...
// assert(link == NULL);
dictSetKeyAtLink(d, kv, &bucket, 1);
```
## dict.h 
- The dict API has became cluttered with many unused functions. I have
removed these from dict.h.
- Additionally, APIs specifically related to hash maps (no_value=0),
primarily those handling key-value access, have been gathered and
isolated.
- Removed entirely internal functions ending with “*ByHash()” that were
originally added for optimization and not required any more.
- Few other legacy dict functions were adapted at API level to work with
the term dictEntLink as well.
- Simplified and generalized an optimization that related to comparison
of length of keys of type strings.

## Hash Field Expiration
Until now each hash object with expiration on fields needed to maintain
a reference to its key-name (of the hash object), such that in case it
will be active-expired, then it will be possible to resolve the key-name
for the notification sake. Now there is no need anymore.

---------

Co-authored-by: debing.sun <debing.sun@redis.com>
This commit is contained in:
Moti Cohen 2025-05-12 10:15:17 +03:00 committed by GitHub
parent 4d9b4d6e51
commit e1789e4368
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
44 changed files with 2346 additions and 1760 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

16
src/aof.c
View File

@ -2374,16 +2374,18 @@ int rewriteAppendOnlyFileRio(rio *aof) {
kvs_it = kvstoreIteratorInit(db->keys);
/* Iterate this DB writing every entry */
while((de = kvstoreIteratorNext(kvs_it)) != NULL) {
sds keystr;
robj key, *o;
long long expiretime;
size_t aof_bytes_before_key = aof->processed_bytes;
keystr = dictGetKey(de);
o = dictGetVal(de);
initStaticStringObject(key,keystr);
expiretime = getExpire(db,&key);
/* Get the value object (of type kvobj) */
kvobj *o = dictGetKV(de);
/* Get the expire time */
expiretime = kvobjGetExpire(o);
/* Set on stack string object for key */
robj key;
initStaticStringObject(key, kvobjGetKey(o));
/* Save the key and associated value */
if (o->type == OBJ_STRING) {

46
src/bitops.c
View File

@ -494,16 +494,17 @@ int getBitfieldTypeFromArgument(client *c, robj *o, int *sign, int *bits) {
*
* (Must provide all the arguments to the function)
*/
static robj *lookupStringForBitCommand(client *c, uint64_t maxbit,
static kvobj *lookupStringForBitCommand(client *c, uint64_t maxbit,
size_t *strOldSize, size_t *strGrowSize)
{
dictEntryLink link;
size_t byte = maxbit >> 3;
robj *o = lookupKeyWrite(c->db,c->argv[1]);
kvobj *o = lookupKeyWriteWithLink(c->db,c->argv[1],&link);
if (checkType(c,o,OBJ_STRING)) return NULL;
if (o == NULL) {
o = createObject(OBJ_STRING,sdsnewlen(NULL, byte+1));
dbAdd(c->db,c->argv[1],o);
dbAddByLink(c->db,c->argv[1],&o,&link);
*strGrowSize = byte + 1;
*strOldSize = 0;
} else {
@ -548,7 +549,6 @@ unsigned char *getObjectReadOnlyString(robj *o, long *len, char *llbuf) {
/* SETBIT key offset bitvalue */
void setbitCommand(client *c) {
robj *o;
char *err = "bit is not an integer or out of range";
uint64_t bitoffset;
ssize_t byte, bit;
@ -568,8 +568,8 @@ void setbitCommand(client *c) {
}
size_t strOldSize, strGrowSize;
if ((o = lookupStringForBitCommand(c,bitoffset,&strOldSize,&strGrowSize)) == NULL)
return;
kvobj *o = lookupStringForBitCommand(c, bitoffset, &strOldSize, &strGrowSize);
if (o == NULL) return;
/* Get current values */
byte = bitoffset >> 3;
@ -603,7 +603,6 @@ void setbitCommand(client *c) {
/* GETBIT key offset */
void getbitCommand(client *c) {
robj *o;
char llbuf[32];
uint64_t bitoffset;
size_t byte, bit;
@ -612,16 +611,16 @@ void getbitCommand(client *c) {
if (getBitOffsetFromArgument(c,c->argv[2],&bitoffset,0,0) != C_OK)
return;
if ((o = lookupKeyReadOrReply(c,c->argv[1],shared.czero)) == NULL ||
checkType(c,o,OBJ_STRING)) return;
kvobj *kv = lookupKeyReadOrReply(c, c->argv[1], shared.czero);
if (kv == NULL || checkType(c,kv,OBJ_STRING)) return;
byte = bitoffset >> 3;
bit = 7 - (bitoffset & 0x7);
if (sdsEncodedObject(o)) {
if (byte < sdslen(o->ptr))
bitval = ((uint8_t*)o->ptr)[byte] & (1 << bit);
if (sdsEncodedObject(kv)) {
if (byte < sdslen(kv->ptr))
bitval = ((uint8_t*)kv->ptr)[byte] & (1 << bit);
} else {
if (byte < (size_t)ll2string(llbuf,sizeof(llbuf),(long)o->ptr))
if (byte < (size_t)ll2string(llbuf,sizeof(llbuf),(long)kv->ptr))
bitval = llbuf[byte] & (1 << bit);
}
@ -632,7 +631,7 @@ void getbitCommand(client *c) {
REDIS_NO_SANITIZE("alignment")
void bitopCommand(client *c) {
char *opname = c->argv[1]->ptr;
robj *o, *targetkey = c->argv[2];
robj *targetkey = c->argv[2];
unsigned long op, j, numkeys;
robj **objects; /* Array of source objects. */
unsigned char **src; /* Array of source strings pointers. */
@ -667,9 +666,9 @@ void bitopCommand(client *c) {
len = zmalloc(sizeof(long) * numkeys);
objects = zmalloc(sizeof(robj*) * numkeys);
for (j = 0; j < numkeys; j++) {
o = lookupKeyRead(c->db,c->argv[j+3]);
kvobj *kv = lookupKeyRead(c->db, c->argv[j + 3]);
/* Handle non-existing keys as empty strings. */
if (o == NULL) {
if (kv == NULL) {
objects[j] = NULL;
src[j] = NULL;
len[j] = 0;
@ -677,7 +676,7 @@ void bitopCommand(client *c) {
continue;
}
/* Return an error if one of the keys is not a string. */
if (checkType(c,o,OBJ_STRING)) {
if (checkType(c, kv, OBJ_STRING)) {
unsigned long i;
for (i = 0; i < j; i++) {
if (objects[i])
@ -688,7 +687,7 @@ void bitopCommand(client *c) {
zfree(objects);
return;
}
objects[j] = getDecodedObject(o);
objects[j] = getDecodedObject(kv);
src[j] = objects[j]->ptr;
len[j] = sdslen(objects[j]->ptr);
if (len[j] > maxlen) maxlen = len[j];
@ -805,10 +804,9 @@ void bitopCommand(client *c) {
/* Store the computed value into the target key */
if (maxlen) {
o = createObject(OBJ_STRING,res);
setKey(c,c->db,targetkey,o,0);
robj *o = createObject(OBJ_STRING, res);
setKey(c, c->db, targetkey, &o, 0);
notifyKeyspaceEvent(NOTIFY_STRING,"set",targetkey,c->db->id);
decrRefCount(o);
server.dirty++;
} else if (dbDelete(c->db,targetkey)) {
signalModifiedKey(c,c->db,targetkey);
@ -820,7 +818,7 @@ void bitopCommand(client *c) {
/* BITCOUNT key [start end [BIT|BYTE]] */
void bitcountCommand(client *c) {
robj *o;
kvobj *o;
long long start, end;
long strlen;
unsigned char *p;
@ -912,7 +910,7 @@ void bitcountCommand(client *c) {
/* BITPOS key bit [start [end [BIT|BYTE]]] */
void bitposCommand(client *c) {
robj *o;
kvobj *o;
long long start, end;
long bit, strlen;
unsigned char *p;
@ -1077,7 +1075,7 @@ struct bitfieldOp {
* when flags is set to BITFIELD_FLAG_READONLY: in this case only the
* GET subcommand is allowed, other subcommands will return an error. */
void bitfieldGeneric(client *c, int flags) {
robj *o;
kvobj *o;
uint64_t bitoffset;
int j, numops = 0, changes = 0;
size_t strOldSize, strGrowSize = 0;

2
src/blocked.c
View File

@ -575,7 +575,7 @@ static void handleClientsBlockedOnKey(readyList *rl) {
long count = listLength(clients);
while ((ln = listNext(&li)) && count--) {
client *receiver = listNodeValue(ln);
robj *o = lookupKeyReadWithFlags(rl->db, rl->key, LOOKUP_NOEFFECTS);
kvobj *o = lookupKeyReadWithFlags(rl->db, rl->key, LOOKUP_NOEFFECTS);
/* 1. In case new key was added/touched we need to verify it satisfy the
* blocked type, since we might process the wrong key type.
* 2. We want to serve clients blocked on module keys

64
src/cluster.c
View File

@ -131,7 +131,7 @@ int verifyDumpPayload(unsigned char *p, size_t len, uint16_t *rdbver_ptr) {
* DUMP is actually not used by Redis Cluster but it is the obvious
* complement of RESTORE and can be useful for different applications. */
void dumpCommand(client *c) {
robj *o;
kvobj *o;
rio payload;
/* Check if the key is here. */
@ -239,14 +239,14 @@ void restoreCommand(client *c) {
}
/* Create the key and set the TTL if any */
dictEntry *de = dbAdd(c->db,key,obj);
kvobj *kv = dbAdd(c->db, key, &obj);
/* If minExpiredField was set, then the object is hash with expiration
* on fields and need to register it in global HFE DS */
if (obj->type == OBJ_HASH) {
uint64_t minExpiredField = hashTypeGetMinExpire(obj, 1);
if (kv->type == OBJ_HASH) {
uint64_t minExpiredField = hashTypeGetMinExpire(kv, 1);
if (minExpiredField != EB_EXPIRE_TIME_INVALID)
hashTypeAddToExpires(c->db, dictGetKey(de), obj, minExpiredField);
hashTypeAddToExpires(c->db, kv, minExpiredField);
}
if (ttl) {
@ -259,7 +259,7 @@ void restoreCommand(client *c) {
rewriteClientCommandArgument(c,c->argc,shared.absttl);
}
}
objectSetLRUOrLFU(obj,lfu_freq,lru_idle,lru_clock,1000);
objectSetLRUOrLFU(kv, lfu_freq, lru_idle, lru_clock, 1000);
signalModifiedKey(c,c->db,key);
notifyKeyspaceEvent(NOTIFY_GENERIC,"restore",key,c->db->id);
addReply(c,shared.ok);
@ -388,8 +388,8 @@ void migrateCommand(client *c) {
char *password = NULL;
long timeout;
long dbid;
robj **ov = NULL; /* Objects to migrate. */
robj **kv = NULL; /* Key names. */
robj **kvArray = NULL; /* Objects to migrate. */
robj **keyArray = NULL; /* Key names. */
robj **newargv = NULL; /* Used to rewrite the command as DEL ... keys ... */
rio cmd, payload;
int may_retry = 1;
@ -453,19 +453,19 @@ void migrateCommand(client *c) {
* the caller there was nothing to migrate. We don't return an error in
* this case, since often this is due to a normal condition like the key
* expiring in the meantime. */
ov = zrealloc(ov,sizeof(robj*)*num_keys);
kv = zrealloc(kv,sizeof(robj*)*num_keys);
int oi = 0;
kvArray = zrealloc(kvArray,sizeof(kvobj*)*num_keys);
keyArray = zrealloc(keyArray,sizeof(robj*)*num_keys);
int num_exists = 0;
for (j = 0; j < num_keys; j++) {
if ((ov[oi] = lookupKeyRead(c->db,c->argv[first_key+j])) != NULL) {
kv[oi] = c->argv[first_key+j];
oi++;
if ((kvArray[num_exists] = lookupKeyRead(c->db,c->argv[first_key+j])) != NULL) {
keyArray[num_exists] = c->argv[first_key+j];
num_exists++;
}
}
num_keys = oi;
num_keys = num_exists;
if (num_keys == 0) {
zfree(ov); zfree(kv);
zfree(kvArray); zfree(keyArray);
addReplySds(c,sdsnew("+NOKEY\r\n"));
return;
}
@ -476,7 +476,7 @@ void migrateCommand(client *c) {
/* Connect */
cs = migrateGetSocket(c,c->argv[1],c->argv[2],timeout);
if (cs == NULL) {
zfree(ov); zfree(kv);
zfree(kvArray); zfree(keyArray);
return; /* error sent to the client by migrateGetSocket() */
}
@ -511,7 +511,7 @@ void migrateCommand(client *c) {
/* Create RESTORE payload and generate the protocol to call the command. */
for (j = 0; j < num_keys; j++) {
long long ttl = 0;
long long expireat = getExpire(c->db,kv[j]);
long long expireat = kvobjGetExpire(kvArray[j]);
if (expireat != -1) {
ttl = expireat-commandTimeSnapshot();
@ -524,8 +524,8 @@ void migrateCommand(client *c) {
/* Relocate valid (non expired) keys and values into the array in successive
* positions to remove holes created by the keys that were present
* in the first lookup but are now expired after the second lookup. */
ov[non_expired] = ov[j];
kv[non_expired++] = kv[j];
kvArray[non_expired] = kvArray[j];
keyArray[non_expired++] = keyArray[j];
serverAssertWithInfo(c,NULL,
rioWriteBulkCount(&cmd,'*',replace ? 5 : 4));
@ -535,14 +535,14 @@ void migrateCommand(client *c) {
rioWriteBulkString(&cmd,"RESTORE-ASKING",14));
else
serverAssertWithInfo(c,NULL,rioWriteBulkString(&cmd,"RESTORE",7));
serverAssertWithInfo(c,NULL,sdsEncodedObject(kv[j]));
serverAssertWithInfo(c,NULL,rioWriteBulkString(&cmd,kv[j]->ptr,
sdslen(kv[j]->ptr)));
serverAssertWithInfo(c,NULL,sdsEncodedObject(keyArray[j]));
serverAssertWithInfo(c,NULL,rioWriteBulkString(&cmd,keyArray[j]->ptr,
sdslen(keyArray[j]->ptr)));
serverAssertWithInfo(c,NULL,rioWriteBulkLongLong(&cmd,ttl));
/* Emit the payload argument, that is the serialized object using
* the DUMP format. */
createDumpPayload(&payload,ov[j],kv[j],dbid);
createDumpPayload(&payload,kvArray[j],keyArray[j],dbid);
serverAssertWithInfo(c,NULL,
rioWriteBulkString(&cmd,payload.io.buffer.ptr,
sdslen(payload.io.buffer.ptr)));
@ -622,14 +622,14 @@ void migrateCommand(client *c) {
} else {
if (!copy) {
/* No COPY option: remove the local key, signal the change. */
dbDelete(c->db,kv[j]);
signalModifiedKey(c,c->db,kv[j]);
notifyKeyspaceEvent(NOTIFY_GENERIC,"del",kv[j],c->db->id);
dbDelete(c->db,keyArray[j]);
signalModifiedKey(c,c->db,keyArray[j]);
notifyKeyspaceEvent(NOTIFY_GENERIC,"del",keyArray[j],c->db->id);
server.dirty++;
/* Populate the argument vector to replace the old one. */
newargv[del_idx++] = kv[j];
incrRefCount(kv[j]);
newargv[del_idx++] = keyArray[j];
incrRefCount(keyArray[j]);
}
}
}
@ -687,7 +687,7 @@ void migrateCommand(client *c) {
}
sdsfree(cmd.io.buffer.ptr);
zfree(ov); zfree(kv); zfree(newargv);
zfree(kvArray); zfree(keyArray); zfree(newargv);
return;
/* On socket errors we try to close the cached socket and try again.
@ -714,7 +714,7 @@ void migrateCommand(client *c) {
}
/* Cleanup we want to do if no retry is attempted. */
zfree(ov); zfree(kv);
zfree(kvArray); zfree(keyArray);
addReplyErrorSds(c, sdscatprintf(sdsempty(),
"-IOERR error or timeout %s to target instance",
write_error ? "writing" : "reading"));
@ -1011,7 +1011,7 @@ void clusterCommand(client *c) {
for (unsigned int i = 0; i < numkeys; i++) {
de = kvstoreDictIteratorNext(kvs_di);
serverAssert(de != NULL);
sds sdskey = dictGetKey(de);
sds sdskey = kvobjGetKey(dictGetKV(de));
addReplyBulkCBuffer(c, sdskey, sdslen(sdskey));
}
kvstoreReleaseDictIterator(kvs_di);

2
src/cluster_legacy.c
View File

@ -5793,7 +5793,7 @@ unsigned int delKeysInSlot(unsigned int hashslot) {
kvs_di = kvstoreGetDictSafeIterator(server.db->keys, hashslot);
while((de = kvstoreDictIteratorNext(kvs_di)) != NULL) {
enterExecutionUnit(1, 0);
sds sdskey = dictGetKey(de);
sds sdskey = kvobjGetKey(dictGetKV(de));
robj *key = createStringObject(sdskey, sdslen(sdskey));
dbDelete(&server.db[0], key);
propagateDeletion(&server.db[0], key, server.lazyfree_lazy_server_del);

674
src/db.c
View File

File diff suppressed because it is too large Load Diff

81
src/debug.c
View File

@ -131,7 +131,7 @@ void mixStringObjectDigest(unsigned char *digest, robj *o) {
void xorObjectDigest(redisDb *db, robj *keyobj, unsigned char *digest, robj *o) {
uint32_t aux = htonl(o->type);
mixDigest(digest,&aux,sizeof(aux));
long long expiretime = getExpire(db,keyobj);
long long expiretime = getExpire(db, keyobj->ptr, NULL);
char buf[128];
/* Save the key and associated value */
@ -290,17 +290,16 @@ void computeDatasetDigest(unsigned char *final) {
/* Iterate this DB writing every entry */
while((de = kvstoreIteratorNext(kvs_it)) != NULL) {
sds key;
robj *keyobj, *o;
robj *keyobj;
memset(digest,0,20); /* This key-val digest */
key = dictGetKey(de);
kvobj *kv = dictGetKV(de);
sds key = kvobjGetKey(kv);
keyobj = createStringObject(key,sdslen(key));
mixDigest(digest,key,sdslen(key));
o = dictGetVal(de);
xorObjectDigest(db,keyobj,digest,o);
xorObjectDigest(db, keyobj, digest, kv);
/* We can finally xor the key-val digest to the final digest */
xorDigest(final,digest,20);
@ -605,22 +604,21 @@ NULL
server.cluster_drop_packet_filter = packet_type;
addReply(c,shared.ok);
} else if (!strcasecmp(c->argv[1]->ptr,"object") && c->argc == 3) {
dictEntry *de;
robj *val;
kvobj *kv;
char *strenc;
if ((de = dbFind(c->db, c->argv[2]->ptr)) == NULL) {
if ((kv = dbFind(c->db, c->argv[2]->ptr)) == NULL) {
addReplyErrorObject(c,shared.nokeyerr);
return;
}
val = dictGetVal(de);
strenc = strEncoding(val->encoding);
strenc = strEncoding(kv->encoding);
char extra[138] = {0};
if (val->encoding == OBJ_ENCODING_QUICKLIST) {
if (kv->encoding == OBJ_ENCODING_QUICKLIST) {
char *nextra = extra;
int remaining = sizeof(extra);
quicklist *ql = val->ptr;
quicklist *ql = kv->ptr;
/* Add number of quicklist nodes */
int used = snprintf(nextra, remaining, " ql_nodes:%lu", ql->len);
nextra += used;
@ -653,38 +651,42 @@ NULL
"Value at:%p refcount:%d "
"encoding:%s serializedlength:%zu "
"lru:%d lru_seconds_idle:%llu%s",
(void*)val, val->refcount,
strenc, rdbSavedObjectLen(val, c->argv[2], c->db->id),
val->lru, estimateObjectIdleTime(val)/1000, extra);
(void*)kv, kv->refcount,
strenc, rdbSavedObjectLen(kv, c->argv[2], c->db->id),
kv->lru, estimateObjectIdleTime(kv)/1000, extra);
} else if (!strcasecmp(c->argv[1]->ptr,"sdslen") && c->argc == 3) {
dictEntry *de;
robj *val;
sds key;
kvobj *kv;
if ((de = dbFind(c->db, c->argv[2]->ptr)) == NULL) {
if ((kv = dbFind(c->db, c->argv[2]->ptr)) == NULL) {
addReplyErrorObject(c,shared.nokeyerr);
return;
}
val = dictGetVal(de);
key = dictGetKey(de);
if (val->type != OBJ_STRING || !sdsEncodedObject(val)) {
val = kv;
key = kvobjGetKey(kv);
if (kv->type != OBJ_STRING || !sdsEncodedObject(val)) {
addReplyError(c,"Not an sds encoded string.");
} else {
/* The key's allocation size reflects the entire robj allocation.
* For embedded values, report an allocation size of 0. */
size_t obj_alloc = zmalloc_usable_size(val);
size_t val_alloc = val->encoding == OBJ_ENCODING_RAW ? sdsAllocSize(val->ptr) : 0;
addReplyStatusFormat(c,
"key_sds_len:%lld, key_sds_avail:%lld, key_zmalloc: %lld, "
"val_sds_len:%lld, val_sds_avail:%lld, val_zmalloc: %lld",
(long long) sdslen(key),
(long long) sdsavail(key),
(long long) sdsZmallocSize(key),
(long long) obj_alloc,
(long long) sdslen(val->ptr),
(long long) sdsavail(val->ptr),
(long long) getStringObjectSdsUsedMemory(val));
(long long) val_alloc);
}
} else if (!strcasecmp(c->argv[1]->ptr,"listpack") && c->argc == 3) {
robj *o;
kvobj *o;
if ((o = objectCommandLookupOrReply(c,c->argv[2],shared.nokeyerr))
if ((o = kvobjCommandLookupOrReply(c, c->argv[2], shared.nokeyerr))
== NULL) return;
if (o->encoding != OBJ_ENCODING_LISTPACK && o->encoding != OBJ_ENCODING_LISTPACK_EX) {
@ -698,9 +700,9 @@ NULL
addReplyStatus(c,"Listpack structure printed on stdout");
}
} else if (!strcasecmp(c->argv[1]->ptr,"quicklist") && (c->argc == 3 || c->argc == 4)) {
robj *o;
kvobj *o;
if ((o = objectCommandLookupOrReply(c,c->argv[2],shared.nokeyerr))
if ((o = kvobjCommandLookupOrReply(c, c->argv[2], shared.nokeyerr))
== NULL) return;
int full = 0;
@ -750,7 +752,7 @@ NULL
val = createStringObject(NULL,valsize);
memcpy(val->ptr, buf, valsize<=buflen? valsize: buflen);
}
dbAdd(c->db,key,val);
dbAdd(c->db, key, &val);
signalModifiedKey(c,c->db,key);
decrRefCount(key);
}
@ -782,8 +784,7 @@ NULL
/* We don't use lookupKey because a debug command should
* work on logically expired keys */
dictEntry *de;
robj *o = ((de = dbFind(c->db, c->argv[j]->ptr)) == NULL) ? NULL : dictGetVal(de);
kvobj *o = dbFind(c->db, c->argv[j]->ptr);
if (o) xorObjectDigest(c->db,c->argv[j],digest,o);
sds d = sdsempty();
@ -901,7 +902,7 @@ NULL
sds sizes = sdsempty();
sizes = sdscatprintf(sizes,"bits:%d ",(sizeof(void*) == 8)?64:32);
sizes = sdscatprintf(sizes,"robj:%d ",(int)sizeof(robj));
sizes = sdscatprintf(sizes,"dictentry:%d ",(int)dictEntryMemUsage());
sizes = sdscatprintf(sizes,"dictentry:%d ",(int)dictEntryMemUsage(0));
sizes = sdscatprintf(sizes,"sdshdr5:%d ",(int)sizeof(struct sdshdr5));
sizes = sdscatprintf(sizes,"sdshdr8:%d ",(int)sizeof(struct sdshdr8));
sizes = sdscatprintf(sizes,"sdshdr16:%d ",(int)sizeof(struct sdshdr16));
@ -937,14 +938,14 @@ NULL
addReplyVerbatim(c,stats,sdslen(stats),"txt");
sdsfree(stats);
} else if (!strcasecmp(c->argv[1]->ptr,"htstats-key") && c->argc >= 3) {
robj *o;
kvobj *o;
dict *ht = NULL;
int full = 0;
if (c->argc >= 4 && !strcasecmp(c->argv[3]->ptr,"full"))
full = 1;
if ((o = objectCommandLookupOrReply(c,c->argv[2],shared.nokeyerr))
if ((o = kvobjCommandLookupOrReply(c,c->argv[2],shared.nokeyerr))
== NULL) return;
/* Get the hash table reference from the object, if possible. */
@ -2190,15 +2191,11 @@ void logCurrentClient(client *cc, const char *title) {
/* Check if the first argument, usually a key, is found inside the
* selected DB, and if so print info about the associated object. */
if (cc->argc > 1) {
robj *val, *key;
dictEntry *de;
key = getDecodedObject(cc->argv[1]);
de = dbFind(cc->db, key->ptr);
if (de) {
val = dictGetVal(de);
robj *key = getDecodedObject(cc->argv[1]);
kvobj *kv = dbFind(cc->db, key->ptr);
if (kv) {
serverLog(LL_WARNING,"key '%s' found in DB containing the following object:", (char*)key->ptr);
serverLogObjectDebugInfo(val);
serverLogObjectDebugInfo(kv);
}
decrRefCount(key);
}

146
src/defrag.c
View File

@ -375,12 +375,14 @@ void activeDefragZsetEntry(zset *zs, dictEntry *de) {
#define DEFRAG_SDS_DICT_VAL_VOID_PTR 3
#define DEFRAG_SDS_DICT_VAL_LUA_SCRIPT 4
void activeDefragSdsDictCallback(void *privdata, const dictEntry *de) {
void activeDefragSdsDictCallback(void *privdata, const dictEntry *de, dictEntryLink plink) {
UNUSED(plink);
UNUSED(privdata);
UNUSED(de);
}
void activeDefragHfieldDictCallback(void *privdata, const dictEntry *de) {
void activeDefragHfieldDictCallback(void *privdata, const dictEntry *de, dictEntryLink plink) {
UNUSED(plink);
dict *d = privdata;
hfield newhf, hf = dictGetKey(de);
@ -392,17 +394,8 @@ void activeDefragHfieldDictCallback(void *privdata, const dictEntry *de) {
ebuckets *eb = hashTypeGetDictMetaHFE(d);
newhf = ebDefragItem(eb, &hashFieldExpireBucketsType, hf, (ebDefragFunction *)activeDefragHfield);
}
if (newhf) {
/* We can't search in dict for that key after we've released
* the pointer it holds, since it won't be able to do the string
* compare, but we can find the entry using key hash and pointer. */
dictUseStoredKeyApi(d, 1);
uint64_t hash = dictGetHash(d, newhf);
dictUseStoredKeyApi(d, 0);
dictEntry *de = dictFindByHashAndPtr(d, hf, hash);
serverAssert(de);
dictSetKey(d, de, newhf);
}
if (newhf) dictSetKey(d, (dictEntry *) de, newhf);
}
/* Defrag a dict with sds key and optional value (either ptr, sds or robj string) */
@ -467,13 +460,13 @@ void activeDefragQuickListNodes(quicklist *ql) {
/* when the value has lots of elements, we want to handle it later and not as
* part of the main dictionary scan. this is needed in order to prevent latency
* spikes when handling large items */
void defragLater(defragKeysCtx *ctx, dictEntry *kde) {
void defragLater(defragKeysCtx *ctx, kvobj *kv) {
if (!ctx->defrag_later) {
ctx->defrag_later = listCreate();
listSetFreeMethod(ctx->defrag_later, sdsfreegeneric);
ctx->defrag_later_cursor = 0;
}
sds key = sdsdup(dictGetKey(kde));
sds key = sdsdup(kvobjGetKey(kv));
listAddNodeTail(ctx->defrag_later, key);
}
@ -524,7 +517,8 @@ typedef struct {
zset *zs;
} scanLaterZsetData;
void scanLaterZsetCallback(void *privdata, const dictEntry *_de) {
void scanLaterZsetCallback(void *privdata, const dictEntry *_de, dictEntryLink plink) {
UNUSED(plink);
dictEntry *de = (dictEntry*)_de;
scanLaterZsetData *data = privdata;
activeDefragZsetEntry(data->zs, de);
@ -541,7 +535,8 @@ void scanLaterZset(robj *ob, unsigned long *cursor) {
}
/* Used as scan callback when all the work is done in the dictDefragFunctions. */
void scanCallbackCountScanned(void *privdata, const dictEntry *de) {
void scanCallbackCountScanned(void *privdata, const dictEntry *de, dictEntryLink plink) {
UNUSED(plink);
UNUSED(privdata);
UNUSED(de);
server.stat_active_defrag_scanned++;
@ -568,20 +563,18 @@ void scanLaterHash(robj *ob, unsigned long *cursor) {
*cursor = dictScanDefrag(d, *cursor, activeDefragHfieldDictCallback, &defragfns, d);
}
void defragQuicklist(defragKeysCtx *ctx, dictEntry *kde) {
robj *ob = dictGetVal(kde);
quicklist *ql = ob->ptr, *newql;
serverAssert(ob->type == OBJ_LIST && ob->encoding == OBJ_ENCODING_QUICKLIST);
void defragQuicklist(defragKeysCtx *ctx, kvobj *kv) {
quicklist *ql = kv->ptr, *newql;
serverAssert(kv->type == OBJ_LIST && kv->encoding == OBJ_ENCODING_QUICKLIST);
if ((newql = activeDefragAlloc(ql)))
ob->ptr = ql = newql;
kv->ptr = ql = newql;
if (ql->len > server.active_defrag_max_scan_fields)
defragLater(ctx, kde);
defragLater(ctx, kv);
else
activeDefragQuickListNodes(ql);
}
void defragZsetSkiplist(defragKeysCtx *ctx, dictEntry *kde) {
robj *ob = dictGetVal(kde);
void defragZsetSkiplist(defragKeysCtx *ctx, kvobj *ob) {
zset *zs = (zset*)ob->ptr;
zset *newzs;
zskiplist *newzsl;
@ -596,7 +589,7 @@ void defragZsetSkiplist(defragKeysCtx *ctx, dictEntry *kde) {
if ((newheader = activeDefragAlloc(zs->zsl->header)))
zs->zsl->header = newheader;
if (dictSize(zs->dict) > server.active_defrag_max_scan_fields)
defragLater(ctx, kde);
defragLater(ctx, ob);
else {
dictIterator *di = dictGetIterator(zs->dict);
while((de = dictNext(di)) != NULL) {
@ -609,13 +602,12 @@ void defragZsetSkiplist(defragKeysCtx *ctx, dictEntry *kde) {
zs->dict = newdict;
}
void defragHash(defragKeysCtx *ctx, dictEntry *kde) {
robj *ob = dictGetVal(kde);
void defragHash(defragKeysCtx *ctx, kvobj *ob) {
dict *d, *newd;
serverAssert(ob->type == OBJ_HASH && ob->encoding == OBJ_ENCODING_HT);
d = ob->ptr;
if (dictSize(d) > server.active_defrag_max_scan_fields)
defragLater(ctx, kde);
defragLater(ctx, ob);
else
activeDefragHfieldDict(d);
/* defrag the dict struct and tables */
@ -623,13 +615,12 @@ void defragHash(defragKeysCtx *ctx, dictEntry *kde) {
ob->ptr = newd;
}
void defragSet(defragKeysCtx *ctx, dictEntry *kde) {
robj *ob = dictGetVal(kde);
void defragSet(defragKeysCtx *ctx, kvobj *ob) {
dict *d, *newd;
serverAssert(ob->type == OBJ_SET && ob->encoding == OBJ_ENCODING_HT);
d = ob->ptr;
if (dictSize(d) > server.active_defrag_max_scan_fields)
defragLater(ctx, kde);
defragLater(ctx, ob);
else
activeDefragSdsDict(d, DEFRAG_SDS_DICT_NO_VAL);
/* defrag the dict struct and tables */
@ -783,8 +774,7 @@ void* defragStreamConsumerGroup(raxIterator *ri, void *privdata) {
return NULL;
}
void defragStream(defragKeysCtx *ctx, dictEntry *kde) {
robj *ob = dictGetVal(kde);
void defragStream(defragKeysCtx *ctx, kvobj *ob) {
serverAssert(ob->type == OBJ_STREAM && ob->encoding == OBJ_ENCODING_STREAM);
stream *s = ob->ptr, *news;
@ -796,7 +786,7 @@ void defragStream(defragKeysCtx *ctx, dictEntry *kde) {
rax *newrax = activeDefragAlloc(s->rax);
if (newrax)
s->rax = newrax;
defragLater(ctx, kde);
defragLater(ctx, ob);
} else
defragRadixTree(&s->rax, 1, NULL, NULL);
@ -807,62 +797,53 @@ void defragStream(defragKeysCtx *ctx, dictEntry *kde) {
/* Defrag a module key. This is either done immediately or scheduled
* for later. Returns then number of pointers defragged.
*/
void defragModule(defragKeysCtx *ctx, redisDb *db, dictEntry *kde) {
robj *obj = dictGetVal(kde);
serverAssert(obj->type == OBJ_MODULE);
void defragModule(defragKeysCtx *ctx, redisDb *db, kvobj *kv) {
serverAssert(kv->type == OBJ_MODULE);
robj keyobj;
initStaticStringObject(keyobj, dictGetKey(kde));
if (!moduleDefragValue(&keyobj, obj, db->id))
defragLater(ctx, kde);
initStaticStringObject(keyobj, kvobjGetKey(kv));
if (!moduleDefragValue(&keyobj, kv, db->id))
defragLater(ctx, kv);
}
/* for each key we scan in the main dict, this function will attempt to defrag
* all the various pointers it has. */
void defragKey(defragKeysCtx *ctx, dictEntry *de) {
void defragKey(defragKeysCtx *ctx, dictEntry *de, dictEntryLink link) {
dictEntryLink exlink = NULL;
kvobj *kvnew, *ob = dictGetKV(de);
redisDb *db = &server.db[ctx->dbid];
int slot = ctx->kvstate.slot;
sds keysds = dictGetKey(de);
robj *newob, *ob = dictGetVal(de);
unsigned char *newzl;
sds newsds;
/* Try to defrag the key name. */
newsds = activeDefragSds(keysds);
if (newsds) {
kvstoreDictSetKey(db->keys, slot, de, newsds);
if (kvstoreDictSize(db->expires, slot)) {
/* We can't search in db->expires for that key after we've released
* the pointer it holds, since it won't be able to do the string
* compare, but we can find the entry using key hash and pointer. */
uint64_t hash = kvstoreGetHash(db->expires, newsds);
dictEntry *expire_de = kvstoreDictFindByHashAndPtr(db->expires, slot, keysds, hash);
if (expire_de) kvstoreDictSetKey(db->expires, slot, expire_de, newsds);
}
/* Update the key's reference in the dict's metadata or the listpackEx. */
if (unlikely(ob->type == OBJ_HASH))
hashTypeUpdateKeyRef(ob, newsds);
}
long long expire = kvobjGetExpire(ob);
/* We can't search in db->expires for that KV after we've released
* the pointer it holds, since it won't be able to do the string
* compare. Search it before, if needed. */
if (expire != -1) {
exlink = kvstoreDictFindLink(db->expires, slot, kvobjGetKey(ob), NULL);
serverAssert(exlink != NULL);
}
/* Try to defrag robj and / or string value. */
if (unlikely(ob->type == OBJ_HASH && hashTypeGetMinExpire(ob, 0) != EB_EXPIRE_TIME_INVALID)) {
/* Update its reference in the ebucket while defragging it. */
newob = ebDefragItem(&db->hexpires, &hashExpireBucketsType, ob,
kvnew = ebDefragItem(&db->hexpires, &hashExpireBucketsType, ob,
(ebDefragFunction *)activeDefragStringOb);
} else {
/* If the dict doesn't have metadata, we directly defrag it. */
newob = activeDefragStringOb(ob);
kvnew = activeDefragStringOb(ob);
}
if (newob) {
kvstoreDictSetVal(db->keys, slot, de, newob);
ob = newob;
if (kvnew) {
kvstoreDictSetAtLink(db->keys, slot, kvnew, &link, 0);
if (expire != -1)
kvstoreDictSetAtLink(db->expires, slot, kvnew, &exlink, 0);
ob = kvnew;
}
if (ob->type == OBJ_STRING) {
/* Already handled in activeDefragStringOb. */
} else if (ob->type == OBJ_LIST) {
if (ob->encoding == OBJ_ENCODING_QUICKLIST) {
defragQuicklist(ctx, de);
defragQuicklist(ctx, ob);
} else if (ob->encoding == OBJ_ENCODING_LISTPACK) {
if ((newzl = activeDefragAlloc(ob->ptr)))
ob->ptr = newzl;
@ -871,7 +852,7 @@ void defragKey(defragKeysCtx *ctx, dictEntry *de) {
}
} else if (ob->type == OBJ_SET) {
if (ob->encoding == OBJ_ENCODING_HT) {
defragSet(ctx, de);
defragSet(ctx, ob);
} else if (ob->encoding == OBJ_ENCODING_INTSET ||
ob->encoding == OBJ_ENCODING_LISTPACK)
{
@ -886,7 +867,7 @@ void defragKey(defragKeysCtx *ctx, dictEntry *de) {
if ((newzl = activeDefragAlloc(ob->ptr)))
ob->ptr = newzl;
} else if (ob->encoding == OBJ_ENCODING_SKIPLIST) {
defragZsetSkiplist(ctx, de);
defragZsetSkiplist(ctx, ob);
} else {
serverPanic("Unknown sorted set encoding");
}
@ -901,23 +882,23 @@ void defragKey(defragKeysCtx *ctx, dictEntry *de) {
if ((newzl = activeDefragAlloc(lpt->lp)))
lpt->lp = newzl;
} else if (ob->encoding == OBJ_ENCODING_HT) {
defragHash(ctx, de);
defragHash(ctx, ob);
} else {
serverPanic("Unknown hash encoding");
}
} else if (ob->type == OBJ_STREAM) {
defragStream(ctx, de);
defragStream(ctx, ob);
} else if (ob->type == OBJ_MODULE) {
defragModule(ctx,db, de);
defragModule(ctx,db, ob);
} else {
serverPanic("Unknown object type");
}
}
/* Defrag scan callback for the main db dictionary. */
static void dbKeysScanCallback(void *privdata, const dictEntry *de) {
static void dbKeysScanCallback(void *privdata, const dictEntry *de, dictEntryLink plink) {
long long hits_before = server.stat_active_defrag_hits;
defragKey((defragKeysCtx *)privdata, (dictEntry *)de);
defragKey((defragKeysCtx *)privdata, (dictEntry *)de, plink);
if (server.stat_active_defrag_hits != hits_before)
server.stat_active_defrag_key_hits++;
else
@ -960,7 +941,8 @@ float getAllocatorFragmentation(size_t *out_frag_bytes) {
}
/* Defrag scan callback for the pubsub dictionary. */
void defragPubsubScanCallback(void *privdata, const dictEntry *de) {
void defragPubsubScanCallback(void *privdata, const dictEntry *de, dictEntryLink plink) {
UNUSED(plink);
defragPubSubCtx *ctx = privdata;
kvstore *pubsub_channels = ctx->kvstate.kvs;
robj *newchannel, *channel = dictGetKey(de);
@ -996,9 +978,8 @@ void defragPubsubScanCallback(void *privdata, const dictEntry *de) {
/* returns 0 more work may or may not be needed (see non-zero cursor),
* and 1 if time is up and more work is needed. */
int defragLaterItem(dictEntry *de, unsigned long *cursor, monotime endtime, int dbid) {
if (de) {
robj *ob = dictGetVal(de);
int defragLaterItem(kvobj *ob, unsigned long *cursor, monotime endtime, int dbid) {
if (ob) {
if (ob->type == OBJ_LIST && ob->encoding == OBJ_ENCODING_QUICKLIST) {
return scanLaterList(ob, cursor, endtime);
} else if (ob->type == OBJ_SET && ob->encoding == OBJ_ENCODING_HT) {
@ -1011,7 +992,7 @@ int defragLaterItem(dictEntry *de, unsigned long *cursor, monotime endtime, int
return scanLaterStreamListpacks(ob, cursor, endtime);
} else if (ob->type == OBJ_MODULE) {
robj keyobj;
initStaticStringObject(keyobj, dictGetKey(de));
initStaticStringObject(keyobj, kvobjGetKey(ob));
return moduleLateDefrag(&keyobj, ob, cursor, endtime, dbid);
} else {
*cursor = 0; /* object type/encoding may have changed since we schedule it for later */
@ -1038,9 +1019,10 @@ static doneStatus defragLaterStep(void *ctx, monotime endtime) {
listNode *head = listFirst(defrag_keys_ctx->defrag_later);
sds key = head->value;
dictEntry *de = kvstoreDictFind(defrag_keys_ctx->kvstate.kvs, defrag_keys_ctx->kvstate.slot, key);
kvobj *kv = de ? dictGetKV(de) : NULL;
long long key_defragged = server.stat_active_defrag_hits;
int timeout = (defragLaterItem(de, &defrag_keys_ctx->defrag_later_cursor, endtime, defrag_keys_ctx->dbid) == 1);
int timeout = (defragLaterItem(kv, &defrag_keys_ctx->defrag_later_cursor, endtime, defrag_keys_ctx->dbid) == 1);
if (key_defragged != server.stat_active_defrag_hits) {
server.stat_active_defrag_key_hits++;
} else {

491
src/dict.c
View File

@ -61,20 +61,30 @@ typedef struct {
/* -------------------------- private prototypes ---------------------------- */
static void _dictExpandIfNeeded(dict *d);
static int _dictExpandIfNeeded(dict *d);
static void _dictShrinkIfNeeded(dict *d);
static void _dictRehashStepIfNeeded(dict *d, uint64_t visitedIdx);
static signed char _dictNextExp(unsigned long size);
static int _dictInit(dict *d, dictType *type);
static dictEntry *dictGetNext(const dictEntry *de);
static dictEntry **dictGetNextRef(dictEntry *de);
static dictEntryLink dictGetNextLink(dictEntry *de);
static void dictSetNext(dictEntry *de, dictEntry *next);
static int dictDefaultCompare(dict *d, const void *key1, const void *key2);
static int dictDefaultCompare(dictCmpCache *cache, const void *key1, const void *key2);
static dictEntryLink dictFindLinkInternal(dict *d, const void *key, dictEntryLink *bucket);
dictEntryLink dictFindLinkForInsert(dict *d, const void *key, dictEntry **existing);
static dictEntry *dictInsertKeyAtLink(dict *d, void *key, dictEntryLink link);
/* -------------------------- unused --------------------------- */
void dictSetSignedIntegerVal(dictEntry *de, int64_t val);
int64_t dictGetSignedIntegerVal(const dictEntry *de);
double dictIncrDoubleVal(dictEntry *de, double val);
void *dictEntryMetadata(dictEntry *de);
int64_t dictIncrSignedIntegerVal(dictEntry *de, int64_t val);
/* -------------------------- misc inline functions -------------------------------- */
typedef int (*keyCmpFunc)(dict *d, const void *key1, const void *key2);
static inline keyCmpFunc dictGetKeyCmpFunc(dict *d) {
typedef int (*keyCmpFunc)(dictCmpCache *cache, const void *key1, const void *key2);
static inline keyCmpFunc dictGetCmpFunc(dict *d) {
if (d->useStoredKeyApi && d->type->storedKeyCompare)
return d->type->storedKeyCompare;
if (d->type->keyCompare)
@ -97,10 +107,6 @@ void dictSetHashFunctionSeed(uint8_t *seed) {
memcpy(dict_hash_function_seed,seed,sizeof(dict_hash_function_seed));
}
uint8_t *dictGetHashFunctionSeed(void) {
return dict_hash_function_seed;
}
/* The default hashing function uses SipHash implementation
* in siphash.c. */
@ -514,54 +520,21 @@ int dictAdd(dict *d, void *key, void *val)
dictEntry *dictAddRaw(dict *d, void *key, dictEntry **existing)
{
/* Get the position for the new key or NULL if the key already exists. */
void *position = dictFindPositionForInsert(d, key, existing);
void *position = dictFindLinkForInsert(d, key, existing);
if (!position) return NULL;
/* Dup the key if necessary. */
if (d->type->keyDup) key = d->type->keyDup(d, key);
return dictInsertAtPosition(d, key, position);
return dictInsertKeyAtLink(d, key, position);
}
/* Low-level add function for non-existing keys:
* This function adds a new entry to the dictionary, assuming the key does not
* already exist.
* Parameters:
* - `dict *d`: Pointer to the dictionary structure.
* - `void *key`: Pointer to the key being added.
* - `const uint64_t hash`: hash of the key being added.
* Guarantees:
* - The key is assumed to be non-existing.
* Note:
* Ensure that the key's uniqueness is managed externally before calling this function. */
dictEntry *dictAddNonExistsByHash(dict *d, void *key, const uint64_t hash) {
/* Get the position for the new key, it should never be NULL. */
unsigned long idx, table;
idx = hash & DICTHT_SIZE_MASK(d->ht_size_exp[0]);
/* Rehash the hash table if needed */
_dictRehashStepIfNeeded(d,idx);
/* Expand the hash table if needed */
_dictExpandIfNeeded(d);
table = dictIsRehashing(d) ? 1 : 0;
idx = hash & DICTHT_SIZE_MASK(d->ht_size_exp[table]);
void *position = &d->ht_table[table][idx];
assert(position!=NULL);
/* Dup the key if necessary. */
if (d->type->keyDup) key = d->type->keyDup(d, key);
return dictInsertAtPosition(d, key, position);
}
/* Adds a key in the dict's hashtable at the position returned by a preceding
* call to dictFindPositionForInsert. This is a low level function which allows
/* Adds a key in the dict's hashtable at the link returned by a preceding
* call to dictFindLinkForInsert(). This is a low level function which allows
* splitting dictAddRaw in two parts. Normally, dictAddRaw or dictAdd should be
* used instead. */
dictEntry *dictInsertAtPosition(dict *d, void *key, void *position) {
dictEntry **bucket = position; /* It's a bucket, but the API hides that. */
* used instead. It assumes that dictExpandIfNeeded() was called before. */
dictEntry *dictInsertKeyAtLink(dict *d, void *key, dictEntryLink link) {
dictEntryLink bucket = link; /* It's a bucket, but the API hides that. */
dictEntry *entry;
/* If rehashing is ongoing, we insert in table 1, otherwise in table 0.
* Assert that the provided bucket is the right table. */
@ -646,6 +619,7 @@ dictEntry *dictAddOrFind(dict *d, void *key) {
* dictDelete() and dictUnlink(), please check the top comment
* of those functions. */
static dictEntry *dictGenericDelete(dict *d, const void *key, int nofree) {
dictCmpCache cmpCache = {0};
uint64_t h, idx;
dictEntry *he, *prevHe;
int table;
@ -659,7 +633,7 @@ static dictEntry *dictGenericDelete(dict *d, const void *key, int nofree) {
/* Rehash the hash table if needed */
_dictRehashStepIfNeeded(d,idx);
keyCmpFunc cmpFunc = dictGetKeyCmpFunc(d);
keyCmpFunc cmpFunc = dictGetCmpFunc(d);
for (table = 0; table <= 1; table++) {
if (table == 0 && (long)idx < d->rehashidx) continue;
@ -668,7 +642,7 @@ static dictEntry *dictGenericDelete(dict *d, const void *key, int nofree) {
prevHe = NULL;
while(he) {
void *he_key = dictGetKey(he);
if (key == he_key || cmpFunc(d, key, he_key)) {
if (key == he_key || cmpFunc(&cmpCache, key, he_key)) {
/* Unlink the element from the list */
if (prevHe)
dictSetNext(prevHe, dictGetNext(he));
@ -777,55 +751,153 @@ void dictRelease(dict *d)
zfree(d);
}
dictEntry *dictFindByHash(dict *d, const void *key, const uint64_t hash) {
dictEntry *he;
uint64_t idx, table;
if (dictSize(d) == 0) return NULL; /* dict is empty */
/* Finds a given key. Like dictFindLink(), yet search bucket even if dict is empty.
*
* Returns dictEntryLink reference if found. Otherwise, return NULL.
*
* bucket - return pointer to bucket that the key was mapped. unless dict is empty.
*/
static dictEntryLink dictFindLinkInternal(dict *d, const void *key, dictEntryLink *bucket) {
dictCmpCache cmpCache = {0};
dictEntryLink link;
uint64_t idx;
int table;
if (bucket) {
*bucket = NULL;
} else {
/* If dict is empty and no need to find bucket, return NULL */
if (dictSize(d) == 0) return NULL;
}
const uint64_t hash = dictHashKey(d, key, d->useStoredKeyApi);
idx = hash & DICTHT_SIZE_MASK(d->ht_size_exp[0]);
keyCmpFunc cmpFunc = dictGetKeyCmpFunc(d);
keyCmpFunc cmpFunc = dictGetCmpFunc(d);
/* Rehash the hash table if needed */
_dictRehashStepIfNeeded(d,idx);
/* Check if we can use the compare function with length to avoid recomputing length of key always */
keyCmpFuncWithLen cmpFuncWithLen = d->type->keyCompareWithLen;
keyLenFunc keyLenFunc = d->type->keyLen;
const int has_len_fn = (keyLenFunc != NULL && cmpFuncWithLen != NULL);
const size_t key_len = has_len_fn ? keyLenFunc(d,key) : 0;
for (table = 0; table <= 1; table++) {
int tables = (dictIsRehashing(d)) ? 2 : 1;
for (table = 0; table < tables; table++) {
if (table == 0 && (long)idx < d->rehashidx) continue;
idx = hash & DICTHT_SIZE_MASK(d->ht_size_exp[table]);
/* Prefetch the bucket at the calculated index */
redis_prefetch_read(&d->ht_table[table][idx]);
he = d->ht_table[table][idx];
while(he) {
void *he_key = dictGetKey(he);
link = &(d->ht_table[table][idx]);
if (bucket) *bucket = link;
while(link && *link) {
void *visitedKey = dictGetKey(*link);
/* Prefetch the next entry to improve cache efficiency */
redis_prefetch_read(dictGetNext(he));
if (key == he_key || (has_len_fn ?
cmpFuncWithLen(d, key, key_len, he_key, keyLenFunc(d,he_key)) :
cmpFunc(d, key, he_key)))
{
return he;
}
he = dictGetNext(he);
redis_prefetch_read(dictGetNext(*link));
if (key == visitedKey || cmpFunc( &cmpCache, key, visitedKey))
return link;
link = dictGetNextLink(*link);
}
/* Use unlikely to optimize branch prediction for the common case */
if (unlikely(!dictIsRehashing(d))) return NULL;
}
return NULL;
}
dictEntry *dictFind(dict *d, const void *key)
{
if (dictSize(d) == 0) return NULL; /* dict is empty */
const uint64_t hash = dictHashKey(d, key, d->useStoredKeyApi);
return dictFindByHash(d,key,hash);
dictEntryLink link = dictFindLink(d, key, NULL);
return (link) ? *link : NULL;
}
/* Find a key and return its dictEntryLink reference. Otherwise, return NULL
*
* A dictEntryLink pointer being used to find preceding dictEntry of searched item.
* It is Useful for deletion, addition, unlinking and updating, especially for
* dict configured with 'no_value'. In such cases returning only `dictEntry` from
* a lookup may be insufficient since it might be opt-out to be the object itself.
* By locating preceding dictEntry (dictEntryLink) these ops can be properly handled.
*
* After calling link = dictFindLink(...), any necessary updates based on returned
* link or bucket must be performed immediately after by calling dictSetKeyAtLink()
* without any intervening operations on given dict. Otherwise, `dictEntryLink` may
* become invalid. Example with kvobj of replacing key with new key:
*
* link = dictFindLink(d, key, &bucket, 0);
* ... Do something, but don't modify the dict ...
* // assert(link != NULL);
* dictSetKeyAtLink(d, kv, &link, 0);
*
* To add new value (If no space for the new key, dict will be expanded by
* dictSetKeyAtLink() and bucket will be looked up again.):
*
* link = dictFindLink(d, key, &bucket);
* ... Do something, but don't modify the dict ...
* // assert(link == NULL);
* dictSetKeyAtLink(d, kv, &bucket, 1);
*
* bucket - return link to bucket that the key was mapped. unless dict is empty.
*/
dictEntryLink dictFindLink(dict *d, const void *key, dictEntryLink *bucket) {
if (bucket) *bucket = NULL;
if (unlikely(dictSize(d) == 0))
return NULL;
return dictFindLinkInternal(d, key, bucket);
}
/* Set the key with link
*
* link: - When `newItem` is set, `link` points to the bucket of the key.
* - When `newItem` is not set, `link` points to the link of the key.
* - If *link is NULL, dictFindLink() will be called to locate the key.
* - On return, get updated, by need, to the inserted key.
*
* newItem: 1 = Add a key with a new dictEntry.
* 0 = Set a key to an existing dictEntry.
*/
void dictSetKeyAtLink(dict *d, void *key, dictEntryLink *link, int newItem) {
dictEntryLink dummy = NULL;
if (link == NULL) link = &dummy;
void *addedKey = (d->type->keyDup) ? d->type->keyDup(d, key) : key;
if (newItem) {
signed char snap[2] = {d->ht_size_exp[0], d->ht_size_exp[1] };
/* Make room if needed for the new key */
dictExpandIfNeeded(d);
/* Lookup key's link if tables reallocated or if given link is set to NULL */
if (snap[0] != d->ht_size_exp[0] || snap[1] != d->ht_size_exp[1] || *link == NULL) {
dictEntryLink bucket;
/* Bypass dictFindLink() to search bucket even if dict is empty!!! */
dictUseStoredKeyApi(d, 1);
*link = dictFindLinkInternal(d, key, &bucket);
dictUseStoredKeyApi(d, 0);
assert(bucket != NULL);
assert(*link == NULL);
*link = bucket; /* On newItem the link should be the bucket */
}
dictInsertKeyAtLink(d, addedKey, *link);
return;
}
/* Setting key of existing dictEntry (newItem == 0)*/
if (*link == NULL) {
*link = dictFindLink(d, key, NULL);
assert(*link != NULL);
}
dictEntry **de = *link;
/* is it regular dict entry of key and next */
if (entryIsNoValue(*de)) {
decodeEntryNoValue(*de)->key = addedKey;
} else if (entryIsKey(*de)) {
/* `de` opt-out to be actually a key. Replace key but keep the lsb flags */
int mask = ((uintptr_t) *de) & ENTRY_PTR_MASK;
*de = encodeMaskedPtr(addedKey, mask);
} else {
(*de)->key = addedKey; /* `de` is a normal key-value dict entry */
}
}
void *dictFetchValue(dict *d, const void *key) {
@ -835,30 +907,31 @@ void *dictFetchValue(dict *d, const void *key) {
return he ? dictGetVal(he) : NULL;
}
/* Find an element from the table, also get the plink of the entry. The entry
* is returned if the element is found, and the user should later call
* `dictTwoPhaseUnlinkFree` with it in order to unlink and release it. Otherwise if
* the key is not found, NULL is returned. These two functions should be used in pair.
/* Find an element from the table. A link is returned if the element is found, and
* the user should later call `dictTwoPhaseUnlinkFree` with it in order to unlink
* and release it. Otherwise if the key is not found, NULL is returned. These two
* functions should be used in pair.
* `dictTwoPhaseUnlinkFind` pauses rehash and `dictTwoPhaseUnlinkFree` resumes rehash.
*
* We can use like this:
*
* dictEntry *de = dictTwoPhaseUnlinkFind(db->dict,key->ptr,&plink, &table);
* dictEntryLink link = dictTwoPhaseUnlinkFind(db->dict,key->ptr, &table);
* // Do something, but we can't modify the dict
* dictTwoPhaseUnlinkFree(db->dict,de,plink,table); // We don't need to lookup again
* dictTwoPhaseUnlinkFree(db->dict, link, table); // We don't need to lookup again
*
* If we want to find an entry before delete this entry, this an optimization to avoid
* dictFind followed by dictDelete. i.e. the first API is a find, and it gives some info
* to the second one to avoid repeating the lookup
*/
dictEntry *dictTwoPhaseUnlinkFind(dict *d, const void *key, dictEntry ***plink, int *table_index) {
dictEntryLink dictTwoPhaseUnlinkFind(dict *d, const void *key, int *table_index) {
dictCmpCache cmpCache = {0};
uint64_t h, idx, table;
if (dictSize(d) == 0) return NULL; /* dict is empty */
if (dictIsRehashing(d)) _dictRehashStep(d);
h = dictHashKey(d, key, d->useStoredKeyApi);
keyCmpFunc cmpFunc = dictGetKeyCmpFunc(d);
h = dictHashKey(d, key, d->useStoredKeyApi);
keyCmpFunc cmpFunc = dictGetCmpFunc(d);
for (table = 0; table <= 1; table++) {
idx = h & DICTHT_SIZE_MASK(d->ht_size_exp[table]);
@ -866,26 +939,27 @@ dictEntry *dictTwoPhaseUnlinkFind(dict *d, const void *key, dictEntry ***plink,
dictEntry **ref = &d->ht_table[table][idx];
while (ref && *ref) {
void *de_key = dictGetKey(*ref);
if (key == de_key || cmpFunc(d, key, de_key)) {
if (key == de_key || cmpFunc(&cmpCache, key, de_key)) {
*table_index = table;
*plink = ref;
dictPauseRehashing(d);
return *ref;
return ref;
}
ref = dictGetNextRef(*ref);
ref = dictGetNextLink(*ref);
}
if (!dictIsRehashing(d)) return NULL;
}
return NULL;
}
void dictTwoPhaseUnlinkFree(dict *d, dictEntry *he, dictEntry **plink, int table_index) {
if (he == NULL) return;
void dictTwoPhaseUnlinkFree(dict *d, dictEntryLink plink, int table_index) {
if (plink == NULL || *plink == NULL) return;
dictEntry *de = *plink;
d->ht_used[table_index]--;
*plink = dictGetNext(he);
dictFreeKey(d, he);
dictFreeVal(d, he);
if (!entryIsKey(he)) zfree(decodeMaskedPtr(he));
*plink = dictGetNext(de);
dictFreeKey(d, de);
dictFreeVal(d, de);
if (!entryIsKey(de)) zfree(decodeMaskedPtr(de));
_dictShrinkIfNeeded(d);
dictResumeRehashing(d);
}
@ -978,7 +1052,7 @@ static dictEntry *dictGetNext(const dictEntry *de) {
/* Returns a pointer to the 'next' field in the entry or NULL if the entry
* doesn't have a next field. */
static dictEntry **dictGetNextRef(dictEntry *de) {
static dictEntryLink dictGetNextLink(dictEntry *de) {
if (entryIsKey(de)) return NULL;
if (entryIsNoValue(de)) return &decodeEntryNoValue(de)->next;
return &de->next;
@ -1001,8 +1075,8 @@ size_t dictMemUsage(const dict *d) {
dictBuckets(d) * sizeof(dictEntry*);
}
size_t dictEntryMemUsage(void) {
return sizeof(dictEntry);
size_t dictEntryMemUsage(int noValueDict) {
return (noValueDict) ? sizeof(dictEntryNoValue) :sizeof(dictEntry);
}
/* A fingerprint is a 64 bit number that represents the state of the dictionary
@ -1284,7 +1358,6 @@ static void dictDefragBucket(dictEntry **bucketref, dictDefragFunctions *defragf
void *newval = defragval ? defragval(dictGetVal(de)) : NULL;
if (entryIsKey(de)) {
if (newkey) *bucketref = newkey;
assert(entryIsKey(*bucketref));
} else if (entryIsNoValue(de)) {
dictEntryNoValue *entry = decodeEntryNoValue(de), *newentry;
if ((newentry = defragalloc(entry))) {
@ -1302,7 +1375,7 @@ static void dictDefragBucket(dictEntry **bucketref, dictDefragFunctions *defragf
if (newde) {
*bucketref = newde;
}
bucketref = dictGetNextRef(*bucketref);
bucketref = dictGetNextLink(*bucketref);
}
}
@ -1434,6 +1507,32 @@ unsigned long dictScan(dict *d,
return dictScanDefrag(d, v, fn, NULL, privdata);
}
void dictScanDefragBucket(dictScanFunction *fn,
dictDefragFunctions *defragfns,
void *privdata,
dictEntry **bucketref) {
dictEntry **plink, *de, *next;
/* Emit entries at bucket */
if (defragfns) dictDefragBucket(bucketref, defragfns);
de = *bucketref;
plink = bucketref;
while (de) {
next = dictGetNext(de);
fn(privdata, de, plink);
if (!next) break; /* if last element, break */
/* if `*plink` still pointing to 'de', then it means that the
* visited item wasn't deleted by fn() */
if (*plink == de)
plink = (entryIsNoValue(de)) ? &(decodeEntryNoValue(de)->next) : &(de->next);
de = next;
}
}
/* Like dictScan, but additionally reallocates the memory used by the dict
* entries using the provided allocation function. This feature was added for
* the active defrag feature.
@ -1449,7 +1548,6 @@ unsigned long dictScanDefrag(dict *d,
void *privdata)
{
int htidx0, htidx1;
const dictEntry *de, *next;
unsigned long m0, m1;
if (dictSize(d) == 0) return 0;
@ -1460,17 +1558,7 @@ unsigned long dictScanDefrag(dict *d,
if (!dictIsRehashing(d)) {
htidx0 = 0;
m0 = DICTHT_SIZE_MASK(d->ht_size_exp[htidx0]);
/* Emit entries at cursor */
if (defragfns) {
dictDefragBucket(&d->ht_table[htidx0][v & m0], defragfns);
}
de = d->ht_table[htidx0][v & m0];
while (de) {
next = dictGetNext(de);
fn(privdata, de);
de = next;
}
dictScanDefragBucket(fn, defragfns, privdata, &d->ht_table[htidx0][v & m0]);
/* Set unmasked bits so incrementing the reversed cursor
* operates on the masked bits */
@ -1494,30 +1582,12 @@ unsigned long dictScanDefrag(dict *d,
m0 = DICTHT_SIZE_MASK(d->ht_size_exp[htidx0]);
m1 = DICTHT_SIZE_MASK(d->ht_size_exp[htidx1]);
/* Emit entries at cursor */
if (defragfns) {
dictDefragBucket(&d->ht_table[htidx0][v & m0], defragfns);
}
de = d->ht_table[htidx0][v & m0];
while (de) {
next = dictGetNext(de);
fn(privdata, de);
de = next;
}
dictScanDefragBucket(fn, defragfns, privdata, &d->ht_table[htidx0][v & m0]);
/* Iterate over indices in larger table that are the expansion
* of the index pointed to by the cursor in the smaller table */
do {
/* Emit entries at cursor */
if (defragfns) {
dictDefragBucket(&d->ht_table[htidx1][v & m1], defragfns);
}
de = d->ht_table[htidx1][v & m1];
while (de) {
next = dictGetNext(de);
fn(privdata, de);
de = next;
}
dictScanDefragBucket(fn, defragfns, privdata, &d->ht_table[htidx1][v & m1]);
/* Increment the reverse cursor not covered by the smaller mask.*/
v |= ~m1;
@ -1575,12 +1645,12 @@ int dictExpandIfNeeded(dict *d) {
return DICT_ERR;
}
/* Expand the hash table if needed */
static void _dictExpandIfNeeded(dict *d) {
/* Expand the hash table if needed (OK=Expanded, ERR=Not expanded) */
static int _dictExpandIfNeeded(dict *d) {
/* Automatic resizing is disallowed. Return */
if (d->pauseAutoResize > 0) return;
dictExpandIfNeeded(d);
if (d->pauseAutoResize > 0) return DICT_ERR;
return dictExpandIfNeeded(d);
}
/* Returning DICT_OK indicates a successful shrinking or the dictionary is undergoing rehashing,
@ -1640,12 +1710,13 @@ static signed char _dictNextExp(unsigned long size)
return 8*sizeof(long) - __builtin_clzl(size-1);
}
/* Finds and returns the position within the dict where the provided key should
* be inserted using dictInsertAtPosition if the key does not already exist in
/* Finds and returns the link within the dict where the provided key should
* be inserted using dictInsertKeyAtLink() if the key does not already exist in
* the dict. If the key exists in the dict, NULL is returned and the optional
* 'existing' entry pointer is populated, if provided. */
void *dictFindPositionForInsert(dict *d, const void *key, dictEntry **existing) {
dictEntryLink dictFindLinkForInsert(dict *d, const void *key, dictEntry **existing) {
unsigned long idx, table;
dictCmpCache cmpCache = {0};
dictEntry *he;
uint64_t hash = dictHashKey(d, key, d->useStoredKeyApi);
if (existing) *existing = NULL;
@ -1656,7 +1727,7 @@ void *dictFindPositionForInsert(dict *d, const void *key, dictEntry **existing)
/* Expand the hash table if needed */
_dictExpandIfNeeded(d);
keyCmpFunc cmpFunc = dictGetKeyCmpFunc(d);
keyCmpFunc cmpFunc = dictGetCmpFunc(d);
for (table = 0; table <= 1; table++) {
if (table == 0 && (long)idx < d->rehashidx) continue;
@ -1665,7 +1736,7 @@ void *dictFindPositionForInsert(dict *d, const void *key, dictEntry **existing)
he = d->ht_table[table][idx];
while(he) {
void *he_key = dictGetKey(he);
if (key == he_key || cmpFunc(d, key, he_key)) {
if (key == he_key || cmpFunc(&cmpCache, key, he_key)) {
if (existing) *existing = he;
return NULL;
}
@ -1706,30 +1777,6 @@ uint64_t dictGetHash(dict *d, const void *key) {
return dictHashKey(d, key, d->useStoredKeyApi);
}
/* Finds the dictEntry using pointer and pre-calculated hash.
* oldkey is a dead pointer and should not be accessed.
* the hash value should be provided using dictGetHash.
* no string / key comparison is performed.
* return value is a pointer to the dictEntry if found, or NULL if not found. */
dictEntry *dictFindByHashAndPtr(dict *d, const void *oldptr, const uint64_t hash) {
dictEntry *he;
unsigned long idx, table;
if (dictSize(d) == 0) return NULL; /* dict is empty */
for (table = 0; table <= 1; table++) {
idx = hash & DICTHT_SIZE_MASK(d->ht_size_exp[table]);
if (table == 0 && (long)idx < d->rehashidx) continue;
he = d->ht_table[table][idx];
while(he) {
if (oldptr == dictGetKey(he))
return he;
he = dictGetNext(he);
}
if (!dictIsRehashing(d)) return NULL;
}
return NULL;
}
/* Provides the old and new ht size for a given dictionary during rehashing. This method
* should only be invoked during initialization/rehashing. */
void dictRehashingInfo(dict *d, unsigned long long *from_size, unsigned long long *to_size) {
@ -1848,8 +1895,8 @@ void dictGetStats(char *buf, size_t bufsize, dict *d, int full) {
orig_buf[orig_bufsize-1] = '\0';
}
static int dictDefaultCompare(dict *d, const void *key1, const void *key2) {
(void)(d); /*unused*/
static int dictDefaultCompare(dictCmpCache *cache, const void *key1, const void *key2) {
(void)(cache); /*unused*/
return key1 == key2;
}
@ -1865,9 +1912,9 @@ uint64_t hashCallback(const void *key) {
return dictGenHashFunction((unsigned char*)key, strlen((char*)key));
}
int compareCallback(dict *d, const void *key1, const void *key2) {
int compareCallback(dictCmpCache *cache, const void *key1, const void *key2) {
int l1,l2;
UNUSED(d);
UNUSED(cache);
l1 = strlen((char*)key1);
l2 = strlen((char*)key2);
@ -2126,57 +2173,6 @@ int dictTest(int argc, char **argv, int flags) {
end_benchmark("Inserting via dictAddRaw() existing (no insertion)");
assert((long)dictSize(d) == count);
dictEmpty(d, NULL);
start_benchmark();
for (j = 0; j < count; j++) {
void *key = stringFromLongLong(j);
const uint64_t hash = dictGetHash(d, key);
de = dictAddNonExistsByHash(d,key,hash);
assert(de != NULL);
}
end_benchmark("Inserting via dictAddNonExistsByHash() non existing");
assert((long)dictSize(d) == count);
/* Wait for rehashing. */
while (dictIsRehashing(d)) {
dictRehashMicroseconds(d,100*1000);
}
dictEmpty(d, NULL);
start_benchmark();
for (j = 0; j < count; j++) {
/* Create a key */
void *key = stringFromLongLong(j);
/* Check if the key exists */
dictEntry *entry = dictFind(d, key);
assert(entry == NULL);
/* Add the key */
dictEntry *de = dictAddRaw(d, key, NULL);
assert(de != NULL);
}
end_benchmark("Find() and inserting via dictFind()+dictAddRaw() non existing");
dictEmpty(d, NULL);
start_benchmark();
for (j = 0; j < count; j++) {
/* Create a key */
void *key = stringFromLongLong(j);
uint64_t hash = dictGetHash(d, key);
/* Check if the key exists */
dictEntry *entry = dictFindByHash(d, key, hash);
assert(entry == NULL);
de = dictAddNonExistsByHash(d, key, hash);
assert(de != NULL);
}
end_benchmark("Find() and inserting via dictGetHash()+dictFindByHash()+dictAddNonExistsByHash() non existing");
assert((long)dictSize(d) == count);
/* Wait for rehashing. */
while (dictIsRehashing(d)) {
dictRehashMicroseconds(d,100*1000);
@ -2275,6 +2271,53 @@ int dictTest(int argc, char **argv, int flags) {
zfree(lookupKeys);
}
TEST("Test dictFindLink() functionality") {
dictType dt = BenchmarkDictType;
dict *d = dictCreate(&dt);
/* find in empty dict */
dictEntryLink link = dictFindLink(d, "key", NULL);
assert(link == NULL);
/* Add keys to dict and test */
for (j = 0; j < 10; j++) {
/* Add another key to dict */
char *key = stringFromLongLong(j);
retval = dictAdd(d, key, (void*)j);
assert(retval == DICT_OK);
/* find existing keys with dictFindLink() */
dictEntryLink link = dictFindLink(d, key, NULL);
assert(link != NULL);
assert(*link != NULL);
assert(dictGetKey(*link) != NULL);
/* Test that the key found is the correct one */
void *foundKey = dictGetKey(*link);
assert(compareCallback( NULL, foundKey, key));
/* Test finding a non-existing key */
char *nonExistingKey = stringFromLongLong(j + 10);
link = dictFindLink(d, nonExistingKey, NULL);
assert(link == NULL);
/* Test with bucket parameter */
dictEntryLink bucket = NULL;
link = dictFindLink(d, key, &bucket);
assert(link != NULL);
assert(bucket != NULL);
/* Test bucket parameter with non-existing key */
link = dictFindLink(d, nonExistingKey, &bucket);
assert(link == NULL);
assert(bucket != NULL); /* Bucket should still be set even for non-existing keys */
/* Clean up */
zfree(nonExistingKey);
}
dictRelease(d);
}
return 0;
}
#endif

76
src/dict.h
View File

@ -29,15 +29,33 @@
typedef struct dictEntry dictEntry; /* opaque */
typedef struct dict dict;
typedef size_t (*keyLenFunc)(dict *d, const void *key1);
typedef int (*keyCmpFuncWithLen)(dict *d, const void *key1, const size_t key1_len, const void *key2, const size_t key2_len);
typedef dictEntry **dictEntryLink; /* See description of dictFindLink() */
/* Searching for a key in a dict may involve few comparisons.
* If extracting the looked-up key is expensive (e.g., sdslen(), kvobjGetKey()),
* caching can be used to reduce those repetitive computations.
*
* This struct, passed to the comparison function as temporary caching, if
* needed by the function across comparison of a given lookup.
* for the looked-up key and resets before each new lookup. */
typedef struct dictCmpCache {
int useCache;
union {
uint64_t u64;
int64_t i64;
int i;
size_t sz;
void *p;
} data[2];
} dictCmpCache;
typedef struct dictType {
/* Callbacks */
uint64_t (*hashFunction)(const void *key);
void *(*keyDup)(dict *d, const void *key);
void *(*valDup)(dict *d, const void *obj);
int (*keyCompare)(dict *d, const void *key1, const void *key2);
int (*keyCompare)(dictCmpCache *cache, const void *key1, const void *key2);
void (*keyDestructor)(dict *d, void *key);
void (*valDestructor)(dict *d, void *obj);
int (*resizeAllowed)(size_t moreMem, double usedRatio);
@ -66,8 +84,6 @@ typedef struct dictType {
/* This flag is required for `no_value` optimization since the optimization
* reuses LSB bits as metadata */
unsigned int keys_are_odd:1;
/* TODO: Add a 'keys_are_even' flag and use a similar optimization if that
* flag is set. */
/* Ensures that the entire hash table is rehashed at once if set. */
unsigned int force_full_rehash:1;
@ -94,14 +110,10 @@ typedef struct dictType {
*
* Set to NULL both functions, if you don't want to support this feature. */
uint64_t (*storedHashFunction)(const void *key);
int (*storedKeyCompare)(dict *d, const void *key1, const void *key2);
int (*storedKeyCompare)(dictCmpCache *cache, const void *key1, const void *key2);
/* Optional callback called when the dict is destroyed. */
void (*onDictRelease)(dict *d);
/* Optional keylen to avoid duplication computation of key lengths. */
keyLenFunc keyLen;
keyCmpFuncWithLen keyCompareWithLen;
} dictType;
#define DICTHT_SIZE(exp) ((exp) == -1 ? 0 : (unsigned long)1<<(exp))
@ -147,7 +159,7 @@ typedef struct dictStats {
unsigned long *clvector;
} dictStats;
typedef void (dictScanFunction)(void *privdata, const dictEntry *de);
typedef void (dictScanFunction)(void *privdata, const dictEntry *de, dictEntry **plink);
typedef void *(dictDefragAllocFunction)(void *ptr);
typedef struct {
dictDefragAllocFunction *defragAlloc; /* Used for entries etc. */
@ -210,40 +222,20 @@ int dictTryExpand(dict *d, unsigned long size);
int dictShrink(dict *d, unsigned long size);
int dictAdd(dict *d, void *key, void *val);
dictEntry *dictAddRaw(dict *d, void *key, dictEntry **existing);
dictEntry *dictAddNonExistsByHash(dict *d, void *key, const uint64_t hash);
void *dictFindPositionForInsert(dict *d, const void *key, dictEntry **existing);
dictEntry *dictInsertAtPosition(dict *d, void *key, void *position);
dictEntry *dictAddOrFind(dict *d, void *key);
int dictReplace(dict *d, void *key, void *val);
int dictDelete(dict *d, const void *key);
dictEntry *dictUnlink(dict *d, const void *key);
void dictFreeUnlinkedEntry(dict *d, dictEntry *he);
dictEntry *dictTwoPhaseUnlinkFind(dict *d, const void *key, dictEntry ***plink, int *table_index);
void dictTwoPhaseUnlinkFree(dict *d, dictEntry *he, dictEntry **plink, int table_index);
dictEntryLink dictTwoPhaseUnlinkFind(dict *d, const void *key, int *table_index);
void dictTwoPhaseUnlinkFree(dict *d, dictEntryLink llink, int table_index);
void dictRelease(dict *d);
dictEntry * dictFind(dict *d, const void *key);
dictEntry *dictFindByHash(dict *d, const void *key, const uint64_t hash);
dictEntry *dictFindByHashAndPtr(dict *d, const void *oldptr, const uint64_t hash);
void *dictFetchValue(dict *d, const void *key);
int dictShrinkIfNeeded(dict *d);
int dictExpandIfNeeded(dict *d);
void dictSetKey(dict *d, dictEntry* de, void *key);
void dictSetVal(dict *d, dictEntry *de, void *val);
void dictSetSignedIntegerVal(dictEntry *de, int64_t val);
void dictSetUnsignedIntegerVal(dictEntry *de, uint64_t val);
void dictSetDoubleVal(dictEntry *de, double val);
int64_t dictIncrSignedIntegerVal(dictEntry *de, int64_t val);
uint64_t dictIncrUnsignedIntegerVal(dictEntry *de, uint64_t val);
double dictIncrDoubleVal(dictEntry *de, double val);
void *dictEntryMetadata(dictEntry *de);
void *dictGetKey(const dictEntry *de);
void *dictGetVal(const dictEntry *de);
int64_t dictGetSignedIntegerVal(const dictEntry *de);
uint64_t dictGetUnsignedIntegerVal(const dictEntry *de);
double dictGetDoubleVal(const dictEntry *de);
double *dictGetDoubleValPtr(dictEntry *de);
size_t dictMemUsage(const dict *d);
size_t dictEntryMemUsage(void);
size_t dictEntryMemUsage(int noValueDict);
dictIterator *dictGetIterator(dict *d);
dictIterator *dictGetSafeIterator(dict *d);
void dictInitIterator(dictIterator *iter, dict *d);
@ -262,7 +254,6 @@ void dictSetResizeEnabled(dictResizeEnable enable);
int dictRehash(dict *d, int n);
int dictRehashMicroseconds(dict *d, uint64_t us);
void dictSetHashFunctionSeed(uint8_t *seed);
uint8_t *dictGetHashFunctionSeed(void);
unsigned long dictScan(dict *d, unsigned long v, dictScanFunction *fn, void *privdata);
unsigned long dictScanDefrag(dict *d, unsigned long v, dictScanFunction *fn, dictDefragFunctions *defragfns, void *privdata);
uint64_t dictGetHash(dict *d, const void *key);
@ -273,6 +264,21 @@ dictStats* dictGetStatsHt(dict *d, int htidx, int full);
void dictCombineStats(dictStats *from, dictStats *into);
void dictFreeStats(dictStats *stats);
dictEntryLink dictFindLink(dict *d, const void *key, dictEntryLink *bucket);
void dictSetKeyAtLink(dict *d, void *key, dictEntryLink *link, int newItem);
/* API relevant only when dict is used as a hash-map (no_value=0) */
void dictSetKey(dict *d, dictEntry* de, void *key);
void dictSetVal(dict *d, dictEntry *de, void *val);
void *dictGetVal(const dictEntry *de);
void dictSetDoubleVal(dictEntry *de, double val);
double dictGetDoubleVal(const dictEntry *de);
double *dictGetDoubleValPtr(dictEntry *de);
void *dictFetchValue(dict *d, const void *key);
void dictSetUnsignedIntegerVal(dictEntry *de, uint64_t val);
uint64_t dictIncrUnsignedIntegerVal(dictEntry *de, uint64_t val);
uint64_t dictGetUnsignedIntegerVal(const dictEntry *de);
#define dictForEach(d, ty, m, ...) do { \
dictIterator *di = dictGetIterator(d); \
dictEntry *de; \

32
src/evict.c
View File

@ -131,27 +131,16 @@ int evictionPoolPopulate(redisDb *db, kvstore *samplekvs, struct evictionPoolEnt
count = kvstoreDictGetSomeKeys(samplekvs,slot,samples,server.maxmemory_samples);
for (j = 0; j < count; j++) {
unsigned long long idle;
sds key;
robj *o;
dictEntry *de;
de = samples[j];
key = dictGetKey(de);
/* If the dictionary we are sampling from is not the main
* dictionary (but the expires one) we need to lookup the key
* again in the key dictionary to obtain the value object. */
if (server.maxmemory_policy != MAXMEMORY_VOLATILE_TTL) {
if (samplekvs != db->keys)
de = kvstoreDictFind(db->keys, slot, key);
o = dictGetVal(de);
}
dictEntry *de = samples[j];
kvobj *kv = dictGetKV(de);
sds key = kvobjGetKey(kv);
/* Calculate the idle time according to the policy. This is called
* idle just because the code initially handled LRU, but is in fact
* just a score where a higher score means better candidate. */
if (server.maxmemory_policy & MAXMEMORY_FLAG_LRU) {
idle = estimateObjectIdleTime(o);
idle = estimateObjectIdleTime(kv);
} else if (server.maxmemory_policy & MAXMEMORY_FLAG_LFU) {
/* When we use an LRU policy, we sort the keys by idle time
* so that we expire keys starting from greater idle time.
@ -160,10 +149,10 @@ int evictionPoolPopulate(redisDb *db, kvstore *samplekvs, struct evictionPoolEnt
* first. So inside the pool we put objects using the inverted
* frequency subtracting the actual frequency to the maximum
* frequency of 255. */
idle = 255-LFUDecrAndReturn(o);
idle = 255-LFUDecrAndReturn(kv);
} else if (server.maxmemory_policy == MAXMEMORY_VOLATILE_TTL) {
/* In this case the sooner the expire the better. */
idle = ULLONG_MAX - dictGetSignedIntegerVal(de);
idle = ULLONG_MAX - kvobjGetExpire(kv);
} else {
serverPanic("Unknown eviction policy in evictionPoolPopulate()");
}
@ -622,7 +611,7 @@ int performEvictions(void) {
/* If the key exists, is our pick. Otherwise it is
* a ghost and we need to try the next element. */
if (de) {
bestkey = dictGetKey(de);
bestkey = kvobjGetKey(dictGetKV(de));
break;
} else {
/* Ghost... Iterate again. */
@ -650,7 +639,8 @@ int performEvictions(void) {
int slot = kvstoreGetFairRandomDictIndex(kvs);
de = kvstoreDictGetRandomKey(kvs, slot);
if (de) {
bestkey = dictGetKey(de);
kvobj *kv = dictGetKV(de);
bestkey = kvobjGetKey(kv);
bestdbid = j;
break;
}

44
src/expire.c
View File

@ -25,8 +25,8 @@
static double avg_ttl_factor[16] = {0.98, 0.9604, 0.941192, 0.922368, 0.903921, 0.885842, 0.868126, 0.850763, 0.833748, 0.817073, 0.800731, 0.784717, 0.769022, 0.753642, 0.738569, 0.723798};
/* Helper function for the activeExpireCycle() function.
* This function will try to expire the key that is stored in the hash table
* entry 'de' of the 'expires' hash table of a Redis database.
* This function will try to expire the key-value entry that is stored in the
* hash table entry 'de' of the 'expires' hash table of a Redis database.
*
* If the key is found to be expired, it is removed from the database and
* 1 is returned. Otherwise no operation is performed and 0 is returned.
@ -35,13 +35,12 @@ static double avg_ttl_factor[16] = {0.98, 0.9604, 0.941192, 0.922368, 0.903921,
*
* The parameter 'now' is the current time in milliseconds as is passed
* to the function to avoid too many gettimeofday() syscalls. */
int activeExpireCycleTryExpire(redisDb *db, dictEntry *de, long long now) {
long long t = dictGetSignedIntegerVal(de);
if (now < t)
int activeExpireCycleTryExpire(redisDb *db, kvobj *kv, long long now) {
if (now < kvobjGetExpire(kv))
return 0;
enterExecutionUnit(1, 0);
sds key = dictGetKey(de);
sds key = kvobjGetKey(kv);
robj *keyobj = createStringObject(key,sdslen(key));
deleteExpiredKeyAndPropagate(db,keyobj);
decrRefCount(keyobj);
@ -109,11 +108,12 @@ typedef struct {
int ttl_samples; /* num keys with ttl not yet expired */
} expireScanData;
void expireScanCallback(void *privdata, const dictEntry *const_de) {
dictEntry *de = (dictEntry *)const_de;
void expireScanCallback(void *privdata, const dictEntry *de, dictEntryLink plink) {
UNUSED(plink);
kvobj *kv = dictGetKV(de);
expireScanData *data = privdata;
long long ttl = dictGetSignedIntegerVal(de) - data->now;
if (activeExpireCycleTryExpire(data->db, de, data->now)) {
long long ttl = kvobjGetExpire(kv) - data->now;
if (activeExpireCycleTryExpire(data->db, kv, data->now)) {
data->expired++;
}
if (ttl > 0) {
@ -464,14 +464,14 @@ void expireSlaveKeys(void) {
while(dbids && dbid < server.dbnum) {
if ((dbids & 1) != 0) {
redisDb *db = server.db+dbid;
dictEntry *expire = dbFindExpires(db, keyname);
int expired = expire && activeExpireCycleTryExpire(server.db+dbid,expire,start);
kvobj *kv = dbFindExpires(db, keyname);
int expired = kv && activeExpireCycleTryExpire(server.db+dbid, kv, start);
/* If the key was not expired in this DB, we need to set the
* corresponding bit in the new bitmap we set as value.
* At the end of the loop if the bitmap is zero, it means we
* no longer need to keep track of this key. */
if (expire && !expired) {
if (kv && !expired) {
noexpire++;
new_dbids |= (uint64_t)1 << dbid;
}
@ -499,7 +499,7 @@ void expireSlaveKeys(void) {
/* Track keys that received an EXPIRE or similar command in the context
* of a writable slave. */
void rememberSlaveKeyWithExpire(redisDb *db, robj *key) {
void rememberSlaveKeyWithExpire(redisDb *db, sds key) {
if (slaveKeysWithExpire == NULL) {
static dictType dt = {
dictSdsHash, /* hash function */
@ -514,13 +514,13 @@ void rememberSlaveKeyWithExpire(redisDb *db, robj *key) {
}
if (db->id > 63) return;
dictEntry *de = dictAddOrFind(slaveKeysWithExpire,key->ptr);
dictEntry *de = dictAddOrFind(slaveKeysWithExpire, key);
/* If the entry was just created, set it to a copy of the SDS string
* representing the key: we don't want to need to take those keys
* in sync with the main DB. The keys will be removed by expireSlaveKeys()
* as it scans to find keys to remove. */
if (dictGetKey(de) == key->ptr) {
dictSetKey(slaveKeysWithExpire, de, sdsdup(key->ptr));
if (dictGetKey(de) == key) {
dictSetKey(slaveKeysWithExpire, de, sdsdup(key));
dictSetUnsignedIntegerVal(de,0);
}
@ -654,13 +654,14 @@ void expireGenericCommand(client *c, long long basetime, int unit) {
when += basetime;
/* No key, return zero. */
if (lookupKeyWrite(c->db,key) == NULL) {
kvobj *kv = lookupKeyWrite(c->db,key);
if (kv == NULL) {
addReply(c,shared.czero);
return;
}
if (flag) {
current_expire = getExpire(c->db, key);
current_expire = kvobjGetExpire(kv);
/* NX option is set, check current expiry */
if (flag & EXPIRE_NX) {
@ -765,14 +766,15 @@ void ttlGenericCommand(client *c, int output_ms, int output_abs) {
long long expire, ttl = -1;
/* If the key does not exist at all, return -2 */
if (lookupKeyReadWithFlags(c->db,c->argv[1],LOOKUP_NOTOUCH) == NULL) {
kvobj *kv = lookupKeyReadWithFlags(c->db,c->argv[1],LOOKUP_NOTOUCH);
if (kv == NULL) {
addReplyLongLong(c,-2);
return;
}
/* The key exists. Return -1 if it has no expire, or the actual
* TTL value otherwise. */
expire = getExpire(c->db,c->argv[1]);
expire = kvobjGetExpire(kv);
if (expire != -1) {
ttl = output_abs ? expire : expire-commandTimeSnapshot();
if (ttl < 0) ttl = 0;

9
src/geo.c
View File

@ -525,7 +525,7 @@ void georadiusGeneric(client *c, int srcKeyIndex, int flags) {
int storedist = 0; /* 0 for STORE, 1 for STOREDIST. */
/* Look up the requested zset */
robj *zobj = lookupKeyRead(c->db, c->argv[srcKeyIndex]);
kvobj *zobj = lookupKeyRead(c->db, c->argv[srcKeyIndex]);
if (checkType(c, zobj, OBJ_ZSET)) return;
/* Find long/lat to use for radius or box search based on inquiry type */
@ -828,8 +828,7 @@ void georadiusGeneric(client *c, int srcKeyIndex, int flags) {
if (returned_items) {
zsetConvertToListpackIfNeeded(zobj,maxelelen,totelelen);
setKey(c,c->db,storekey,zobj,0);
decrRefCount(zobj);
setKey(c,c->db,storekey,&zobj,0);
notifyKeyspaceEvent(NOTIFY_ZSET,flags & GEOSEARCH ? "geosearchstore" : "georadiusstore",storekey,
c->db->id);
server.dirty += returned_items;
@ -880,7 +879,7 @@ void geohashCommand(client *c) {
int j;
/* Look up the requested zset */
robj *zobj = lookupKeyRead(c->db, c->argv[1]);
kvobj *zobj = lookupKeyRead(c->db, c->argv[1]);
if (checkType(c, zobj, OBJ_ZSET)) return;
/* Geohash elements one after the other, using a null bulk reply for
@ -983,7 +982,7 @@ void geodistCommand(client *c) {
}
/* Look up the requested zset */
robj *zobj = NULL;
kvobj *zobj = NULL;
if ((zobj = lookupKeyReadOrReply(c, c->argv[1], shared.null[c->resp]))
== NULL || checkType(c, zobj, OBJ_ZSET)) return;

62
src/hyperloglog.c
View File

@ -1414,29 +1414,29 @@ invalid:
/* PFADD var ele ele ele ... ele => :0 or :1 */
void pfaddCommand(client *c) {
robj *o = lookupKeyWrite(c->db,c->argv[1]);
uint64_t oldlen;
dictEntryLink link;
kvobj *kv = lookupKeyWriteWithLink(c->db,c->argv[1], &link);
struct hllhdr *hdr;
int updated = 0, j;
if (o == NULL) {
if (kv == NULL) {
/* Create the key with a string value of the exact length to
* hold our HLL data structure. sdsnewlen() when NULL is passed
* is guaranteed to return bytes initialized to zero. */
o = createHLLObject();
dbAdd(c->db,c->argv[1],o);
robj *o = createHLLObject();
kv = dbAddByLink(c->db, c->argv[1], &o, &link);
updated++;
} else {
if (isHLLObjectOrReply(c,o) != C_OK) return;
o = dbUnshareStringValue(c->db,c->argv[1],o);
if (isHLLObjectOrReply(c,kv) != C_OK) return;
kv = dbUnshareStringValue(c->db,c->argv[1],kv);
}
/* HLL might change from sparse to dense. No way to predict KEYSIZES diff.
* Update as if the key is being removed. After the for-loop update it back */
int64_t oldlen = stringObjectLen(o);
oldlen = stringObjectLen(kv);
/* Perform the low level ADD operation for every element. */
for (j = 2; j < c->argc; j++) {
int retval = hllAdd(o, (unsigned char*)c->argv[j]->ptr,
int retval = hllAdd(kv, (unsigned char*)c->argv[j]->ptr,
sdslen(c->argv[j]->ptr));
switch(retval) {
case 1:
@ -1448,8 +1448,8 @@ void pfaddCommand(client *c) {
}
}
updateKeysizesHist(c->db, getKeySlot(c->argv[1]->ptr), OBJ_STRING, oldlen, stringObjectLen(o));
hdr = o->ptr;
hdr = kv->ptr;
updateKeysizesHist(c->db, getKeySlot(c->argv[1]->ptr), OBJ_STRING, oldlen, stringObjectLen(kv));
if (updated) {
HLL_INVALIDATE_CACHE(hdr);
signalModifiedKey(c,c->db,c->argv[1]);
@ -1461,7 +1461,6 @@ void pfaddCommand(client *c) {
/* PFCOUNT var -> approximated cardinality of set. */
void pfcountCommand(client *c) {
robj *o;
struct hllhdr *hdr;
uint64_t card;
@ -1480,7 +1479,7 @@ void pfcountCommand(client *c) {
registers = max + HLL_HDR_SIZE;
for (j = 1; j < c->argc; j++) {
/* Check type and size. */
robj *o = lookupKeyRead(c->db,c->argv[j]);
kvobj *o = lookupKeyRead(c->db,c->argv[j]);
if (o == NULL) continue; /* Assume empty HLL for non existing var.*/
if (isHLLObjectOrReply(c,o) != C_OK) return;
@ -1508,7 +1507,7 @@ void pfcountCommand(client *c) {
* logically expired key on a replica is deleted, while with lookupKeyRead
* it isn't, but the lookup returns NULL either way if the key is logically
* expired, which is what matters here. */
o = lookupKeyRead(c->db,c->argv[1]);
kvobj *o = lookupKeyRead(c->db, c->argv[1]);
if (o == NULL) {
/* No key? Cardinality is zero since no element was added, otherwise
* we would have a key as HLLADD creates it as a side effect. */
@ -1568,9 +1567,9 @@ void pfmergeCommand(client *c) {
memset(max,0,sizeof(max));
for (j = 1; j < c->argc; j++) {
/* Check type and size. */
robj *o = lookupKeyRead(c->db,c->argv[j]);
kvobj *o = lookupKeyRead(c->db, c->argv[j]);
if (o == NULL) continue; /* Assume empty HLL for non existing var. */
if (isHLLObjectOrReply(c,o) != C_OK) return;
if (isHLLObjectOrReply(c, o) != C_OK) return;
/* If at least one involved HLL is dense, use the dense representation
* as target ASAP to save time and avoid the conversion step. */
@ -1586,25 +1585,26 @@ void pfmergeCommand(client *c) {
}
/* Create / unshare the destination key's value if needed. */
robj *o = lookupKeyWrite(c->db,c->argv[1]);
if (o == NULL) {
dictEntryLink link;
kvobj *kv = lookupKeyWriteWithLink(c->db,c->argv[1],&link);
if (kv == NULL) {
/* Create the key with a string value of the exact length to
* hold our HLL data structure. sdsnewlen() when NULL is passed
* is guaranteed to return bytes initialized to zero. */
o = createHLLObject();
dbAdd(c->db,c->argv[1],o);
robj *o = createHLLObject();
kv = dbAddByLink(c->db, c->argv[1], &o, &link);
} else {
/* If key exists we are sure it's of the right type/size
* since we checked when merging the different HLLs, so we
* don't check again. */
o = dbUnshareStringValue(c->db,c->argv[1],o);
kv = dbUnshareStringValue(c->db,c->argv[1],kv);
}
uint64_t oldLen = stringObjectLen(o);
uint64_t oldLen = stringObjectLen(kv);
/* Convert the destination object to dense representation if at least
* one of the inputs was dense. */
if (use_dense && hllSparseToDense(o) == C_ERR) {
if (use_dense && hllSparseToDense(kv) == C_ERR) {
addReplyError(c,invalid_hll_err);
return;
}
@ -1612,19 +1612,19 @@ void pfmergeCommand(client *c) {
/* Write the resulting HLL to the destination HLL registers and
* invalidate the cached value. */
if (use_dense) {
hdr = o->ptr;
hdr = kv->ptr;
hllDenseCompress(hdr->registers, max);
} else {
for (j = 0; j < HLL_REGISTERS; j++) {
if (max[j] == 0) continue;
hdr = o->ptr;
hdr = kv->ptr;
switch (hdr->encoding) {
case HLL_DENSE: hllDenseSet(hdr->registers,j,max[j]); break;
case HLL_SPARSE: hllSparseSet(o,j,max[j]); break;
case HLL_SPARSE: hllSparseSet(kv,j,max[j]); break;
}
}
}
hdr = o->ptr; /* o->ptr may be different now, as a side effect of
hdr = kv->ptr; /* o->ptr may be different now, as a side effect of
last hllSparseSet() call. */
HLL_INVALIDATE_CACHE(hdr);
@ -1634,7 +1634,7 @@ void pfmergeCommand(client *c) {
notifyKeyspaceEvent(NOTIFY_STRING,"pfadd",c->argv[1],c->db->id);
updateKeysizesHist(c->db, getKeySlot(c->argv[1]->ptr),
OBJ_STRING, oldLen, stringObjectLen(o));
OBJ_STRING, oldLen, stringObjectLen(kv));
server.dirty++;
addReply(c,shared.ok);
}
@ -1758,7 +1758,7 @@ cleanup:
void pfdebugCommand(client *c) {
char *cmd = c->argv[1]->ptr;
struct hllhdr *hdr;
robj *o;
kvobj *o;
int j;
if (!strcasecmp(cmd, "simd")) {

93
src/kvstore.c
View File

@ -132,7 +132,9 @@ static int getAndClearDictIndexFromCursor(kvstore *kvs, unsigned long long *curs
/* Updates binary index tree (also known as Fenwick tree), increasing key count for a given dict.
* You can read more about this data structure here https://en.wikipedia.org/wiki/Fenwick_tree
* Time complexity is O(log(kvs->num_dicts)). */
* Time complexity is O(log(kvs->num_dicts)). Take care to call it only after
* adding or removing keys from the kvstore.
*/
static void cumulativeKeyCountAdd(kvstore *kvs, int didx, long delta) {
kvs->key_count += delta;
@ -373,7 +375,7 @@ size_t kvstoreMemUsage(kvstore *kvs) {
metaSize = sizeof(kvstoreDictMetaEx);
unsigned long long keys_count = kvstoreSize(kvs);
mem += keys_count * dictEntryMemUsage() +
mem += keys_count * dictEntryMemUsage(kvs->dtype.no_value) +
kvstoreBuckets(kvs) * sizeof(dictEntry*) +
kvs->allocated_dicts * (sizeof(dict) + metaSize);
@ -775,14 +777,6 @@ dictEntry *kvstoreDictGetFairRandomKey(kvstore *kvs, int didx)
return dictGetFairRandomKey(d);
}
dictEntry *kvstoreDictFindByHashAndPtr(kvstore *kvs, int didx, const void *oldptr, uint64_t hash)
{
dict *d = kvstoreGetDict(kvs, didx);
if (!d)
return NULL;
return dictFindByHashAndPtr(d, oldptr, hash);
}
unsigned int kvstoreDictGetSomeKeys(kvstore *kvs, int didx, dictEntry **des, unsigned int count)
{
dict *d = kvstoreGetDict(kvs, didx);
@ -836,16 +830,12 @@ unsigned long kvstoreDictLUTDefrag(kvstore *kvs, unsigned long cursor, kvstoreDi
return 0;
}
uint64_t kvstoreGetHash(kvstore *kvs, const void *key)
{
return kvs->dtype.hashFunction(key);
}
void *kvstoreDictFetchValue(kvstore *kvs, int didx, const void *key)
{
dict *d = kvstoreGetDict(kvs, didx);
if (!d)
return NULL;
assert(d->type->no_value == 0);
return dictFetchValue(d, key);
}
@ -856,9 +846,67 @@ dictEntry *kvstoreDictFind(kvstore *kvs, int didx, void *key) {
return dictFind(d, key);
}
/* Find a link to a key in the specified kvstore. If not found return NULL.
*
* This function is a wrapper around dictFindLink(), used to locate a key in a dict
* from a kvstore.
*
* The caller may provide a bucket pointer to receive the reference to the bucket
* where the key is stored or need to be added.
*
* Returns:
* A reference to the dictEntry if found, otherwise NULL.
*
* Important:
* After calling kvstoreDictFindLink(), any necessary updates based on returned
* link or bucket must be made immediately after, commonly by kvstoreDictSetAtLink()
* without any operations in between that might modify the dict. Otherwise,
* the link or bucket may become invalid. Example usage:
*
* link = kvstoreDictFindLink(kvs, didx, key, &bucket);
* ... Do something, but don't modify kvs->dicts[didx] ...
* if (link)
* kvstoreDictSetAtLink(kvs, didx, kv, &link, 0); // Update existing entry
* else
* kvstoreDictSetAtLink(kvs, didx, kv, &bucket, 1); // Insert new entry
*/
dictEntryLink kvstoreDictFindLink(kvstore *kvs, int didx, void *key, dictEntryLink *bucket) {
if (bucket) *bucket = NULL;
dict *d = kvstoreGetDict(kvs, didx);
if (!d) return NULL;
return dictFindLink(d, key, bucket);
}
/* Set a key (or key-value) in the specified kvstore.
*
* This function inserts a new key or updates an existing one, depending on
* the `newItem` flag.
*
* Parameters:
* link: - When `newItem` is set, `link` points to the bucket of the key.
* - When `newItem` is not set, `link` points to the link of the key.
* - If link is NULL, dictFindLink() will be called to locate the link.
*
* newItem: - If set, add a new key with a new dictEntry.
* - If not set, update the key of an existing dictEntry.
*/
void kvstoreDictSetAtLink(kvstore *kvs, int didx, void *kv, dictEntryLink *link, int newItem) {
dict *d;
if (newItem) {
d = createDictIfNeeded(kvs, didx);
dictSetKeyAtLink(d, kv, link, newItem);
cumulativeKeyCountAdd(kvs, didx, 1); /* must be called only after updating dict */
} else {
d = kvstoreGetDict(kvs, didx);
dictSetKeyAtLink(d, kv, link, newItem);
}
}
dictEntry *kvstoreDictAddRaw(kvstore *kvs, int didx, void *key, dictEntry **existing) {
dict *d = createDictIfNeeded(kvs, didx);
dictUseStoredKeyApi(d, 1);
dictEntry *ret = dictAddRaw(d, key, existing);
dictUseStoredKeyApi(d, 0);
if (ret)
cumulativeKeyCountAdd(kvs, didx, 1);
return ret;
@ -871,19 +919,20 @@ void kvstoreDictSetKey(kvstore *kvs, int didx, dictEntry* de, void *key) {
void kvstoreDictSetVal(kvstore *kvs, int didx, dictEntry *de, void *val) {
dict *d = kvstoreGetDict(kvs, didx);
assert(d->type->no_value == 0);
dictSetVal(d, de, val);
}
dictEntry *kvstoreDictTwoPhaseUnlinkFind(kvstore *kvs, int didx, const void *key, dictEntry ***plink, int *table_index) {
dictEntryLink kvstoreDictTwoPhaseUnlinkFind(kvstore *kvs, int didx, const void *key, int *table_index) {
dict *d = kvstoreGetDict(kvs, didx);
if (!d)
return NULL;
return dictTwoPhaseUnlinkFind(kvstoreGetDict(kvs, didx), key, plink, table_index);
return dictTwoPhaseUnlinkFind(kvstoreGetDict(kvs, didx), key, table_index);
}
void kvstoreDictTwoPhaseUnlinkFree(kvstore *kvs, int didx, dictEntry *he, dictEntry **plink, int table_index) {
void kvstoreDictTwoPhaseUnlinkFree(kvstore *kvs, int didx, dictEntryLink link, int table_index) {
dict *d = kvstoreGetDict(kvs, didx);
dictTwoPhaseUnlinkFree(d, he, plink, table_index);
dictTwoPhaseUnlinkFree(d, link, table_index);
cumulativeKeyCountAdd(kvs, didx, -1);
freeDictIfNeeded(kvs, didx);
}
@ -981,10 +1030,14 @@ int kvstoreTest(int argc, char **argv, int flags) {
kvstoreIterator *kvs_it;
kvstoreDictIterator *kvs_di;
/* Test also dictType with no_value=1 */
dictType KvstoreDictNovalTestType = KvstoreDictTestType;
KvstoreDictNovalTestType.no_value = 1;
int didx = 0;
int curr_slot = 0;
kvstore *kvs1 = kvstoreCreate(&KvstoreDictTestType, 0, KVSTORE_ALLOCATE_DICTS_ON_DEMAND);
kvstore *kvs2 = kvstoreCreate(&KvstoreDictTestType, 0, KVSTORE_ALLOCATE_DICTS_ON_DEMAND | KVSTORE_FREE_EMPTY_DICTS);
kvstore *kvs2 = kvstoreCreate(&KvstoreDictNovalTestType, 0, KVSTORE_ALLOCATE_DICTS_ON_DEMAND | KVSTORE_FREE_EMPTY_DICTS);
TEST("Add 16 keys") {
for (i = 0; i < 16; i++) {

16
src/kvstore.h
View File

@ -63,7 +63,6 @@ int kvstoreGetNextNonEmptyDictIndex(kvstore *kvs, int didx);
int kvstoreNumNonEmptyDicts(kvstore *kvs);
int kvstoreNumAllocatedDicts(kvstore *kvs);
int kvstoreNumDicts(kvstore *kvs);
uint64_t kvstoreGetHash(kvstore *kvs, const void *key);
/* kvstore iterator specific functions */
kvstoreIterator *kvstoreIteratorInit(kvstore *kvs);
@ -87,7 +86,6 @@ void kvstoreReleaseDictIterator(kvstoreDictIterator *kvs_id);
dictEntry *kvstoreDictIteratorNext(kvstoreDictIterator *kvs_di);
dictEntry *kvstoreDictGetRandomKey(kvstore *kvs, int didx);
dictEntry *kvstoreDictGetFairRandomKey(kvstore *kvs, int didx);
dictEntry *kvstoreDictFindByHashAndPtr(kvstore *kvs, int didx, const void *oldptr, uint64_t hash);
unsigned int kvstoreDictGetSomeKeys(kvstore *kvs, int didx, dictEntry **des, unsigned int count);
int kvstoreDictExpand(kvstore *kvs, int didx, unsigned long size);
unsigned long kvstoreDictScanDefrag(kvstore *kvs, int didx, unsigned long v, dictScanFunction *fn, dictDefragFunctions *defragfns, void *privdata);
@ -96,14 +94,20 @@ unsigned long kvstoreDictLUTDefrag(kvstore *kvs, unsigned long cursor, kvstoreDi
void *kvstoreDictFetchValue(kvstore *kvs, int didx, const void *key);
dictEntry *kvstoreDictFind(kvstore *kvs, int didx, void *key);
dictEntry *kvstoreDictAddRaw(kvstore *kvs, int didx, void *key, dictEntry **existing);
void kvstoreDictSetKey(kvstore *kvs, int didx, dictEntry* de, void *key);
void kvstoreDictSetVal(kvstore *kvs, int didx, dictEntry *de, void *val);
dictEntry *kvstoreDictTwoPhaseUnlinkFind(kvstore *kvs, int didx, const void *key, dictEntry ***plink, int *table_index);
void kvstoreDictTwoPhaseUnlinkFree(kvstore *kvs, int didx, dictEntry *he, dictEntry **plink, int table_index);
dictEntryLink kvstoreDictTwoPhaseUnlinkFind(kvstore *kvs, int didx, const void *key, int *table_index);
void kvstoreDictTwoPhaseUnlinkFree(kvstore *kvs, int didx, dictEntryLink plink, int table_index);
int kvstoreDictDelete(kvstore *kvs, int didx, const void *key);
kvstoreDictMetadata *kvstoreGetDictMetadata(kvstore *kvs, int didx);
kvstoreMetadata *kvstoreGetMetadata(kvstore *kvs);
dictEntryLink kvstoreDictFindLink(kvstore *kvs, int didx, void *key, dictEntryLink *bucket);
void kvstoreDictSetAtLink(kvstore *kvs, int didx, void *kv, dictEntryLink *link, int newItem);
/* dict with distinct key & value (no_value=1) currently is used only by pubsub. */
void kvstoreDictSetKey(kvstore *kvs, int didx, dictEntry* de, void *key);
void kvstoreDictSetVal(kvstore *kvs, int didx, dictEntry *de, void *val);
void *kvstoreDictFetchValue(kvstore *kvs, int didx, const void *key);
#ifdef REDIS_TEST
int kvstoreTest(int argc, char *argv[], int flags);
#endif

4
src/latency.c
View File

@ -17,8 +17,8 @@
#include "hdr_histogram.h"
/* Dictionary type for latency events. */
int dictStringKeyCompare(dict *d, const void *key1, const void *key2) {
UNUSED(d);
int dictStringKeyCompare(dictCmpCache *cache, const void *key1, const void *key2) {
UNUSED(cache);
return strcmp(key1,key2) == 0;
}

430
src/module.c
View File

File diff suppressed because it is too large Load Diff

8
src/multi.c
View File

@ -304,7 +304,7 @@ void watchForKey(client *c, robj *key) {
wk->key = key;
wk->client = c;
wk->db = c->db;
wk->expired = keyIsExpired(c->db, key);
wk->expired = keyIsExpired(c->db, key->ptr, NULL);
incrRefCount(key);
listAddNodeTail(c->watched_keys, wk);
watchedKeyLinkToClients(clients, wk);
@ -349,7 +349,7 @@ int isWatchedKeyExpired(client *c) {
while ((ln = listNext(&li))) {
wk = listNodeValue(ln);
if (wk->expired) continue; /* was expired when WATCH was called */
if (keyIsExpired(wk->db, wk->key)) return 1;
if (keyIsExpired(wk->db, wk->key->ptr, NULL)) return 1;
}
return 0;
@ -436,11 +436,11 @@ void touchAllWatchedKeysInDb(redisDb *emptied, redisDb *replaced_with) {
* flag. Deleted keys are not flagged as expired. */
wk->expired = 0;
continue;
} else if (keyIsExpired(replaced_with, key)) {
} else if (keyIsExpired(replaced_with, key->ptr, NULL)) {
/* Expired key remains expired. */
continue;
}
} else if (!exists_in_emptied && keyIsExpired(replaced_with, key)) {
} else if (!exists_in_emptied && keyIsExpired(replaced_with, key->ptr, NULL)) {
/* Non-existing key is replaced with an expired key. */
wk->expired = 1;
continue;

355
src/object.c
View File

@ -2,6 +2,9 @@
*
* Copyright (c) 2009-Present, Redis Ltd.
* All rights reserved.
*
* Copyright (c) 2024-present, Valkey contributors.
* All rights reserved.
*
* Licensed under your choice of (a) the Redis Source Available License 2.0
* (RSALv2); or (b) the Server Side Public License v1 (SSPLv1); or (c) the
@ -18,8 +21,69 @@
#define strtold(a,b) ((long double)strtod((a),(b)))
#endif
/* For objects with large embedded keys, we reserve space for an expire field,
* so if expire is set later, we don't need to reallocate the object. */
#define KEY_SIZE_TO_INCLUDE_EXPIRE_THRESHOLD 128
/* ===================== Creation and parsing of objects ==================== */
/* Creates an object, with embedded key and expire fields. The key and expire
* fields can be omitted by passing NULL and -1, respectively.
*
* Example of kvobj "mykey" WITH expiry (16+8+1+7=32bytes):
*
* +-----------+------------+------------------+------------------------+
* | robj (16) | expiry (8) | key-hdr-size (1) | sdshdr5 "mykey" \0 (7) |
* +-----------+------------+------------------+------------------------+
*/
kvobj *kvobjCreate(int type, const sds key, void *ptr, long long expire) {
/* Determine embedded key and expiration flags */
serverAssert(key != NULL);
int has_expire = ((expire != -1) || (sdslen(key) >= KEY_SIZE_TO_INCLUDE_EXPIRE_THRESHOLD));
/* Calculate embedded key size */
size_t key_sds_len = sdslen(key);
char key_sds_type = sdsReqType(key_sds_len);
size_t key_sds_size = sdsReqSize(key_sds_len, key_sds_type);
/* Compute the base object size */
size_t min_size = sizeof(robj);
if (has_expire) min_size += sizeof(long long);
min_size += 1 + key_sds_size; /* 1 byte for SDS header size */
/* Allocate object memory */
size_t bufsize = 0;
robj *o = zmalloc_usable(min_size, &bufsize);
o->type = type;
o->encoding = OBJ_ENCODING_RAW;
o->ptr = ptr;
o->refcount = 1;
o->lru = 0;
o->iskvobj = 1;
/* If extra space allows, pre-allocate anyway expiration */
if ((!has_expire) && (bufsize >= min_size + sizeof(long long))) {
has_expire = 1;
min_size += sizeof(long long);
}
o->expirable = has_expire;
/* The memory after the struct where we embedded data. */
char *data = (void *)(o + 1);
/* Set the expire field. */
if (o->expirable) {
*(long long *)data = expire;
data += sizeof(long long);
}
/* Store embedded key. */
*data++ = sdsHdrSize(key_sds_type);
sdsnewplacement(data, key_sds_size, key_sds_type, key, key_sds_len);
return o;
}
robj *createObject(int type, void *ptr) {
robj *o = zmalloc(sizeof(*o));
o->type = type;
@ -27,6 +91,8 @@ robj *createObject(int type, void *ptr) {
o->ptr = ptr;
o->refcount = 1;
o->lru = 0;
o->iskvobj = 0;
o->expirable = 0;
return o;
}
@ -66,33 +132,191 @@ robj *createRawStringObject(const char *ptr, size_t len) {
return createObject(OBJ_STRING, sdsnewlen(ptr,len));
}
/* Create a string object with encoding OBJ_ENCODING_EMBSTR, that is
* an object where the sds string is actually an unmodifiable string
* allocated in the same chunk as the object itself. */
robj *createEmbeddedStringObject(const char *ptr, size_t len) {
robj *o = zmalloc(sizeof(robj)+sizeof(struct sdshdr8)+len+1);
struct sdshdr8 *sh = (void*)(o+1);
/* Creates a new embedded string object and copies the content of key, val and
* expire to the new object. LRU is set to 0.
*
* Example of kvobj "mykey" with embedded "myvalue" (16+1+7+11 = 35bytes):
* +-----------+------------------+------------------------+----------------------------+
* | robj (16) | key-hdr-size (1) | sdshdr5 "mykey" \0 (7) | sdshdr8 "myvalue" \0 (11) |
* +-----------+------------------+------------------------+----------------------------+
*/
static kvobj *kvobjCreateEmbedString(const char *val_ptr, size_t val_len,
const sds key, long long expire)
{
serverAssert(key != NULL);
/* Calculate sizes for embedded key */
size_t key_sds_len = sdslen(key);
char key_sds_type = sdsReqType(key_sds_len);
size_t key_sds_size = sdsReqSize(key_sds_len, key_sds_type);
/* Calculate size for embedded value (always SDS_TYPE_8) */
size_t val_sds_size = sdsReqSize(val_len, SDS_TYPE_8);
/* Compute base object size */
size_t min_size = sizeof(robj) + val_sds_size;
if (expire != -1) min_size += sizeof(long long);
min_size += 1 + key_sds_size; /* 1 byte for SDS header size */
/* Allocate object memory */
size_t bufsize = 0;
robj *o = zmalloc_usable(min_size, &bufsize);
o->type = OBJ_STRING;
o->encoding = OBJ_ENCODING_EMBSTR;
o->ptr = sh+1;
o->refcount = 1;
o->lru = 0;
o->expirable = (expire != -1);
o->iskvobj = 1;
sh->len = len;
sh->alloc = len;
sh->flags = SDS_TYPE_8;
if (ptr == SDS_NOINIT)
sh->buf[len] = '\0';
else if (ptr) {
memcpy(sh->buf,ptr,len);
sh->buf[len] = '\0';
} else {
memset(sh->buf,0,len+1);
/* If the allocation has enough space for an expire field, add it even if we
* don't need it now. Then we don't need to realloc if it's needed later. */
if (!o->expirable && bufsize >= min_size + sizeof(long long)) {
o->expirable = 1;
min_size += sizeof(long long);
}
/* The memory after the struct where we embedded data. */
char *data = (char *)(o + 1);
/* Set the expire field. */
if (o->expirable) {
*(long long *)data = expire;
data += sizeof(long long);
}
/* Store embedded key */
*data++ = sdsHdrSize(key_sds_type);
sdsnewplacement(data, key_sds_size, key_sds_type, key, key_sds_len);
data += key_sds_size;
/* Copy embedded value (EMBSTR) always as SDS TYPE 8. Account for unused
* memory in the SDS alloc field. */
size_t remaining_size = bufsize - (data - (char *)(void *)o);
o->ptr = sdsnewplacement(data, remaining_size, SDS_TYPE_8, val_ptr, val_len);
return o;
}
/* Create a string object with encoding OBJ_ENCODING_EMBSTR, that is
* an object where the sds string is actually an unmodifiable string
* allocated in the same chunk as the object itself.
*
* Example of robj with embedded "myvalue" (16+1+11 = 28 bytes):
* +-----------+------------------+----------------------------+
* | robj (16) | key-hdr-size (1) | sdshdr8 "myvalue" \0 (11) |
* +-----------+------------------+----------------------------+
*/
robj *createEmbeddedStringObject(const char *val_ptr, size_t val_len) {
/* Calculate size for embedded value (always SDS_TYPE_8) */
size_t val_sds_size = sdsReqSize(val_len, SDS_TYPE_8);
/* Allocate object memory */
size_t bufsize = 0;
robj *o = zmalloc_usable(sizeof(robj) + val_sds_size, &bufsize);
o->type = OBJ_STRING;
o->encoding = OBJ_ENCODING_EMBSTR;
o->refcount = 1;
o->lru = 0;
o->expirable = 0;
o->iskvobj = 0;
/* The memory after the struct where we embedded data. */
char *data = (char *)(o + 1);
/* Copy embedded value (EMBSTR) always as SDS TYPE 8. Account for unused
* memory in the SDS alloc field. */
size_t remaining_size = bufsize - (data - (char *)(void *)o);
o->ptr = sdsnewplacement(data, remaining_size, SDS_TYPE_8, val_ptr, val_len);
return o;
}
sds kvobjGetKey(const kvobj *kv) {
unsigned char *data = (void *)(kv + 1);
if (kv->expirable) {
/* Skip expire field */
data += sizeof(long long);
}
if (kv->iskvobj) {
uint8_t hdr_size = *(uint8_t *)data;
data += 1 + hdr_size;
return (sds)data;
}
return NULL;
}
long long kvobjGetExpire(const kvobj *kv) {
unsigned char *data = (void *)(kv + 1);
if (kv->expirable) {
return *(long long *)data;
} else {
return -1;
}
}
/* This functions may reallocate the value. The new allocation is returned and
* the old object's reference counter is decremented and possibly freed. Use the
* returned object instead of 'val' after calling this function. */
kvobj *kvobjSetExpire(kvobj *kv, long long expire) {
if (kv->expirable) {
/* Update existing expire field. */
unsigned char *data = (void *)(kv + 1);
*(long long *)data = expire;
return kv;
} else if (expire == -1) {
return kv;
} else {
return kvobjSet(kvobjGetKey(kv), kv, expire);
}
}
/* This functions may reallocate the value. The new allocation is returned and
* the old object's reference counter is decremented and possibly freed. Use the
* returned object instead of 'val' after calling this function. */
kvobj *kvobjSet(sds key, robj *val, long long expire) {
if (val->type == OBJ_STRING && val->encoding == OBJ_ENCODING_EMBSTR) {
kvobj *kv;
size_t len = sdslen(val->ptr);
/* Embed when the sum is up to 64 bytes. */
size_t size = sizeof(kvobj);
size += (key != NULL) * (sdslen(key) + 3); /* hdr size (1) + hdr (1) + nullterm (1) */
size += (expire != -1) * sizeof(long long);
size += 4 + len; /* embstr header (3) + nullterm (1) */
if (size <= CACHE_LINE_SIZE) {
kv = kvobjCreateEmbedString(val->ptr, len, key, expire);
} else {
kv = kvobjCreate(OBJ_STRING, key, sdsnewlen(val->ptr, len), expire);
}
kv->lru = val->lru;
decrRefCount(val);
return kv;
}
/* Create a new object with embedded key. Reuse ptr if possible. */
void *valptr;
if (val->refcount == 1) {
/* Reuse the ptr. There are no other references to val. */
valptr = val->ptr;
val->ptr = NULL;
} else if (val->type == OBJ_STRING && val->encoding == OBJ_ENCODING_INT) {
/* The pointer is not allocated memory. We can just copy the pointer. */
valptr = val->ptr;
} else if (val->type == OBJ_STRING && val->encoding == OBJ_ENCODING_RAW) {
/* Dup the string. */
valptr = sdsdup(val->ptr);
} else {
/* There are multiple references to this non-string object. Most types
* can be duplicated, but for a module type is not always possible. */
serverPanic("Not implemented");
}
robj *new = kvobjCreate(val->type, key, valptr, expire);
new->encoding = val->encoding;
new->lru = val->lru;
decrRefCount(val);
return new;
}
/* Create a string object with EMBSTR encoding if it is smaller than
* OBJ_ENCODING_EMBSTR_SIZE_LIMIT, otherwise the RAW encoding is
* used.
@ -369,15 +593,17 @@ void decrRefCount(robj *o) {
}
if (--(o->refcount) == 0) {
switch(o->type) {
case OBJ_STRING: freeStringObject(o); break;
case OBJ_LIST: freeListObject(o); break;
case OBJ_SET: freeSetObject(o); break;
case OBJ_ZSET: freeZsetObject(o); break;
case OBJ_HASH: freeHashObject(o); break;
case OBJ_MODULE: freeModuleObject(o); break;
case OBJ_STREAM: freeStreamObject(o); break;
default: serverPanic("Unknown object type"); break;
if (o->ptr != NULL) {
switch(o->type) {
case OBJ_STRING: freeStringObject(o); break;
case OBJ_LIST: freeListObject(o); break;
case OBJ_SET: freeSetObject(o); break;
case OBJ_ZSET: freeZsetObject(o); break;
case OBJ_HASH: freeHashObject(o); break;
case OBJ_MODULE: freeModuleObject(o); break;
case OBJ_STREAM: freeStreamObject(o); break;
default: serverPanic("Unknown object type"); break;
}
}
zfree(o);
}
@ -561,13 +787,6 @@ void dismissObject(robj *o, size_t size_hint) {
#endif
}
/* This variant of decrRefCount() gets its argument as void, and is useful
* as free method in data structures that expect a 'void free_object(void*)'
* prototype for the free method. */
void decrRefCountVoid(void *o) {
decrRefCount(o);
}
int checkType(client *c, robj *o, int type) {
/* A NULL is considered an empty key */
if (o && o->type != type) {
@ -636,27 +855,15 @@ robj *tryObjectEncodingEx(robj *o, int try_trim) {
* representable as a 32 nor 64 bit integer. */
len = sdslen(s);
if (len <= 20 && string2l(s,len,&value)) {
/* This object is encodable as a long. Try to use a shared object.
* Note that we avoid using shared integers when maxmemory is used
* because every object needs to have a private LRU field for the LRU
* algorithm to work well. */
if ((server.maxmemory == 0 ||
!(server.maxmemory_policy & MAXMEMORY_FLAG_NO_SHARED_INTEGERS)) &&
value >= 0 &&
value < OBJ_SHARED_INTEGERS)
{
/* This object is encodable as a long. */
if (o->encoding == OBJ_ENCODING_RAW) {
sdsfree(o->ptr);
o->encoding = OBJ_ENCODING_INT;
o->ptr = (void*) value;
return o;
} else if (o->encoding == OBJ_ENCODING_EMBSTR) {
decrRefCount(o);
return shared.integers[value];
} else {
if (o->encoding == OBJ_ENCODING_RAW) {
sdsfree(o->ptr);
o->encoding = OBJ_ENCODING_INT;
o->ptr = (void*) value;
return o;
} else if (o->encoding == OBJ_ENCODING_EMBSTR) {
decrRefCount(o);
return createStringObjectFromLongLongForValue(value);
}
return createStringObjectFromLongLongForValue(value);
}
}
@ -1029,7 +1236,7 @@ size_t objectComputeSize(robj *key, robj *o, size_t sample_size, int dbid) {
asize = sizeof(*o)+sizeof(dict)+(sizeof(struct dictEntry*)*dictBuckets(d));
while((de = dictNext(di)) != NULL && samples < sample_size) {
sds ele = dictGetKey(de);
elesize += dictEntryMemUsage() + sdsZmallocSize(ele);
elesize += dictEntryMemUsage(0) + sdsZmallocSize(ele);
samples++;
}
dictReleaseIterator(di);
@ -1053,7 +1260,7 @@ size_t objectComputeSize(robj *key, robj *o, size_t sample_size, int dbid) {
zmalloc_size(zsl->header);
while(znode != NULL && samples < sample_size) {
elesize += sdsZmallocSize(znode->ele);
elesize += dictEntryMemUsage()+zmalloc_size(znode);
elesize += dictEntryMemUsage(1)+zmalloc_size(znode);
samples++;
znode = znode->level[0].forward;
}
@ -1075,7 +1282,7 @@ size_t objectComputeSize(robj *key, robj *o, size_t sample_size, int dbid) {
hfield ele = dictGetKey(de);
sds ele2 = dictGetVal(de);
elesize += hfieldZmallocSize(ele) + sdsZmallocSize(ele2);
elesize += dictEntryMemUsage();
elesize += dictEntryMemUsage(0);
samples++;
}
dictReleaseIterator(di);
@ -1453,20 +1660,20 @@ int objectSetLRUOrLFU(robj *val, long long lfu_freq, long long lru_idle,
/* This is a helper function for the OBJECT command. We need to lookup keys
* without any modification of LRU or other parameters. */
robj *objectCommandLookup(client *c, robj *key) {
kvobj *kvobjCommandLookup(client *c, robj *key) {
return lookupKeyReadWithFlags(c->db,key,LOOKUP_NOTOUCH|LOOKUP_NONOTIFY);
}
robj *objectCommandLookupOrReply(client *c, robj *key, robj *reply) {
robj *o = objectCommandLookup(c,key);
if (!o) addReplyOrErrorObject(c, reply);
return o;
kvobj *kvobjCommandLookupOrReply(client *c, robj *key, robj *reply) {
kvobj *kv = kvobjCommandLookup(c,key);
if (!kv) addReplyOrErrorObject(c, reply);
return kv;
}
/* Object command allows to inspect the internals of a Redis Object.
* Usage: OBJECT <refcount|encoding|idletime|freq> <key> */
void objectCommand(client *c) {
robj *o;
kvobj *kv;
if (c->argc == 2 && !strcasecmp(c->argv[1]->ptr,"help")) {
const char *help[] = {
@ -1486,23 +1693,23 @@ NULL
};
addReplyHelp(c, help);
} else if (!strcasecmp(c->argv[1]->ptr,"refcount") && c->argc == 3) {
if ((o = objectCommandLookupOrReply(c,c->argv[2],shared.null[c->resp]))
if ((kv = kvobjCommandLookupOrReply(c, c->argv[2], shared.null[c->resp]))
== NULL) return;
addReplyLongLong(c,o->refcount);
addReplyLongLong(c, kv->refcount);
} else if (!strcasecmp(c->argv[1]->ptr,"encoding") && c->argc == 3) {
if ((o = objectCommandLookupOrReply(c,c->argv[2],shared.null[c->resp]))
if ((kv = kvobjCommandLookupOrReply(c, c->argv[2], shared.null[c->resp]))
== NULL) return;
addReplyBulkCString(c,strEncoding(o->encoding));
addReplyBulkCString(c,strEncoding(kv->encoding));
} else if (!strcasecmp(c->argv[1]->ptr,"idletime") && c->argc == 3) {
if ((o = objectCommandLookupOrReply(c,c->argv[2],shared.null[c->resp]))
if ((kv = kvobjCommandLookupOrReply(c, c->argv[2], shared.null[c->resp]))
== NULL) return;
if (server.maxmemory_policy & MAXMEMORY_FLAG_LFU) {
addReplyError(c,"An LFU maxmemory policy is selected, idle time not tracked. Please note that when switching between policies at runtime LRU and LFU data will take some time to adjust.");
return;
}
addReplyLongLong(c,estimateObjectIdleTime(o)/1000);
addReplyLongLong(c, estimateObjectIdleTime(kv) / 1000);
} else if (!strcasecmp(c->argv[1]->ptr,"freq") && c->argc == 3) {
if ((o = objectCommandLookupOrReply(c,c->argv[2],shared.null[c->resp]))
if ((kv = kvobjCommandLookupOrReply(c, c->argv[2], shared.null[c->resp]))
== NULL) return;
if (!(server.maxmemory_policy & MAXMEMORY_FLAG_LFU)) {
addReplyError(c,"An LFU maxmemory policy is not selected, access frequency not tracked. Please note that when switching between policies at runtime LRU and LFU data will take some time to adjust.");
@ -1512,7 +1719,7 @@ NULL
* in case of the key has not been accessed for a long time,
* because we update the access time only
* when the key is read or overwritten. */
addReplyLongLong(c,LFUDecrAndReturn(o));
addReplyLongLong(c,LFUDecrAndReturn(kv));
} else {
addReplySubcommandSyntaxError(c);
}
@ -1540,7 +1747,7 @@ NULL
};
addReplyHelp(c, help);
} else if (!strcasecmp(c->argv[1]->ptr,"usage") && c->argc >= 3) {
dictEntry *de;
kvobj *kv;
long long samples = OBJ_COMPUTE_SIZE_DEF_SAMPLES;
for (int j = 3; j < c->argc; j++) {
if (!strcasecmp(c->argv[j]->ptr,"samples") &&
@ -1559,13 +1766,11 @@ NULL
return;
}
}
if ((de = dbFind(c->db, c->argv[2]->ptr)) == NULL) {
if ((kv = dbFind(c->db, c->argv[2]->ptr)) == NULL) {
addReplyNull(c);
return;
}
size_t usage = objectComputeSize(c->argv[2],dictGetVal(de),samples,c->db->id);
usage += sdsZmallocSize(dictGetKey(de));
usage += dictEntryMemUsage();
size_t usage = objectComputeSize(c->argv[2], (robj *)kv, samples, c->db->id);
addReplyLongLong(c,usage);
} else if (!strcasecmp(c->argv[1]->ptr,"stats") && c->argc == 2) {
struct redisMemOverhead *mh = getMemoryOverheadData();

7
src/pubsub.c
View File

@ -243,8 +243,9 @@ int pubsubSubscribeChannel(client *c, robj *channel, pubsubtype type) {
unsigned int slot = 0;
/* Add the channel to the client -> channels hash table */
void *position = dictFindPositionForInsert(type.clientPubSubChannels(c),channel,NULL);
if (position) { /* Not yet subscribed to this channel */
dictEntryLink bucket;
dictEntryLink link = dictFindLink(type.clientPubSubChannels(c),channel,&bucket);
if (link == NULL) { /* Not yet subscribed to this channel */
retval = 1;
/* Add the client to the channel -> list of clients hash table */
if (server.cluster_enabled && type.shard) {
@ -263,7 +264,7 @@ int pubsubSubscribeChannel(client *c, robj *channel, pubsubtype type) {
}
serverAssert(dictAdd(clients, c, NULL) != DICT_ERR);
serverAssert(dictInsertAtPosition(type.clientPubSubChannels(c), channel, position));
dictSetKeyAtLink(type.clientPubSubChannels(c), channel, &bucket, 1);
incrRefCount(channel);
}
/* Notify the client */

36
src/rdb.c
View File

@ -1412,21 +1412,21 @@ ssize_t rdbSaveDb(rio *rdb, int dbid, int rdbflags, long *key_counter) {
written += res;
last_slot = curr_slot;
}
sds keystr = dictGetKey(de);
robj key, *o = dictGetVal(de);
kvobj *kv = dictGetKV(de);
robj key;
long long expire;
size_t rdb_bytes_before_key = rdb->processed_bytes;
initStaticStringObject(key,keystr);
expire = getExpire(db,&key);
if ((res = rdbSaveKeyValuePair(rdb, &key, o, expire, dbid)) < 0) goto werr;
initStaticStringObject(key,kvobjGetKey(kv));
expire = kvobjGetExpire(kv);
if ((res = rdbSaveKeyValuePair(rdb, &key, kv, expire, dbid)) < 0) goto werr;
written += res;
/* In fork child process, we can try to release memory back to the
* OS and possibly avoid or decrease COW. We give the dismiss
* mechanism a hint about an estimated size of the object we stored. */
size_t dump_size = rdb->processed_bytes - rdb_bytes_before_key;
if (server.in_fork_child) dismissObject(o, dump_size);
if (server.in_fork_child) dismissObject(kv, dump_size);
/* Update child info every 1 second (approximately).
* in order to avoid calling mstime() on each iteration, we will
@ -2300,7 +2300,7 @@ robj *rdbLoadObject(int rdbtype, rio *rdb, sds key, int dbid, int *error)
if (len > server.hash_max_listpack_entries) {
hashTypeConvert(o, OBJ_ENCODING_HT, NULL);
dictTypeAddMeta((dict**)&o->ptr, &mstrHashDictTypeWithHFE);
initDictExpireMetadata(key, o);
initDictExpireMetadata(o);
} else {
hashTypeConvert(o, OBJ_ENCODING_LISTPACK_EX, NULL);
if (deep_integrity_validation) {
@ -2749,7 +2749,6 @@ robj *rdbLoadObject(int rdbtype, rio *rdb, sds key, int dbid, int *error)
(rdbtype == RDB_TYPE_HASH_LISTPACK_EX_PRE_GA) ) {
listpackEx *lpt = listpackExCreate();
lpt->lp = encoded;
lpt->key = key;
o->ptr = lpt;
o->encoding = OBJ_ENCODING_LISTPACK_EX;
} else
@ -3628,15 +3627,16 @@ int rdbLoadRioWithLoadingCtx(rio *rdb, int rdbflags, rdbSaveInfo *rsi, rdbLoadin
initStaticStringObject(keyobj,key);
/* Add the new object in the hash table */
int added = dbAddRDBLoad(db,key,val);
kvobj *kv = dbAddRDBLoad(db, key, &val, expiretime);
server.rdb_last_load_keys_loaded++;
if (!added) {
if (!kv) {
if (rdbflags & RDBFLAGS_ALLOW_DUP) {
/* This flag is useful for DEBUG RELOAD special modes.
* When it's set we allow new keys to replace the current
* keys with the same name. */
dbSyncDelete(db,&keyobj);
dbAddRDBLoad(db,key,val);
kv = dbAddRDBLoad(db, key, &val, expiretime);
serverAssert(kv != NULL);
} else {
serverLog(LL_WARNING,
"RDB has duplicated key '%s' in DB %d",key,db->id);
@ -3646,15 +3646,10 @@ int rdbLoadRioWithLoadingCtx(rio *rdb, int rdbflags, rdbSaveInfo *rsi, rdbLoadin
/* If minExpiredField was set, then the object is hash with expiration
* on fields and need to register it in global HFE DS */
if (val->type == OBJ_HASH) {
uint64_t minExpiredField = hashTypeGetMinExpire(val, 1);
if (kv->type == OBJ_HASH) {
uint64_t minExpiredField = hashTypeGetMinExpire(kv, 1);
if (minExpiredField != EB_EXPIRE_TIME_INVALID)
hashTypeAddToExpires(db, key, val, minExpiredField);
}
/* Set the expire time if needed */
if (expiretime != -1) {
setExpire(NULL,db,&keyobj,expiretime);
hashTypeAddToExpires(db, kv, minExpiredField);
}
/* Set usage information (for eviction). */
@ -3662,6 +3657,9 @@ int rdbLoadRioWithLoadingCtx(rio *rdb, int rdbflags, rdbSaveInfo *rsi, rdbLoadin
/* call key space notification on key loaded for modules only */
moduleNotifyKeyspaceEvent(NOTIFY_LOADED, "loaded", &keyobj, db->id);
/* Release key (sds), dictEntry stores a copy of it in embedded data */
sdsfree(key);
}
/* Loading the database more slowly is useful in order to test

6
src/redis-benchmark.c
View File

@ -183,7 +183,7 @@ int showThroughput(struct aeEventLoop *eventLoop, long long id,
/* Dict callbacks */
static uint64_t dictSdsHash(const void *key);
static int dictSdsKeyCompare(dict *d, const void *key1, const void *key2);
static int dictSdsKeyCompare(dictCmpCache *cache, const void *key1, const void *key2);
/* Implementation */
static long long ustime(void) {
@ -204,10 +204,10 @@ static uint64_t dictSdsHash(const void *key) {
return dictGenHashFunction((unsigned char*)key, sdslen((char*)key));
}
static int dictSdsKeyCompare(dict *d, const void *key1, const void *key2)
static int dictSdsKeyCompare(dictCmpCache *cache, const void *key1, const void *key2)
{
int l1,l2;
UNUSED(d);
UNUSED(cache);
l1 = sdslen((sds)key1);
l2 = sdslen((sds)key2);

6
src/redis-cli.c
View File

@ -163,7 +163,7 @@ static struct termios orig_termios; /* To restore terminal at exit.*/
/* Dict Helpers */
static uint64_t dictSdsHash(const void *key);
static int dictSdsKeyCompare(dict *d, const void *key1,
static int dictSdsKeyCompare(dictCmpCache *cache, const void *key1,
const void *key2);
static void dictSdsDestructor(dict *d, void *val);
static void dictListDestructor(dict *d, void *val);
@ -369,10 +369,10 @@ static uint64_t dictSdsHash(const void *key) {
return dictGenHashFunction((unsigned char*)key, sdslen((char*)key));
}
static int dictSdsKeyCompare(dict *d, const void *key1, const void *key2)
static int dictSdsKeyCompare(dictCmpCache *cache, const void *key1, const void *key2)
{
int l1,l2;
UNUSED(d);
UNUSED(cache);
l1 = sdslen((sds)key1);
l2 = sdslen((sds)key2);

6
src/redisassert.h
View File

@ -26,4 +26,10 @@
void _serverAssert(const char *estr, const char *file, int line);
void _serverPanic(const char *file, int line, const char *msg, ...);
#ifdef DEBUG_ASSERTIONS
#define debugAssert(_e) assert(_e)
#else
#define debugAssert(_e) ((void)0)
#endif
#endif

219
src/sds.c
View File

@ -2,7 +2,10 @@
*
* Copyright (c) 2006-Present, Redis Ltd.
* All rights reserved.
*
*
* Copyright (c) 2024-present, Valkey contributors.
* All rights reserved.
*
* Licensed under your choice of (a) the Redis Source Available License 2.0
* (RSALv2); or (b) the Server Side Public License v1 (SSPLv1); or (c) the
* GNU Affero General Public License v3 (AGPLv3).
@ -20,32 +23,19 @@
const char *SDS_NOINIT = "SDS_NOINIT";
static inline int sdsHdrSize(char type) {
switch(type&SDS_TYPE_MASK) {
case SDS_TYPE_5:
return sizeof(struct sdshdr5);
case SDS_TYPE_8:
return sizeof(struct sdshdr8);
case SDS_TYPE_16:
return sizeof(struct sdshdr16);
case SDS_TYPE_32:
return sizeof(struct sdshdr32);
case SDS_TYPE_64:
return sizeof(struct sdshdr64);
}
return 0;
}
static inline char sdsReqType(size_t string_size) {
if (string_size < 1<<5)
return SDS_TYPE_5;
if (string_size < 1<<8)
return SDS_TYPE_8;
if (string_size < 1<<16)
return SDS_TYPE_16;
/* Returns the minimum SDS type required to store a string of the given length.
*
* Previously, the SDS type was determined solely by the logical string
* length, ignoring header size, which could lead to inconsistencies. If the
* allocated buffer was larger than the maximum size supported by a type, `alloc`
* would be truncated, creating a mismatch between `alloc` and the actual buffer size.
*/
char sdsReqType(size_t string_size) {
if (string_size < 1 << 5) return SDS_TYPE_5;
if (string_size <= (1 << 8) - sizeof(struct sdshdr8) - 1) return SDS_TYPE_8;
if (string_size <= (1 << 16) - sizeof(struct sdshdr16) - 1) return SDS_TYPE_16;
#if (LONG_MAX == LLONG_MAX)
if (string_size < 1ll<<32)
return SDS_TYPE_32;
if (string_size <= (1ll << 32) - sizeof(struct sdshdr32) - 1) return SDS_TYPE_32;
return SDS_TYPE_64;
#else
return SDS_TYPE_32;
@ -66,6 +56,32 @@ static inline size_t sdsTypeMaxSize(char type) {
return -1; /* this is equivalent to the max SDS_TYPE_64 or SDS_TYPE_32 */
}
/*
* Adjusts the SDS type if the allocated buffer size exceeds the maximum size
* addressable by the current type.
*
* The SDS type is initially determined based on the logical length of the string.
* However, allocators like jemalloc may return a buffer larger than requested,
* potentially exceeding the maximum size the selected SDS type can handle. This
* can cause a mismatch between the `alloc` field and the actual buffer size,
* leading to wasted memory and possible inconsistencies.
*
* This function ensures that the SDS type is selected based on the actual buffer
* size rather than just the logical length. If the buffer size supports a larger
* SDS type, it updates `type` and `hdrlen` accordingly.
*
* Returns 1 if the type was adjusted, 0 otherwise.
*/
static inline int adjustTypeIfNeeded(char *type, int *hdrlen, size_t bufsize) {
size_t usable = bufsize - *hdrlen - 1;
if (*type != SDS_TYPE_5 && usable > sdsTypeMaxSize(*type)) {
*type = sdsReqType(usable);
*hdrlen = sdsHdrSize(*type);
return 1;
}
return 0;
}
/* Create a new sds string with the content specified by the 'init' pointer
* and 'initlen'.
* If NULL is used for 'init' the string is initialized with zero bytes.
@ -81,29 +97,45 @@ static inline size_t sdsTypeMaxSize(char type) {
* \0 characters in the middle, as the length is stored in the sds header. */
sds _sdsnewlen(const void *init, size_t initlen, int trymalloc) {
void *sh;
sds s;
char type = sdsReqType(initlen);
/* Empty strings are usually created in order to append. Use type 8
* since type 5 is not good at this. */
if (type == SDS_TYPE_5 && initlen == 0) type = SDS_TYPE_8;
int hdrlen = sdsHdrSize(type);
unsigned char *fp; /* flags pointer. */
size_t usable;
size_t bufsize;
assert(initlen + hdrlen + 1 > initlen); /* Catch size_t overflow */
sh = trymalloc?
s_trymalloc_usable(hdrlen+initlen+1, &usable) :
s_malloc_usable(hdrlen+initlen+1, &usable);
s_trymalloc_usable(hdrlen+initlen+1, &bufsize) :
s_malloc_usable(hdrlen+initlen+1, &bufsize);
if (sh == NULL) return NULL;
if (init==SDS_NOINIT)
init = NULL;
else if (!init)
memset(sh, 0, hdrlen+initlen+1);
s = (char*)sh+hdrlen;
fp = ((unsigned char*)s)-1;
usable = usable-hdrlen-1;
if (usable > sdsTypeMaxSize(type))
usable = sdsTypeMaxSize(type);
adjustTypeIfNeeded(&type, &hdrlen, bufsize);
return sdsnewplacement(sh, bufsize, type, init, initlen);
}
/* Initializes an SDS within pre-allocated buffer. Like, placement new in C++.
*
* Parameters:
* - `buf` : A pre-allocated buffer for the SDS.
* - `bufsize`: Total size of the buffer (>= `sdsReqSize(initlen, type)`). Can use
* a larger `bufsize` than required, but usable size won't be greater
* than `sdsTypeMaxSize(type)`.
* - `type` : The SDS type. Can assist `sdsReqType(length)` to compute the type.
* - `init` : Initial string to copy, or `SDS_NOINIT` to skip initialization.
* - `initlen`: Length of the initial string.
*
* Returns:
* - A pointer to the SDS inside `buf`.
*/
sds sdsnewplacement(char *buf, size_t bufsize, char type, const char *init, size_t initlen) {
assert(bufsize >= sdsReqSize(initlen, type));
int hdrlen = sdsHdrSize(type);
size_t usable = bufsize - hdrlen - 1;
sds s = buf + hdrlen;
unsigned char *fp = ((unsigned char *)s) - 1; /* flags pointer. */
switch(type) {
case SDS_TYPE_5: {
*fp = type | (initlen << SDS_TYPE_BITS);
@ -112,6 +144,7 @@ sds _sdsnewlen(const void *init, size_t initlen, int trymalloc) {
case SDS_TYPE_8: {
SDS_HDR_VAR(8,s);
sh->len = initlen;
debugAssert(usable <= sdsTypeMaxSize(type));
sh->alloc = usable;
*fp = type;
break;
@ -119,6 +152,7 @@ sds _sdsnewlen(const void *init, size_t initlen, int trymalloc) {
case SDS_TYPE_16: {
SDS_HDR_VAR(16,s);
sh->len = initlen;
debugAssert(usable <= sdsTypeMaxSize(type));
sh->alloc = usable;
*fp = type;
break;
@ -126,6 +160,7 @@ sds _sdsnewlen(const void *init, size_t initlen, int trymalloc) {
case SDS_TYPE_32: {
SDS_HDR_VAR(32,s);
sh->len = initlen;
debugAssert(usable <= sdsTypeMaxSize(type));
sh->alloc = usable;
*fp = type;
break;
@ -133,13 +168,19 @@ sds _sdsnewlen(const void *init, size_t initlen, int trymalloc) {
case SDS_TYPE_64: {
SDS_HDR_VAR(64,s);
sh->len = initlen;
debugAssert(usable <= sdsTypeMaxSize(type));
sh->alloc = usable;
*fp = type;
break;
}
}
if (initlen && init)
if (init == SDS_NOINIT)
init = NULL;
else if (!init)
memset(s, 0, initlen);
else if (initlen)
memcpy(s, init, initlen);
s[initlen] = '\0';
return s;
}
@ -224,9 +265,10 @@ sds _sdsMakeRoomFor(sds s, size_t addlen, int greedy) {
void *sh, *newsh;
size_t avail = sdsavail(s);
size_t len, newlen, reqlen;
char type, oldtype = s[-1] & SDS_TYPE_MASK;
char type, oldtype = sdsType(s);
int hdrlen;
size_t usable;
size_t bufsize, usable;
int use_realloc;
/* Return ASAP if there is enough space left. */
if (avail >= addlen) return s;
@ -251,24 +293,31 @@ sds _sdsMakeRoomFor(sds s, size_t addlen, int greedy) {
hdrlen = sdsHdrSize(type);
assert(hdrlen + newlen + 1 > reqlen); /* Catch size_t overflow */
if (oldtype==type) {
newsh = s_realloc_usable(sh, hdrlen+newlen+1, &usable);
use_realloc = (oldtype == type);
if (use_realloc) {
newsh = s_realloc_usable(sh, hdrlen + newlen + 1, &bufsize);
if (newsh == NULL) return NULL;
s = (char*)newsh+hdrlen;
s = (char*)newsh + hdrlen;
if (adjustTypeIfNeeded(&type, &hdrlen, bufsize)) {
memmove((char *)newsh + hdrlen, s, len + 1);
s = (char *)newsh + hdrlen;
s[-1] = type;
sdssetlen(s, len);
}
} else {
/* Since the header size changes, need to move the string forward,
* and can't use realloc */
newsh = s_malloc_usable(hdrlen+newlen+1, &usable);
newsh = s_malloc_usable(hdrlen + newlen + 1, &bufsize);
if (newsh == NULL) return NULL;
adjustTypeIfNeeded(&type, &hdrlen, bufsize);
memcpy((char*)newsh+hdrlen, s, len+1);
s_free(sh);
s = (char*)newsh+hdrlen;
s[-1] = type;
sdssetlen(s, len);
}
usable = usable-hdrlen-1;
if (usable > sdsTypeMaxSize(type))
usable = sdsTypeMaxSize(type);
usable = bufsize - hdrlen - 1;
assert(type == SDS_TYPE_5 || usable <= sdsTypeMaxSize(type));
sdssetalloc(s, usable);
return s;
}
@ -304,8 +353,8 @@ sds sdsRemoveFreeSpace(sds s, int would_regrow) {
* allocation size, this is done in order to avoid repeated calls to this
* function when the caller detects that it has excess space. */
sds sdsResize(sds s, size_t size, int would_regrow) {
void *sh, *newsh;
char type, oldtype = s[-1] & SDS_TYPE_MASK;
void *sh, *newsh = NULL;
char type, oldtype = sdsType(s);
int hdrlen, oldhdrlen = sdsHdrSize(oldtype);
size_t len = sdslen(s);
sh = (char*)s-oldhdrlen;
@ -332,6 +381,8 @@ sds sdsResize(sds s, size_t size, int would_regrow) {
* type. */
int use_realloc = (oldtype==type || (type < oldtype && type > SDS_TYPE_8));
size_t newlen = use_realloc ? oldhdrlen+size+1 : hdrlen+size+1;
size_t bufsize = 0;
size_t newsize;
if (use_realloc) {
int alloc_already_optimal = 0;
@ -340,24 +391,37 @@ sds sdsResize(sds s, size_t size, int would_regrow) {
* We aim to avoid calling realloc() when using Jemalloc if there is no
* change in the allocation size, as it incurs a cost even if the
* allocation size stays the same. */
alloc_already_optimal = (je_nallocx(newlen, 0) == zmalloc_size(sh));
bufsize = sdsAllocSize(s);
alloc_already_optimal = (je_nallocx(newlen, 0) == bufsize);
#endif
if (!alloc_already_optimal) {
newsh = s_realloc(sh, newlen);
newsh = s_realloc_usable(sh, newlen, &bufsize);
if (newsh == NULL) return NULL;
s = (char*)newsh+oldhdrlen;
s = (char *)newsh + oldhdrlen;
if (adjustTypeIfNeeded(&oldtype, &oldhdrlen, bufsize)) {
memmove((char *)newsh + oldhdrlen, s, len + 1);
s = (char *)newsh + oldhdrlen;
s[-1] = oldtype;
sdssetlen(s, len);
}
}
newsize = bufsize - oldhdrlen - 1;
debugAssert(oldtype == SDS_TYPE_5 || newsize <= sdsTypeMaxSize(oldtype));
} else {
newsh = s_malloc(newlen);
newsh = s_malloc_usable(newlen, &bufsize);
if (newsh == NULL) return NULL;
memcpy((char*)newsh+hdrlen, s, len);
adjustTypeIfNeeded(&type, &hdrlen, bufsize);
memcpy((char *)newsh + hdrlen, s, len + 1);
s_free(sh);
s = (char*)newsh+hdrlen;
s = (char *)newsh + hdrlen;
s[-1] = type;
newsize = bufsize - hdrlen - 1;
debugAssert(type == SDS_TYPE_5 || newsize <= sdsTypeMaxSize(type));
}
s[len] = 0;
sdssetlen(s, len);
sdssetalloc(s, size);
sdssetalloc(s, newsize);
return s;
}
@ -403,12 +467,11 @@ void *sdsAllocPtr(sds s) {
* sdsIncrLen(s, nread);
*/
void sdsIncrLen(sds s, ssize_t incr) {
unsigned char flags = s[-1];
size_t len;
switch(flags&SDS_TYPE_MASK) {
switch(sdsType(s)) {
case SDS_TYPE_5: {
unsigned char *fp = ((unsigned char*)s)-1;
unsigned char oldlen = SDS_TYPE_5_LEN(flags);
unsigned char oldlen = SDS_TYPE_5_LEN(s);
assert((incr > 0 && oldlen+incr < 32) || (incr < 0 && oldlen >= (unsigned int)(-incr)));
*fp = SDS_TYPE_5 | ((oldlen+incr) << SDS_TYPE_BITS);
len = oldlen+incr;
@ -1401,7 +1464,7 @@ int sdsTest(int argc, char **argv, int flags) {
for (i = 0; i < 10; i++) {
size_t oldlen = sdslen(x);
x = sdsMakeRoomFor(x,step);
int type = x[-1]&SDS_TYPE_MASK;
int type = sdsType(x);
test_cond("sdsMakeRoomFor() len", sdslen(x) == oldlen);
if (type != SDS_TYPE_5) {
@ -1473,6 +1536,36 @@ int sdsTest(int argc, char **argv, int flags) {
test_cond("sdsresize() crop strlen", strlen(x) == 4);
test_cond("sdsresize() crop alloc", sdsalloc(x) == 4);
sdsfree(x);
{ /* Test adjustTypeIfNeeded() */
/* Test case: Type should be adjusted when buffer size exceeds max size for current type */
char type = SDS_TYPE_8;
int hdrlen = sdsHdrSize(type);
size_t bufsize = (1<<8) + hdrlen + 1; /* Exceeds SDS_TYPE_8 max size */
int result = adjustTypeIfNeeded(&type, &hdrlen, bufsize);
test_cond("adjustTypeIfNeeded() returns 1 when type needs adjustment", result == 1);
test_cond("adjustTypeIfNeeded() adjusts type correctly", type == SDS_TYPE_16);
test_cond("adjustTypeIfNeeded() adjusts header length correctly", hdrlen == sdsHdrSize(SDS_TYPE_16));
/* Test case: Type should not be adjusted when buffer size is within max size for current type */
type = SDS_TYPE_8;
hdrlen = sdsHdrSize(type);
bufsize = (1<<8) - 10 + hdrlen + 1; /* Within SDS_TYPE_8 max size */
result = adjustTypeIfNeeded(&type, &hdrlen, bufsize);
test_cond("adjustTypeIfNeeded() returns 0 when type doesn't need adjustment", result == 0);
test_cond("adjustTypeIfNeeded() doesn't change type when not needed", type == SDS_TYPE_8);
test_cond("adjustTypeIfNeeded() doesn't change header length when not needed", hdrlen == sdsHdrSize(SDS_TYPE_8));
/* Test case 3: Type 5 should never be adjusted */
type = SDS_TYPE_5;
hdrlen = sdsHdrSize(type);
bufsize = 1000; /* Large buffer size */
result = adjustTypeIfNeeded(&type, &hdrlen, bufsize);
test_cond("adjustTypeIfNeeded() returns 0 for SDS_TYPE_5", result == 0);
test_cond("adjustTypeIfNeeded() doesn't change SDS_TYPE_5", type == SDS_TYPE_5);
test_cond("adjustTypeIfNeeded() doesn't change header length for SDS_TYPE_5", hdrlen == sdsHdrSize(SDS_TYPE_5));
}
}
return 0;
}

60
src/sds.h
View File

@ -2,6 +2,9 @@
*
* Copyright (c) 2006-Present, Redis Ltd.
* All rights reserved.
*
* Copyright (c) 2024-present, Valkey contributors.
* All rights reserved.
*
* Licensed under your choice of (a) the Redis Source Available License 2.0
* (RSALv2); or (b) the Server Side Public License v1 (SSPLv1); or (c) the
@ -60,13 +63,16 @@ struct __attribute__ ((__packed__)) sdshdr64 {
#define SDS_TYPE_BITS 3
#define SDS_HDR_VAR(T,s) struct sdshdr##T *sh = (void*)((s)-(sizeof(struct sdshdr##T)));
#define SDS_HDR(T,s) ((struct sdshdr##T *)((s)-(sizeof(struct sdshdr##T))))
#define SDS_TYPE_5_LEN(f) ((f)>>SDS_TYPE_BITS)
#define SDS_TYPE_5_LEN(s) (((unsigned char)(s[-1])) >> SDS_TYPE_BITS)
static inline unsigned char sdsType(const sds s) {
unsigned char flags = s[-1];
return flags & SDS_TYPE_MASK;
}
static inline size_t sdslen(const sds s) {
unsigned char flags = s[-1];
switch(flags&SDS_TYPE_MASK) {
case SDS_TYPE_5:
return SDS_TYPE_5_LEN(flags);
switch (sdsType(s)) {
case SDS_TYPE_5: return SDS_TYPE_5_LEN(s);
case SDS_TYPE_8:
return SDS_HDR(8,s)->len;
case SDS_TYPE_16:
@ -80,8 +86,7 @@ static inline size_t sdslen(const sds s) {
}
static inline size_t sdsavail(const sds s) {
unsigned char flags = s[-1];
switch(flags&SDS_TYPE_MASK) {
switch(sdsType(s)) {
case SDS_TYPE_5: {
return 0;
}
@ -106,8 +111,7 @@ static inline size_t sdsavail(const sds s) {
}
static inline void sdssetlen(sds s, size_t newlen) {
unsigned char flags = s[-1];
switch(flags&SDS_TYPE_MASK) {
switch(sdsType(s)) {
case SDS_TYPE_5:
{
unsigned char *fp = ((unsigned char*)s)-1;
@ -130,12 +134,11 @@ static inline void sdssetlen(sds s, size_t newlen) {
}
static inline void sdsinclen(sds s, size_t inc) {
unsigned char flags = s[-1];
switch(flags&SDS_TYPE_MASK) {
switch(sdsType(s)) {
case SDS_TYPE_5:
{
unsigned char *fp = ((unsigned char*)s)-1;
unsigned char newlen = SDS_TYPE_5_LEN(flags)+inc;
unsigned char newlen = SDS_TYPE_5_LEN(s)+inc;
*fp = SDS_TYPE_5 | (newlen << SDS_TYPE_BITS);
}
break;
@ -156,10 +159,9 @@ static inline void sdsinclen(sds s, size_t inc) {
/* sdsalloc() = sdsavail() + sdslen() */
static inline size_t sdsalloc(const sds s) {
unsigned char flags = s[-1];
switch(flags&SDS_TYPE_MASK) {
switch(sdsType(s)) {
case SDS_TYPE_5:
return SDS_TYPE_5_LEN(flags);
return SDS_TYPE_5_LEN(s);
case SDS_TYPE_8:
return SDS_HDR(8,s)->alloc;
case SDS_TYPE_16:
@ -173,8 +175,7 @@ static inline size_t sdsalloc(const sds s) {
}
static inline void sdssetalloc(sds s, size_t newlen) {
unsigned char flags = s[-1];
switch(flags&SDS_TYPE_MASK) {
switch(sdsType(s)) {
case SDS_TYPE_5:
/* Nothing to do, this type has no total allocation info. */
break;
@ -193,9 +194,27 @@ static inline void sdssetalloc(sds s, size_t newlen) {
}
}
static inline int sdsHdrSize(char type) {
switch(type&SDS_TYPE_MASK) {
case SDS_TYPE_5:
return sizeof(struct sdshdr5);
case SDS_TYPE_8:
return sizeof(struct sdshdr8);
case SDS_TYPE_16:
return sizeof(struct sdshdr16);
case SDS_TYPE_32:
return sizeof(struct sdshdr32);
case SDS_TYPE_64:
return sizeof(struct sdshdr64);
}
return 0;
}
sds sdsnewlen(const void *init, size_t initlen);
sds sdstrynewlen(const void *init, size_t initlen);
sds sdsnew(const char *init);
sds sdsnewplacement(char *buf, size_t bufsize, char type, const char *init, size_t initlen);
sds sdsempty(void);
sds sdsdup(const sds s);
void sdsfree(sds s);
@ -243,6 +262,7 @@ typedef sds (*sdstemplate_callback_t)(const sds variable, void *arg);
sds sdstemplate(const char *template, sdstemplate_callback_t cb_func, void *cb_arg);
/* Low level functions exposed to the user API */
char sdsReqType(size_t string_size);
sds sdsMakeRoomFor(sds s, size_t addlen);
sds sdsMakeRoomForNonGreedy(sds s, size_t addlen);
void sdsIncrLen(sds s, ssize_t incr);
@ -251,6 +271,12 @@ sds sdsResize(sds s, size_t size, int would_regrow);
size_t sdsAllocSize(sds s);
void *sdsAllocPtr(sds s);
/* Returns the minimum required size to store an sds string of the given length
* and type. */
static inline size_t sdsReqSize(size_t len, char type) {
return len + sdsHdrSize(type) + 1;
}
/* Export the allocator used by SDS to the program using SDS.
* Sometimes the program SDS is linked to, may use a different set of
* allocators, but may want to allocate or free things that SDS will

2
src/sentinel.c
View File

@ -4446,7 +4446,7 @@ void sentinelSetCommand(client *c) {
/* If the target name is the same as the source name there
* is no need to add an entry mapping to itself. */
if (!dictSdsKeyCaseCompare(ri->renamed_commands,oldname,newname)) {
if (strcasecmp(oldname, newname) != 0) {
oldname = sdsdup(oldname);
newname = sdsdup(newname);
dictAdd(ri->renamed_commands,oldname,newname);

102
src/server.c
View File

@ -295,19 +295,53 @@ size_t dictSdsKeyLen(dict *d, const void *key) {
return sdslen((sds)key);
}
int dictSdsKeyCompareWithLen(dict *d, const void *key1, const size_t l1,
const void *key2, const size_t l2)
{
UNUSED(d);
static uint64_t dictHashKV(const void *kv) {
sds sdsKey = kvobjGetKey((kvobj *) kv);
return dictGenHashFunction(sdsKey, sdslen(sdsKey));
}
int dictCompareKV(dictCmpCache *cache, const void *kv1, const void *kv2) {
/* Use caching to avoid compute key&len for each comparison on given lookup */
if (cache->useCache == 0) {
cache->useCache = 1;
cache->data[0].p = kvobjGetKey((kvobj *) kv1);
cache->data[1].sz = sdslen((sds) cache->data[0].p);
}
sds key1 = cache->data[0].p;
sds key2 = kvobjGetKey((kvobj *) kv2);
int l1 = (int) cache->data[1].sz;
int l2 = sdslen((sds)key2);
if (l1 != l2) return 0;
return memcmp(key1, key2, l1) == 0;
}
int dictSdsKeyCompare(dict *d, const void *key1,
int dictSdsCompareKV(dictCmpCache *cache, const void *sdsLookup, const void *kv)
{
/* is first cmp call of a new lookup */
if (cache->useCache == 0) {
cache->useCache = 1;
cache->data[0].sz = sdslen((sds) sdsLookup);
}
sds key2 = kvobjGetKey((kvobj *)kv);
size_t l1 = cache->data[0].sz;
size_t l2 = sdslen((sds)key2);
if (l1 != l2) return 0;
return memcmp(sdsLookup, key2, l1) == 0;
}
static void dictDestructorKV(dict *d, void *kv) {
UNUSED(d);
if (kv == NULL) return;
decrRefCount(kv);
}
int dictSdsKeyCompare(dictCmpCache *cache, const void *key1,
const void *key2)
{
int l1,l2;
UNUSED(d);
UNUSED(cache);
l1 = sdslen((sds)key1);
l2 = sdslen((sds)key2);
@ -315,10 +349,10 @@ int dictSdsKeyCompare(dict *d, const void *key1,
return memcmp(key1, key2, l1) == 0;
}
int dictSdsMstrKeyCompare(dict *d, const void *sdsLookup, const void *mstrStored)
int dictSdsMstrKeyCompare(dictCmpCache *cache, const void *sdsLookup, const void *mstrStored)
{
int l1,l2;
UNUSED(d);
UNUSED(cache);
l1 = sdslen((sds)sdsLookup);
l2 = hfieldlen((hfield)mstrStored);
@ -329,10 +363,10 @@ int dictSdsMstrKeyCompare(dict *d, const void *sdsLookup, const void *mstrStored
/* A case insensitive version used for the command lookup table and other
* places where case insensitive non binary-safe comparison is needed. */
int dictSdsKeyCaseCompare(dict *d, const void *key1,
int dictSdsKeyCaseCompare(dictCmpCache *cache, const void *key1,
const void *key2)
{
UNUSED(d);
UNUSED(cache);
return strcasecmp(key1, key2) == 0;
}
@ -354,11 +388,11 @@ void *dictSdsDup(dict *d, const void *key) {
return sdsdup((const sds) key);
}
int dictObjKeyCompare(dict *d, const void *key1,
int dictObjKeyCompare(dictCmpCache *cache, const void *key1,
const void *key2)
{
const robj *o1 = key1, *o2 = key2;
return dictSdsKeyCompare(d, o1->ptr,o2->ptr);
return dictSdsKeyCompare(cache, o1->ptr,o2->ptr);
}
uint64_t dictObjHash(const void *key) {
@ -394,15 +428,15 @@ uint64_t dictClientHash(const void *key) {
}
/* Dict compare function for client */
int dictClientKeyCompare(dict *d, const void *key1, const void *key2) {
UNUSED(d);
int dictClientKeyCompare(dictCmpCache *cache, const void *key1, const void *key2) {
UNUSED(cache);
return ((client *)key1)->id == ((client *)key2)->id;
}
/* Dict compare function for null terminated string */
int dictCStrKeyCompare(dict *d, const void *key1, const void *key2) {
int dictCStrKeyCompare(dictCmpCache *cache, const void *key1, const void *key2) {
int l1,l2;
UNUSED(d);
UNUSED(cache);
l1 = strlen((char*)key1);
l2 = strlen((char*)key2);
@ -411,12 +445,12 @@ int dictCStrKeyCompare(dict *d, const void *key1, const void *key2) {
}
/* Dict case insensitive compare function for null terminated string */
int dictCStrKeyCaseCompare(dict *d, const void *key1, const void *key2) {
UNUSED(d);
int dictCStrKeyCaseCompare(dictCmpCache *cache, const void *key1, const void *key2) {
UNUSED(cache);
return strcasecmp(key1, key2) == 0;
}
int dictEncObjKeyCompare(dict *d, const void *key1, const void *key2)
int dictEncObjKeyCompare(dictCmpCache *cache, const void *key1, const void *key2)
{
robj *o1 = (robj*) key1, *o2 = (robj*) key2;
int cmp;
@ -431,7 +465,7 @@ int dictEncObjKeyCompare(dict *d, const void *key1, const void *key2)
* objects as well. */
if (o1->refcount != OBJ_STATIC_REFCOUNT) o1 = getDecodedObject(o1);
if (o2->refcount != OBJ_STATIC_REFCOUNT) o2 = getDecodedObject(o2);
cmp = dictSdsKeyCompare(d,o1->ptr,o2->ptr);
cmp = dictSdsKeyCompare(cache,o1->ptr,o2->ptr);
if (o1->refcount != OBJ_STATIC_REFCOUNT) decrRefCount(o1);
if (o2->refcount != OBJ_STATIC_REFCOUNT) decrRefCount(o2);
return cmp;
@ -518,17 +552,19 @@ dictType zsetDictType = {
NULL, /* allow to expand */
};
/* Db->dict, keys are sds strings, vals are Redis objects. */
/* Db->dict, keys are of type kvobj, unification of key and value */
dictType dbDictType = {
dictSdsHash, /* hash function */
NULL, /* key dup */
NULL, /* val dup */
dictSdsKeyCompare, /* key compare */
dictSdsDestructor, /* key destructor */
dictObjectDestructor, /* val destructor */
dictResizeAllowed, /* allow to resize */
.keyLen = dictSdsKeyLen, /* key length */
.keyCompareWithLen = dictSdsKeyCompareWithLen /* key compare with length */
dictSdsHash, /* hash function */
NULL, /* key dup */
NULL, /* val dup */
dictSdsCompareKV, /* lookup key compare */
dictDestructorKV, /* key destructor */
NULL, /* val destructor */
dictResizeAllowed, /* allow to resize */
.no_value = 1, /* keys and values are unified (kvobj) */
.keys_are_odd = 0, /* simple kvobj (robj) struct */
.storedHashFunction = dictHashKV, /* stored hash function */
.storedKeyCompare = dictCompareKV, /* stored key compare */
};
/* Db->expires */
@ -536,10 +572,14 @@ dictType dbExpiresDictType = {
dictSdsHash, /* hash function */
NULL, /* key dup */
NULL, /* val dup */
dictSdsKeyCompare, /* key compare */
dictSdsCompareKV, /* key compare */
NULL, /* key destructor */
NULL, /* val destructor */
dictResizeAllowed, /* allow to resize */
.no_value = 1, /* keys and values are unified (kvobj) */
.keys_are_odd = 0, /* simple kvobj (robj) struct */
.storedHashFunction = dictHashKV, /* stored hash function */
.storedKeyCompare = dictCompareKV, /* stored key compare */
};
/* Command table. sds string -> command struct pointer. */

125
src/server.h
View File

@ -66,6 +66,29 @@ typedef long long ustime_t; /* microsecond time type. */
#define REDISMODULE_CORE 1
typedef struct redisObject robj;
/* kvobj - A specific type of robj that holds also embedded key
*
* Since robj is being overused as general purpose object, `kvobj` distincts only
* at the declarative level. This distinction assist to the clarity of the code
* and can optionally enforce explicit casting later on. An `robj` is identified
* to be `kvobj` if `iskvobj` flag is set.
*
* Example to kvobj layout with key "mykey" and expiration time:
* +--------------+--------------+--------------+--------------------+
* | serverObject | Expiry Time | key-hdr-size | sdshdr5 "mykey" \0 |
* | 16 bytes | 8 byte | 1 byte | 1 + 5 + 1 |
* +--------------+--------------+--------------+--------------------+
*
* Example to kvobj layout with key and embedded value "myvalue":
* +--------------+--------------+--------------------+----------------------+
* | serverObject | key-hdr-size | sdshdr5 "mykey" \0 | sdshdr8 "myvalue" \0 |
* | 16 bytes | 1 byte | 1 + 5 + 1 | 3 + 7 + 1 |
* +--------------+--------------+--------------------+----------------------+
*
*/
typedef struct redisObject kvobj;
#include "redismodule.h" /* Redis modules API defines. */
/* Following includes allow test functions to be called from Redis main() */
@ -998,16 +1021,21 @@ struct RedisModuleDigest {
#define LRU_CLOCK_MAX ((1<<LRU_BITS)-1) /* Max value of obj->lru */
#define LRU_CLOCK_RESOLUTION 1000 /* LRU clock resolution in ms */
#define OBJ_SHARED_REFCOUNT INT_MAX /* Global object never destroyed. */
#define OBJ_STATIC_REFCOUNT (INT_MAX-1) /* Object allocated in the stack. */
#define OBJ_REFCOUNT_BITS 30
#define OBJ_SHARED_REFCOUNT ((1 << OBJ_REFCOUNT_BITS) - 1) /* Global object never destroyed. */
#define OBJ_STATIC_REFCOUNT ((1 << OBJ_REFCOUNT_BITS) - 2) /* Object allocated in the stack. */
#define OBJ_FIRST_SPECIAL_REFCOUNT OBJ_STATIC_REFCOUNT
struct redisObject {
unsigned type:4;
unsigned encoding:4;
unsigned lru:LRU_BITS; /* LRU time (relative to global lru_clock) or
* LFU data (least significant 8 bits frequency
* and most significant 16 bits access time). */
int refcount;
unsigned iskvobj : 1; /* 1 if this struct serves as a kvobj base */
unsigned expirable : 1; /* 1 if this key has expiration time attached.
* If set, then this object is of type kvobj */
unsigned refcount : OBJ_REFCOUNT_BITS;
void *ptr;
};
@ -1024,6 +1052,8 @@ char *getObjectTypeName(robj*);
_var.refcount = OBJ_STATIC_REFCOUNT; \
_var.type = OBJ_STRING; \
_var.encoding = OBJ_ENCODING_RAW; \
_var.expirable = 0; \
_var.iskvobj = 0; \
_var.ptr = _ptr; \
} while(0)
@ -2719,7 +2749,7 @@ void moduleUnblockClient(client *c);
int moduleBlockedClientMayTimeout(client *c);
int moduleClientIsBlockedOnKeys(client *c);
void moduleNotifyUserChanged(client *c);
void moduleNotifyKeyUnlink(robj *key, robj *val, int dbid, int flags);
void moduleNotifyKeyUnlink(robj *key, kvobj *kv, int dbid, int flags);
size_t moduleGetFreeEffort(robj *key, robj *val, int dbid);
size_t moduleGetMemUsage(robj *key, robj *val, size_t sample_size, int dbid);
robj *moduleTypeDupOrReply(client *c, robj *fromkey, robj *tokey, int todb, robj *value);
@ -2961,7 +2991,6 @@ void execCommandAbort(client *c, sds error);
/* Redis object implementation */
void decrRefCount(robj *o);
void decrRefCountVoid(void *o);
void incrRefCount(robj *o);
robj *makeObjectShared(robj *o);
void freeStringObject(robj *o);
@ -2974,7 +3003,6 @@ robj *createObject(int type, void *ptr);
void initObjectLRUOrLFU(robj *o);
robj *createStringObject(const char *ptr, size_t len);
robj *createRawStringObject(const char *ptr, size_t len);
robj *createEmbeddedStringObject(const char *ptr, size_t len);
robj *tryCreateRawStringObject(const char *ptr, size_t len);
robj *tryCreateStringObject(const char *ptr, size_t len);
robj *dupStringObject(const robj *o);
@ -3018,6 +3046,12 @@ unsigned long long estimateObjectIdleTime(robj *o);
void trimStringObjectIfNeeded(robj *o, int trim_small_values);
#define sdsEncodedObject(objptr) (objptr->encoding == OBJ_ENCODING_RAW || objptr->encoding == OBJ_ENCODING_EMBSTR)
kvobj *kvobjCreate(int type, const sds key, void *ptr, long long expire);
kvobj *kvobjSet(sds key, robj *val, long long expire);
kvobj *kvobjSetExpire(kvobj *kv, long long expire);
sds kvobjGetKey(const kvobj *kv);
long long kvobjGetExpire(const kvobj *val);
/* Synchronous I/O with timeout */
ssize_t syncWrite(int fd, char *ptr, ssize_t size, long long timeout);
ssize_t syncRead(int fd, char *ptr, ssize_t size, long long timeout);
@ -3345,8 +3379,9 @@ int calculateKeySlot(sds key);
/* kvstore wrappers */
int dbExpand(redisDb *db, uint64_t db_size, int try_expand);
int dbExpandExpires(redisDb *db, uint64_t db_size, int try_expand);
dictEntry *dbFind(redisDb *db, void *key);
dictEntry *dbFindExpires(redisDb *db, void *key);
kvobj *dbFind(redisDb *db, sds key);
kvobj *dbFindByLink(redisDb *db, sds key, dictEntryLink *link);
kvobj *dbFindExpires(redisDb *db, sds key);
unsigned long long dbSize(redisDb *db);
unsigned long long dbScan(redisDb *db, unsigned long long cursor, dictScanFunction *scan_cb, void *privdata);
@ -3376,10 +3411,6 @@ typedef struct listpackEx {
minimum, hash-field to expire. TTL value might be
inaccurate up-to few seconds due to optimization
consideration. */
sds key; /* reference to the key, same one that stored in
db->dict. Will be used from active-expiration flow
for notification and deletion of the object, if
needed. */
void *lp; /* listpack that contains 'key-value-ttl' tuples which
are ordered by ttl. */
} listpackEx;
@ -3394,10 +3425,6 @@ typedef struct dictExpireMetadata {
inaccurate up-to few seconds due to optimization
consideration. */
ebuckets hfe; /* DS of Hash Fields Expiration, associated to each hash */
sds key; /* reference to the key, same one that stored in
db->dict. Will be used from active-expiration flow
for notification and deletion of the object, if
needed. */
} dictExpireMetadata;
/* Hash data type */
@ -3418,7 +3445,7 @@ typedef struct dictExpireMetadata {
void hashTypeConvert(robj *o, int enc, ebuckets *hexpires);
void hashTypeTryConversion(redisDb *db, robj *subject, robj **argv, int start, int end);
int hashTypeExists(redisDb *db, robj *o, sds key, int hfeFlags, int *isHashDeleted);
int hashTypeExists(redisDb *db, kvobj *kv, sds field, int hfeFlags, int *isHashDeleted);
int hashTypeDelete(robj *o, void *key, int isSdsField);
unsigned long hashTypeLength(const robj *o, int subtractExpiredFields);
hashTypeIterator *hashTypeInitIterator(robj *subject);
@ -3435,19 +3462,18 @@ void hashTypeCurrentObject(hashTypeIterator *hi, int what, unsigned char **vstr,
unsigned int *vlen, long long *vll, uint64_t *expireTime);
sds hashTypeCurrentObjectNewSds(hashTypeIterator *hi, int what);
hfield hashTypeCurrentObjectNewHfield(hashTypeIterator *hi);
int hashTypeGetValueObject(redisDb *db, robj *o, sds field, int hfeFlags,
int hashTypeGetValueObject(redisDb *db, kvobj *kv, sds field, int hfeFlags,
robj **val, uint64_t *expireTime, int *isHashDeleted);
int hashTypeSet(redisDb *db, robj *o, sds field, sds value, int flags);
robj *hashTypeDup(robj *o, sds newkey, uint64_t *minHashExpire);
robj *hashTypeDup(kvobj *kv, uint64_t *minHashExpire);
uint64_t hashTypeRemoveFromExpires(ebuckets *hexpires, robj *o);
void hashTypeAddToExpires(redisDb *db, sds key, robj *hashObj, uint64_t expireTime);
void hashTypeAddToExpires(redisDb *db, kvobj *hashObj, uint64_t expireTime);
void hashTypeFree(robj *o);
int hashTypeIsExpired(const robj *o, uint64_t expireAt);
unsigned char *hashTypeListpackGetLp(robj *o);
uint64_t hashTypeGetMinExpire(robj *o, int accurate);
void hashTypeUpdateKeyRef(robj *o, sds newkey);
ebuckets *hashTypeGetDictMetaHFE(dict *d);
void initDictExpireMetadata(sds key, robj *o);
void initDictExpireMetadata(robj *o);
struct listpackEx *listpackExCreate(void);
void listpackExAddNew(robj *o, char *field, size_t flen,
char *value, size_t vlen, uint64_t expireAt);
@ -3561,21 +3587,21 @@ int removeExpire(redisDb *db, robj *key);
void deleteExpiredKeyAndPropagate(redisDb *db, robj *keyobj);
void deleteEvictedKeyAndPropagate(redisDb *db, robj *keyobj, long long *key_mem_freed);
void propagateDeletion(redisDb *db, robj *key, int lazy);
int keyIsExpired(redisDb *db, robj *key);
long long getExpire(redisDb *db, robj *key);
void setExpire(client *c, redisDb *db, robj *key, long long when);
void setExpireWithDictEntry(client *c, redisDb *db, robj *key, long long when, dictEntry *kde);
int keyIsExpired(redisDb *db, sds key, kvobj *kv);
long long getExpire(redisDb *db, sds key, kvobj *kv);
kvobj *setExpire(client *c, redisDb *db, robj *key, long long when);
kvobj *setExpireByLink(client *c, redisDb *db, sds key, long long when, dictEntryLink link);
int checkAlreadyExpired(long long when);
int parseExtendedExpireArgumentsOrReply(client *c, int *flags);
robj *lookupKeyRead(redisDb *db, robj *key);
robj *lookupKeyWrite(redisDb *db, robj *key);
robj *lookupKeyWriteWithDictEntry(redisDb *db, robj *key, dictEntry **deref);
robj *lookupKeyReadOrReply(client *c, robj *key, robj *reply);
robj *lookupKeyWriteOrReply(client *c, robj *key, robj *reply);
robj *lookupKeyReadWithFlags(redisDb *db, robj *key, int flags);
robj *lookupKeyWriteWithFlags(redisDb *db, robj *key, int flags);
robj *objectCommandLookup(client *c, robj *key);
robj *objectCommandLookupOrReply(client *c, robj *key, robj *reply);
kvobj *lookupKeyRead(redisDb *db, robj *key);
kvobj *lookupKeyWrite(redisDb *db, robj *key);
kvobj *lookupKeyWriteWithLink(redisDb *db, robj *key, dictEntryLink *link);
kvobj *lookupKeyReadOrReply(client *c, robj *key, robj *reply);
kvobj *lookupKeyWriteOrReply(client *c, robj *key, robj *reply);
kvobj *lookupKeyReadWithFlags(redisDb *db, robj *key, int flags);
kvobj *lookupKeyWriteWithFlags(redisDb *db, robj *key, int flags);
kvobj *kvobjCommandLookup(client *c, robj *key);
kvobj *kvobjCommandLookupOrReply(client *c, robj *key, robj *reply);
int objectSetLRUOrLFU(robj *val, long long lfu_freq, long long lru_idle,
long long lru_clock, int lru_multiplier);
#define LOOKUP_NONE 0
@ -3587,25 +3613,27 @@ int objectSetLRUOrLFU(robj *val, long long lfu_freq, long long lru_idle,
#define LOOKUP_ACCESS_EXPIRED (1<<5) /* Allow lookup to expired key. */
#define LOOKUP_NOEFFECTS (LOOKUP_NONOTIFY | LOOKUP_NOSTATS | LOOKUP_NOTOUCH | LOOKUP_NOEXPIRE) /* Avoid any effects from fetching the key */
dictEntry *dbAdd(redisDb *db, robj *key, robj *val);
int dbAddRDBLoad(redisDb *db, sds key, robj *val);
void dbReplaceValue(redisDb *db, robj *key, robj *val);
void dbReplaceValueWithDictEntry(redisDb *db, robj *key, robj *val, dictEntry *de);
static inline kvobj *dictGetKV(const dictEntry *de) {return (kvobj *) dictGetKey(de);}
kvobj *dbAdd(redisDb *db, robj *key, robj **valref);
kvobj *dbAddByLink(redisDb *db, robj *key, robj **valref, dictEntryLink *link);
kvobj *dbAddRDBLoad(redisDb *db, sds key, robj **valref, long long expire);
void dbReplaceValue(redisDb *db, robj *key, kvobj **ioKeyVal);
void dbReplaceValueWithLink(redisDb *db, robj *key, robj **val, dictEntryLink link);
#define SETKEY_KEEPTTL 1
#define SETKEY_NO_SIGNAL 2
#define SETKEY_ALREADY_EXIST 4
#define SETKEY_DOESNT_EXIST 8
#define SETKEY_ADD_OR_UPDATE 16 /* Key most likely doesn't exists */
void setKey(client *c, redisDb *db, robj *key, robj *val, int flags);
void setKeyWithDictEntry(client *c, redisDb *db, robj *key, robj *val, int flags, dictEntry *de);
void setKey(client *c, redisDb *db, robj *key, robj **ioval, int flags);
void setKeyByLink(client *c, redisDb *db, robj *key, robj **valref, int flags, dictEntryLink *link);
robj *dbRandomKey(redisDb *db);
int dbGenericDelete(redisDb *db, robj *key, int async, int flags);
int dbSyncDelete(redisDb *db, robj *key);
int dbDelete(redisDb *db, robj *key);
int dbDeleteSkipKeysizesUpdate(redisDb *db, robj *key);
robj *dbUnshareStringValue(redisDb *db, robj *key, robj *o);
robj *dbUnshareStringValueWithDictEntry(redisDb *db, robj *key, robj *o, dictEntry *de);
kvobj *dbUnshareStringValue(redisDb *db, robj *key, kvobj *o);
kvobj *dbUnshareStringValueByLink(redisDb *db, robj *key, kvobj *kv, dictEntryLink link);
#define FLUSH_TYPE_ALL 0
#define FLUSH_TYPE_DB 1
@ -3755,7 +3783,7 @@ int clientsCronHandleTimeout(client *c, mstime_t now_ms);
/* expire.c -- Handling of expired keys */
void activeExpireCycle(int type);
void expireSlaveKeys(void);
void rememberSlaveKeyWithExpire(redisDb *db, robj *key);
void rememberSlaveKeyWithExpire(redisDb *db, sds key);
void flushSlaveKeysWithExpireList(void);
size_t getSlaveKeyWithExpireCount(void);
uint64_t hashTypeDbActiveExpire(redisDb *db, uint32_t maxFieldsToExpire);
@ -3777,10 +3805,9 @@ uint64_t dictSdsHash(const void *key);
uint64_t dictPtrHash(const void *key);
uint64_t dictSdsCaseHash(const void *key);
size_t dictSdsKeyLen(dict *d, const void *key);
int dictSdsKeyCompare(dict *d, const void *key1, const void *key2);
int dictSdsKeyCompareWithLen(dict *d, const void *key1, const size_t l1,const void *key2, const size_t l2);
int dictSdsMstrKeyCompare(dict *d, const void *sdsLookup, const void *mstrStored);
int dictSdsKeyCaseCompare(dict *d, const void *key1, const void *key2);
int dictSdsKeyCompare(dictCmpCache *cache, const void *key1, const void *key2);
int dictSdsMstrKeyCompare(dictCmpCache *cache, const void *sdsLookup, const void *mstrStored);
int dictSdsKeyCaseCompare(dictCmpCache *cache, const void *key1, const void *key2);
void dictSdsDestructor(dict *d, void *val);
void dictListDestructor(dict *d, void *val);
void *dictSdsDup(dict *d, const void *key);

40
src/sort.c
View File

@ -40,9 +40,11 @@ redisSortOperation *createSortOperation(int type, robj *pattern) {
* The returned object will always have its refcount increased by 1
* when it is non-NULL. */
robj *lookupKeyByPattern(redisDb *db, robj *pattern, robj *subst) {
kvobj *kv;
char *p, *f, *k;
sds spat, ssub;
robj *keyobj, *fieldobj = NULL, *o, *val;
robj *keyobj, *fieldobj = NULL, *val;
int prefixlen, sublen, postfixlen, fieldlen;
/* If the pattern is "#" return the substitution object itself in order
@ -87,31 +89,31 @@ robj *lookupKeyByPattern(redisDb *db, robj *pattern, robj *subst) {
decrRefCount(subst); /* Incremented by decodeObject() */
/* Lookup substituted key */
o = lookupKeyRead(db, keyobj);
if (o == NULL) goto noobj;
kv = lookupKeyRead(db, keyobj);
if (kv == NULL) goto noobj;
if (fieldobj) {
if (o->type != OBJ_HASH) goto noobj;
if (kv->type != OBJ_HASH) goto noobj;
/* Retrieve value from hash by the field name. The returned object
* is a new object with refcount already incremented. */
int isHashDeleted;
hashTypeGetValueObject(db, o, fieldobj->ptr, HFE_LAZY_EXPIRE, &val, NULL, &isHashDeleted);
o = val;
hashTypeGetValueObject(db, kv, fieldobj->ptr, HFE_LAZY_EXPIRE, &val, NULL, &isHashDeleted);
kv = val;
if (isHashDeleted)
goto noobj;
} else {
if (o->type != OBJ_STRING) goto noobj;
if (kv->type != OBJ_STRING) goto noobj;
/* Every object that this function returns needs to have its refcount
* increased. sortCommand decreases it again. */
incrRefCount(o);
incrRefCount(kv);
}
decrRefCount(keyobj);
if (fieldobj) decrRefCount(fieldobj);
return o;
return kv;
noobj:
decrRefCount(keyobj);
@ -600,18 +602,26 @@ void sortCommandGeneric(client *c, int readonly) {
}
}
}
if (outputlen) {
listTypeTryConversion(sobj,LIST_CONV_AUTO,NULL,NULL);
setKey(c,c->db,storekey,sobj,0);
setKey(c, c->db, storekey, &sobj, 0);
/* Ownership of sobj transferred to the db. Set to NULL to prevent
* freeing it below. */
sobj = NULL;
notifyKeyspaceEvent(NOTIFY_LIST,"sortstore",storekey,
c->db->id);
server.dirty += outputlen;
} else if (dbDelete(c->db,storekey)) {
signalModifiedKey(c,c->db,storekey);
notifyKeyspaceEvent(NOTIFY_GENERIC,"del",storekey,c->db->id);
server.dirty++;
/* Ownership of sobj transferred to the db. No need to free it. */
} else {
if (dbDelete(c->db, storekey)) {
signalModifiedKey(c, c->db, storekey);
notifyKeyspaceEvent(NOTIFY_GENERIC, "del", storekey, c->db->id);
server.dirty++;
}
decrRefCount(sobj);
}
decrRefCount(sobj);
addReplyLongLong(c,outputlen);
}

247
src/t_hash.c
View File

@ -51,17 +51,17 @@ static ExpireMeta* hfieldGetExpireMeta(const eItem field);
static ExpireMeta *hashGetExpireMeta(const eItem hash);
static void hexpireGenericCommand(client *c, long long basetime, int unit);
static ExpireAction hashTypeActiveExpire(eItem hashObj, void *ctx);
static uint64_t hashTypeExpire(robj *o, ExpireCtx *expireCtx, int updateGlobalHFE);
static uint64_t hashTypeExpire(kvobj *kv, ExpireCtx *expireCtx, int updateGlobalHFE);
static void hfieldPersist(robj *hashObj, hfield field);
static void propagateHashFieldDeletion(redisDb *db, sds key, char *field, size_t fieldLen);
/* hash dictType funcs */
static int dictHfieldKeyCompare(dict *d, const void *key1, const void *key2);
static int dictHfieldKeyCompare(dictCmpCache *cache, const void *key1, const void *key2);
static uint64_t dictMstrHash(const void *key);
static void dictHfieldDestructor(dict *d, void *field);
static size_t hashDictWithExpireMetadataBytes(dict *d);
static void hashDictWithExpireOnRelease(dict *d);
static robj* hashTypeLookupWriteOrCreate(client *c, robj *key);
static kvobj* hashTypeLookupWriteOrCreate(client *c, robj *key);
/*-----------------------------------------------------------------------------
* Define dictType of hash
@ -221,7 +221,7 @@ typedef struct HashTypeSetEx {
const char *cmd;
} HashTypeSetEx;
int hashTypeSetExInit(robj *key, robj *o, client *c, redisDb *db,
int hashTypeSetExInit(robj *key, kvobj *kvo, client *c, redisDb *db,
ExpireSetCond expireSetCond, HashTypeSetEx *ex);
SetExRes hashTypeSetEx(robj *o, sds field, uint64_t expireAt, HashTypeSetEx *exInfo);
@ -232,10 +232,10 @@ void hashTypeSetExDone(HashTypeSetEx *e);
* Accessor functions for dictType of hash
*----------------------------------------------------------------------------*/
static int dictHfieldKeyCompare(dict *d, const void *key1, const void *key2)
static int dictHfieldKeyCompare(dictCmpCache *cache, const void *key1, const void *key2)
{
int l1,l2;
UNUSED(d);
UNUSED(cache);
l1 = hfieldlen((hfield)key1);
l2 = hfieldlen((hfield)key2);
@ -282,12 +282,12 @@ static void hashDictWithExpireOnRelease(dict *d) {
/*
* If any of hash field expiration command is called on a listpack hash object
* for the first time, we convert it to OBJ_ENCODING_LISTPACK_EX encoding.
* We allocate "struct listpackEx" which holds listpack pointer and metadata to
* register key to the global DS. In the listpack, we append another TTL entry
* for each field-value pair. From now on, listpack will have triplets in it:
* field-value-ttl. If TTL is not set for a field, we store 'zero' as the TTL
* value. 'zero' is encoded as two bytes in the listpack. Memory overhead of a
* non-existing TTL will be two bytes per field.
* We allocate "struct listpackEx" which holds listpack pointer and expiry
* metadata. In the listpack string, we append another TTL entry for each field
* value pair. From now on, listpack will have triplets in it: field-value-ttl.
* If TTL is not set for a field, we store 'zero' as the TTL value. 'zero' is
* encoded as two bytes in the listpack. Memory overhead of a non-existing TTL
* will be two bytes per field.
*
* Fields in the listpack will be ordered by TTL. Field with the smallest expiry
* time will be the first item. Fields without TTL will be at the end of the
@ -300,7 +300,6 @@ struct listpackEx *listpackExCreate(void) {
listpackEx *lpt = zcalloc(sizeof(*lpt));
lpt->meta.trash = 1;
lpt->lp = NULL;
lpt->key = NULL;
return lpt;
}
@ -387,14 +386,16 @@ static uint64_t listpackExGetMinExpire(robj *o) {
}
/* Walk over fields and delete the expired ones. */
void listpackExExpire(redisDb *db, robj *o, ExpireInfo *info) {
serverAssert(o->encoding == OBJ_ENCODING_LISTPACK_EX);
void listpackExExpire(redisDb *db, kvobj *kv, ExpireInfo *info) {
serverAssert(kv->encoding == OBJ_ENCODING_LISTPACK_EX);
uint64_t expired = 0, min = EB_EXPIRE_TIME_INVALID;
unsigned char *ptr;
listpackEx *lpt = o->ptr;
listpackEx *lpt = kv->ptr;
ptr = lpFirst(lpt->lp);
sds key = kvobjGetKey(kv);
while (ptr != NULL && (info->itemsExpired < info->maxToExpire)) {
long long val;
int64_t flen;
@ -412,7 +413,7 @@ void listpackExExpire(redisDb *db, robj *o, ExpireInfo *info) {
if (val == HASH_LP_NO_TTL || (uint64_t) val > info->now)
break;
propagateHashFieldDeletion(db, ((listpackEx *) o->ptr)->key, (char *)((fref) ? fref : intbuf), flen);
propagateHashFieldDeletion(db, key, (char *)((fref) ? fref : intbuf), flen);
server.stat_expired_subkeys++;
ptr = lpNext(lpt->lp, ptr);
@ -426,10 +427,10 @@ void listpackExExpire(redisDb *db, robj *o, ExpireInfo *info) {
/* update keysizes */
unsigned long l = lpLength(lpt->lp) / 3;
updateKeysizesHist(db, getKeySlot(lpt->key), OBJ_HASH, l + expired, l);
updateKeysizesHist(db, getKeySlot(key), OBJ_HASH, l + expired, l);
}
min = hashTypeGetMinExpire(o, 1 /*accurate*/);
min = hashTypeGetMinExpire(kv, 1 /*accurate*/);
info->nextExpireTime = min;
}
@ -720,7 +721,7 @@ GetFieldRes hashTypeGetFromHashTable(robj *o, sds field, sds *value, uint64_t *e
* expiredAt - if the field has an expiration time, it will be set to the expiration
* time of the field. Otherwise, will be set to EB_EXPIRE_TIME_INVALID.
*/
GetFieldRes hashTypeGetValue(redisDb *db, robj *o, sds field, unsigned char **vstr,
GetFieldRes hashTypeGetValue(redisDb *db, kvobj *o, sds field, unsigned char **vstr,
unsigned int *vlen, long long *vll,
int hfeFlags, uint64_t *expiredAt)
{
@ -768,10 +769,7 @@ GetFieldRes hashTypeGetValue(redisDb *db, robj *o, sds field, unsigned char **vs
(isPausedActionsWithUpdate(PAUSE_ACTION_EXPIRE)))
return GETF_EXPIRED;
if (o->encoding == OBJ_ENCODING_LISTPACK_EX)
key = ((listpackEx *) o->ptr)->key;
else
key = ((dictExpireMetadata *) dictMetadata((dict*)o->ptr))->key;
key = kvobjGetKey(o);
/* delete the field and propagate the deletion */
serverAssert(hashTypeDelete(o, field, 1) == 1);
@ -815,7 +813,7 @@ GetFieldRes hashTypeGetValue(redisDb *db, robj *o, sds field, unsigned char **vs
*
* Returns 1 if the field exists, and 0 when it doesn't.
*/
int hashTypeGetValueObject(redisDb *db, robj *o, sds field, int hfeFlags,
int hashTypeGetValueObject(redisDb *db, kvobj *o, sds field, int hfeFlags,
robj **val, uint64_t *expireTime, int *isHashDeleted) {
unsigned char *vstr;
unsigned int vlen;
@ -857,7 +855,7 @@ int hashTypeGetValueObject(redisDb *db, robj *o, sds field, int hfeFlags,
*
* Returns 1 if the field exists, and 0 when it doesn't.
*/
int hashTypeExists(redisDb *db, robj *o, sds field, int hfeFlags, int *isHashDeleted) {
int hashTypeExists(redisDb *db, kvobj *o, sds field, int hfeFlags, int *isHashDeleted) {
unsigned char *vstr = NULL;
unsigned int vlen = UINT_MAX;
long long vll = LLONG_MAX;
@ -973,27 +971,25 @@ int hashTypeSet(redisDb *db, robj *o, sds field, sds value, int flags) {
} else if (o->encoding == OBJ_ENCODING_HT) {
dict *ht = o->ptr;
dictEntry *de, *existing;
const uint64_t hash = dictGetHash(ht,field);
dictEntry *de;
/* check if field already exists */
existing = dictFindByHash(ht, field, hash);
dictEntryLink bucket, link = dictFindLink(ht, field, &bucket);
/* check if field already exists */
if (existing == NULL) {
if (link == NULL) {
hfield newField = hfieldNew(field, sdslen(field), 0);
dictUseStoredKeyApi(ht, 1);
de = dictAddNonExistsByHash(ht, newField, hash);
dictUseStoredKeyApi(ht, 0);
dictSetKeyAtLink(ht, newField, &bucket, 1);
de = *bucket;
} else {
/* If attached TTL to the old field, then remove it from hash's
* private ebuckets when HASH_SET_KEEP_TTL is not set. */
if (!(flags & HASH_SET_KEEP_TTL)) {
hfield oldField = dictGetKey(existing);
hfield oldField = dictGetKey(*link);
hfieldPersist(o, oldField);
}
/* Free the old value */
sdsfree(dictGetVal(existing));
sdsfree(dictGetVal(*link));
update = 1;
de = existing;
de = *link;
}
if (flags & HASH_SET_TAKE_VALUE) {
@ -1133,17 +1129,16 @@ SetExRes hashTypeSetEx(robj *o, sds field, uint64_t expireAt, HashTypeSetEx *exI
return HSETEX_OK; /* never reach here */
}
void initDictExpireMetadata(sds key, robj *o) {
void initDictExpireMetadata(robj *o) {
dict *ht = o->ptr;
dictExpireMetadata *m = (dictExpireMetadata *) dictMetadata(ht);
m->key = key;
m->hfe = ebCreate(); /* Allocate HFE DS */
m->expireMeta.trash = 1; /* mark as trash (as long it wasn't ebAdd()) */
}
/* Init HashTypeSetEx struct before calling hashTypeSetEx() */
int hashTypeSetExInit(robj *key, robj *o, client *c, redisDb *db,
int hashTypeSetExInit(robj *key, kvobj *o, client *c, redisDb *db,
ExpireSetCond expireSetCond, HashTypeSetEx *ex)
{
dict *ht = o->ptr;
@ -1158,25 +1153,6 @@ int hashTypeSetExInit(robj *key, robj *o, client *c, redisDb *db,
/* Take care that HASH support expiration */
if (o->encoding == OBJ_ENCODING_LISTPACK) {
hashTypeConvert(o, OBJ_ENCODING_LISTPACK_EX, &c->db->hexpires);
listpackEx *lpt = o->ptr;
dictEntry *de = dbFind(c->db, key->ptr);
serverAssert(de != NULL);
lpt->key = dictGetKey(de);
} else if (o->encoding == OBJ_ENCODING_LISTPACK_EX) {
listpackEx *lpt = o->ptr;
/* If the hash previously had HFEs but later no longer does, the key ref
* (lpt->key) in the hash might become outdated after a MOVE/COPY/RENAME/RESTORE
* operation. These commands maintain the key ref only if HFEs are present.
* That is, we can only be sure that key ref is valid as long as it is not
* "trash". (TODO: dbFind() can be avoided. Instead need to extend the
* lookupKey*() to return dictEntry). */
if (lpt->meta.trash) {
dictEntry *de = dbFind(c->db, key->ptr);
serverAssert(de != NULL);
lpt->key = dictGetKey(de);
}
} else if (o->encoding == OBJ_ENCODING_HT) {
/* Take care dict has HFE metadata */
if (!isDictWithMetaHFE(ht)) {
@ -1187,25 +1163,10 @@ int hashTypeSetExInit(robj *key, robj *o, client *c, redisDb *db,
/* Find the key in the keyspace. Need to keep reference to the key for
* notifications or even removal of the hash */
dictEntry *de = dbFind(db, key->ptr);
serverAssert(de != NULL);
/* Fillup dict HFE metadata */
m->key = dictGetKey(de); /* reference key in keyspace */
m->hfe = ebCreate(); /* Allocate HFE DS */
m->expireMeta.trash = 1; /* mark as trash (as long it wasn't ebAdd()) */
} else {
dictExpireMetadata *m = (dictExpireMetadata *) dictMetadata(ht);
/* If the hash previously had HFEs but later no longer does, the key ref
* (m->key) in the hash might become outdated after a MOVE/COPY/RENAME/RESTORE
* operation. These commands maintain the key ref only if HFEs are present.
* That is, we can only be sure that key ref is valid as long as it is not
* "trash". */
if (m->expireMeta.trash) {
dictEntry *de = dbFind(db, key->ptr);
serverAssert(de != NULL);
m->key = dictGetKey(de); /* reference key in keyspace */
}
}
}
@ -1555,15 +1516,16 @@ hfield hashTypeCurrentObjectNewHfield(hashTypeIterator *hi) {
return hf;
}
static robj *hashTypeLookupWriteOrCreate(client *c, robj *key) {
robj *o = lookupKeyWrite(c->db,key);
if (checkType(c,o,OBJ_HASH)) return NULL;
static kvobj *hashTypeLookupWriteOrCreate(client *c, robj *key) {
dictEntryLink link;
kvobj *kv = lookupKeyWriteWithLink(c->db, key, &link);
if (checkType(c, kv, OBJ_HASH)) return NULL;
if (o == NULL) {
o = createHashObject();
dbAdd(c->db,key,o);
if (kv == NULL) {
robj *o = createHashObject();
kv = dbAddByLink(c->db, key, &o, &link);
}
return o;
return kv;
}
@ -1646,7 +1608,6 @@ void hashTypeConvertListpackEx(robj *o, int enc, ebuckets *hexpires) {
dictExpireMeta = (dictExpireMetadata *) dictMetadata(dict);
/* Fillup dict HFE metadata */
dictExpireMeta->key = lpt->key; /* reference key in keyspace */
dictExpireMeta->hfe = ebCreate(); /* Allocate HFE DS */
dictExpireMeta->expireMeta.trash = 1; /* mark as trash (as long it wasn't ebAdd()) */
@ -1700,7 +1661,7 @@ void hashTypeConvert(robj *o, int enc, ebuckets *hexpires) {
* has the same encoding as the original one.
*
* The resulting object always has refcount set to 1 */
robj *hashTypeDup(robj *o, sds newkey, uint64_t *minHashExpire) {
robj *hashTypeDup(kvobj *o, uint64_t *minHashExpire) {
robj *hobj;
hashTypeIterator *hi;
@ -1720,7 +1681,6 @@ robj *hashTypeDup(robj *o, sds newkey, uint64_t *minHashExpire) {
*minHashExpire = ebGetMetaExpTime(&lpt->meta);
listpackEx *dup = listpackExCreate();
dup->key = newkey;
size_t sz = lpBytes(lpt->lp);
dup->lp = lpNew(sz);
@ -1740,7 +1700,6 @@ robj *hashTypeDup(robj *o, sds newkey, uint64_t *minHashExpire) {
d = dictCreate(&mstrHashDictTypeWithHFE);
dictExpireMetaSrc = (dictExpireMetadata *) dictMetadata((dict *) o->ptr);
dictExpireMetaDst = (dictExpireMetadata *) dictMetadata(d);
dictExpireMetaDst->key = newkey; /* reference key in keyspace */
dictExpireMetaDst->hfe = ebCreate(); /* Allocate HFE DS */
dictExpireMetaDst->expireMeta.trash = 1; /* mark as trash (as long it wasn't ebAdd()) */
@ -1844,7 +1803,7 @@ static ExpireAction hashTypeActiveExpire(eItem item, void *ctx) {
if (expireCtx->fieldsToExpireQuota == 0)
return ACT_STOP_ACTIVE_EXP;
uint64_t nextExpTime = hashTypeExpire((robj *) item, expireCtx, 0);
uint64_t nextExpTime = hashTypeExpire((kvobj *) item, expireCtx, 0);
/* If hash has no more fields to expire or got deleted, indicate
* to remove it from HFE DB to the caller ebExpire() */
@ -1867,10 +1826,9 @@ static ExpireAction hashTypeActiveExpire(eItem item, void *ctx) {
* - 0 if hash got deleted
* - EB_EXPIRE_TIME_INVALID if no more fields to expire
*/
static uint64_t hashTypeExpire(robj *o, ExpireCtx *expireCtx, int updateGlobalHFE) {
static uint64_t hashTypeExpire(kvobj *o, ExpireCtx *expireCtx, int updateGlobalHFE) {
uint64_t noExpireLeftRes = EB_EXPIRE_TIME_INVALID;
redisDb *db = expireCtx->db;
sds keystr = NULL;
ExpireInfo info = {0};
if (o->encoding == OBJ_ENCODING_LISTPACK_EX) {
@ -1880,7 +1838,6 @@ static uint64_t hashTypeExpire(robj *o, ExpireCtx *expireCtx, int updateGlobalHF
.itemsExpired = 0};
listpackExExpire(db, o, &info);
keystr = ((listpackEx*)o->ptr)->key;
} else {
serverAssert(o->encoding == OBJ_ENCODING_HT);
@ -1897,7 +1854,6 @@ static uint64_t hashTypeExpire(robj *o, ExpireCtx *expireCtx, int updateGlobalHF
};
ebExpire(&dictExpireMeta->hfe, &hashFieldExpireBucketsType, &info);
keystr = dictExpireMeta->key;
}
/* Update quota left */
@ -1907,6 +1863,7 @@ static uint64_t hashTypeExpire(robj *o, ExpireCtx *expireCtx, int updateGlobalHF
* As a result, active-expire might not expire any fields, in such cases,
* we don't need to send notifications or perform other operations for this key. */
if (info.itemsExpired) {
sds keystr = kvobjGetKey(o);
robj *key = createStringObject(keystr, sdslen(keystr));
notifyKeyspaceEvent(NOTIFY_HASH, "hexpired", key, db->id);
@ -1914,8 +1871,8 @@ static uint64_t hashTypeExpire(robj *o, ExpireCtx *expireCtx, int updateGlobalHF
ebRemove(&db->hexpires, &hashExpireBucketsType, o);
if (hashTypeLength(o, 0) == 0) {
dbDelete(db, key);
notifyKeyspaceEvent(NOTIFY_GENERIC, "del", key, db->id);
dbDelete(db, key);
noExpireLeftRes = 0;
} else {
if ((updateGlobalHFE) && (info.nextExpireTime != EB_EXPIRE_TIME_INVALID))
@ -1936,7 +1893,7 @@ static uint64_t hashTypeExpire(robj *o, ExpireCtx *expireCtx, int updateGlobalHF
* Return 1 if the entire hash was deleted, 0 otherwise.
* This function might be pricy in case there are many expired fields.
*/
static int hashTypeExpireIfNeeded(redisDb *db, robj *o) {
static int hashTypeExpireIfNeeded(redisDb *db, kvobj *o) {
uint64_t nextExpireTime;
uint64_t minExpire = hashTypeGetMinExpire(o, 1 /*accurate*/);
@ -2045,7 +2002,6 @@ int hashTypeIsFieldsWithExpire(robj *o) {
/* Add hash to global HFE DS and update key for notifications.
*
* key - must be the same key instance that is persisted in db->dict
* expireTime - expiration in msec.
* If eq. 0 then the hash will be added to the global HFE DS with
* the minimum expiration time that is already written in advance
@ -2054,13 +2010,12 @@ int hashTypeIsFieldsWithExpire(robj *o) {
*
* Precondition: It is a hash of type listpackex or HT with HFE metadata.
*/
void hashTypeAddToExpires(redisDb *db, sds key, robj *hashObj, uint64_t expireTime) {
void hashTypeAddToExpires(redisDb *db, kvobj *hashObj, uint64_t expireTime) {
if (expireTime > EB_EXPIRE_TIME_MAX)
return;
if (hashObj->encoding == OBJ_ENCODING_LISTPACK_EX) {
listpackEx *lpt = hashObj->ptr;
lpt->key = key;
expireTime = (expireTime) ? expireTime : ebGetMetaExpTime(&lpt->meta);
ebAdd(&db->hexpires, &hashExpireBucketsType, hashObj, expireTime);
} else if (hashObj->encoding == OBJ_ENCODING_HT) {
@ -2068,7 +2023,6 @@ void hashTypeAddToExpires(redisDb *db, sds key, robj *hashObj, uint64_t expireTi
if (isDictWithMetaHFE(d)) {
dictExpireMetadata *meta = (dictExpireMetadata *) dictMetadata(d);
expireTime = (expireTime) ? expireTime : ebGetMetaExpTime(&meta->expireMeta);
meta->key = key;
ebAdd(&db->hexpires, &hashExpireBucketsType, hashObj, expireTime);
}
}
@ -2077,7 +2031,7 @@ void hashTypeAddToExpires(redisDb *db, sds key, robj *hashObj, uint64_t expireTi
/* DB active expire and update hashes with time-expiration on fields.
*
* The callback function hashTypeActiveExpire() is invoked for each hash registered
* in the HFE DB (db->expires) with an expiration-time less than or equal to the
* in the HFE DB (db->hexpires) with an expiration-time less than or equal to the
* current time. This callback performs the following actions for each hash:
* - If the hash has one or more fields to expire, it will delete those fields.
* - If there are more fields to expire, it will update the hash with the next
@ -2126,19 +2080,6 @@ void hashTypeFree(robj *o) {
}
}
/* Attempts to update the reference to the new key. Now it's only used in defrag. */
void hashTypeUpdateKeyRef(robj *o, sds newkey) {
if (o->encoding == OBJ_ENCODING_LISTPACK_EX) {
listpackEx *lpt = o->ptr;
lpt->key = newkey;
} else if (o->encoding == OBJ_ENCODING_HT && isDictWithMetaHFE(o->ptr)) {
dictExpireMetadata *dictExpireMeta = (dictExpireMetadata *)dictMetadata((dict*)o->ptr);
dictExpireMeta->key = newkey;
} else {
/* Nothing to do. */
}
}
ebuckets *hashTypeGetDictMetaHFE(dict *d) {
dictExpireMetadata *dictExpireMeta = (dictExpireMetadata *) dictMetadata(d);
return &dictExpireMeta->hfe;
@ -2151,44 +2092,44 @@ ebuckets *hashTypeGetDictMetaHFE(dict *d) {
void hsetnxCommand(client *c) {
unsigned long hlen;
int isHashDeleted;
robj *o;
if ((o = hashTypeLookupWriteOrCreate(c,c->argv[1])) == NULL) return;
robj *kv = hashTypeLookupWriteOrCreate(c,c->argv[1]);
if (kv == NULL) return;
if (hashTypeExists(c->db, o, c->argv[2]->ptr, HFE_LAZY_EXPIRE, &isHashDeleted)) {
if (hashTypeExists(c->db, kv, c->argv[2]->ptr, HFE_LAZY_EXPIRE, &isHashDeleted)) {
addReply(c, shared.czero);
return;
}
/* Field expired and in turn hash deleted. Create new one! */
if (isHashDeleted) {
o = createHashObject();
dbAdd(c->db,c->argv[1],o);
robj *o = createHashObject();
kv = dbAdd(c->db,c->argv[1],&o);
}
hashTypeTryConversion(c->db, o,c->argv,2,3);
hashTypeSet(c->db, o,c->argv[2]->ptr,c->argv[3]->ptr,HASH_SET_COPY);
hashTypeTryConversion(c->db, kv, c->argv, 2, 3);
hashTypeSet(c->db, kv, c->argv[2]->ptr, c->argv[3]->ptr, HASH_SET_COPY);
addReply(c, shared.cone);
signalModifiedKey(c,c->db,c->argv[1]);
notifyKeyspaceEvent(NOTIFY_HASH,"hset",c->argv[1],c->db->id);
hlen = hashTypeLength(o, 0);
hlen = hashTypeLength(kv, 0);
updateKeysizesHist(c->db, getKeySlot(c->argv[1]->ptr), OBJ_HASH, hlen - 1, hlen);
server.dirty++;
}
void hsetCommand(client *c) {
int i, created = 0;
robj *o;
kvobj *kv;
if ((c->argc % 2) == 1) {
addReplyErrorArity(c);
return;
}
if ((o = hashTypeLookupWriteOrCreate(c,c->argv[1])) == NULL) return;
hashTypeTryConversion(c->db,o,c->argv,2,c->argc-1);
if ((kv = hashTypeLookupWriteOrCreate(c,c->argv[1])) == NULL) return;
hashTypeTryConversion(c->db, kv, c->argv, 2, c->argc-1);
for (i = 2; i < c->argc; i += 2)
created += !hashTypeSet(c->db, o,c->argv[i]->ptr,c->argv[i+1]->ptr,HASH_SET_COPY);
created += !hashTypeSet(c->db, kv, c->argv[i]->ptr, c->argv[i+1]->ptr, HASH_SET_COPY);
/* HMSET (deprecated) and HSET return value is different. */
char *cmdname = c->argv[0]->ptr;
@ -2200,7 +2141,7 @@ void hsetCommand(client *c) {
addReply(c, shared.ok);
}
signalModifiedKey(c,c->db,c->argv[1]);
unsigned long l = hashTypeLength(o, 0);
unsigned long l = hashTypeLength(kv, 0);
updateKeysizesHist(c->db, getKeySlot(c->argv[1]->ptr), OBJ_HASH, l - created, l);
notifyKeyspaceEvent(NOTIFY_HASH,"hset",c->argv[1],c->db->id);
server.dirty += (c->argc - 2)/2;
@ -2388,14 +2329,14 @@ void hsetexCommand(client *c) {
int updated = 0, deleted = 0, set_expiry;
long long expire_time = EB_EXPIRE_TIME_INVALID;
int64_t oldlen, newlen;
robj *o;
HashTypeSetEx setex;
dictEntryLink link;
if (hsetexParseArgs(c, &flags, &expire_time, &expire_time_pos,
&first_field_pos, &field_count) != C_OK)
return;
o = lookupKeyWrite(c->db, c->argv[1]);
kvobj *o = lookupKeyWriteWithLink(c->db, c->argv[1], &link);
if (checkType(c, o, OBJ_HASH))
return;
@ -2405,7 +2346,7 @@ void hsetexCommand(client *c) {
return;
}
o = createHashObject();
dbAdd(c->db, c->argv[1], o);
dbAddByLink(c->db, c->argv[1], &o, &link);
}
oldlen = (int64_t) hashTypeLength(o, 0);
@ -2508,7 +2449,7 @@ out:
void hincrbyCommand(client *c) {
long long value, incr, oldvalue;
robj *o;
kvobj *o;
sds new;
unsigned char *vstr;
unsigned int vlen;
@ -2532,7 +2473,7 @@ void hincrbyCommand(client *c) {
} else {
/* Field expired and in turn hash deleted. Create new one! */
o = createHashObject();
dbAdd(c->db,c->argv[1],o);
dbAdd(c->db,c->argv[1],&o);
value = 0;
updateKeysizesHist(c->db, getKeySlot(c->argv[1]->ptr), OBJ_HASH, 0, 1);
}
@ -2555,7 +2496,7 @@ void hincrbyCommand(client *c) {
void hincrbyfloatCommand(client *c) {
long double value, incr;
long long ll;
robj *o;
kvobj *o;
sds new;
unsigned char *vstr;
unsigned int vlen;
@ -2584,7 +2525,7 @@ void hincrbyfloatCommand(client *c) {
} else {
/* Field expired and in turn hash deleted. Create new one! */
o = createHashObject();
dbAdd(c->db,c->argv[1],o);
dbAdd(c->db, c->argv[1], &o);
value = 0;
updateKeysizesHist(c->db, getKeySlot(c->argv[1]->ptr), OBJ_HASH, 0, 1);
}
@ -2614,7 +2555,7 @@ void hincrbyfloatCommand(client *c) {
decrRefCount(newobj);
}
static GetFieldRes addHashFieldToReply(client *c, robj *o, sds field, int hfeFlags) {
static GetFieldRes addHashFieldToReply(client *c, kvobj *o, sds field, int hfeFlags) {
if (o == NULL) {
addReplyNull(c);
return GETF_NOT_FOUND;
@ -2638,7 +2579,7 @@ static GetFieldRes addHashFieldToReply(client *c, robj *o, sds field, int hfeFla
}
void hgetCommand(client *c) {
robj *o;
kvobj *o;
if ((o = lookupKeyReadOrReply(c,c->argv[1],shared.null[c->resp])) == NULL ||
checkType(c,o,OBJ_HASH)) return;
@ -2648,13 +2589,12 @@ void hgetCommand(client *c) {
void hmgetCommand(client *c) {
GetFieldRes res = GETF_OK;
robj *o;
int i;
int expired = 0, deleted = 0;
/* Don't abort when the key cannot be found. Non-existing keys are empty
* hashes, where HMGET should respond with a series of null bulks. */
o = lookupKeyRead(c->db, c->argv[1]);
kvobj *o = lookupKeyRead(c->db, c->argv[1]);
if (checkType(c,o,OBJ_HASH)) return;
addReplyArrayLen(c, c->argc-2);
@ -2686,9 +2626,8 @@ void hgetdelCommand(client *c) {
int res = 0, hfe = 0, deleted = 0, expired = 0;
int64_t oldlen = -1; /* not exists as long as it is not set */
long num_fields = 0;
robj *o;
o = lookupKeyWrite(c->db, c->argv[1]);
kvobj *o = lookupKeyWrite(c->db, c->argv[1]);
if (checkType(c, o, OBJ_HASH))
return;
@ -2785,10 +2724,9 @@ void hgetexCommand(client *c) {
long num_fields;
int64_t oldlen = 0, newlen = -1;
long long expire_time = 0;
robj *o;
HashTypeSetEx setex;
o = lookupKeyWrite(c->db, c->argv[1]);
kvobj *o = lookupKeyWrite(c->db, c->argv[1]);
if (checkType(c, o, OBJ_HASH))
return;
@ -2920,7 +2858,7 @@ void hgetexCommand(client *c) {
/* Key may become empty due to lazy expiry in addHashFieldToReply()
* or the new expiration time is in the past.*/
newlen = hashTypeLength(o, 0);
updateKeysizesHist(c->db, getKeySlot(c->argv[1]->ptr), OBJ_HASH, oldlen, newlen);
if (newlen == 0) {
dbDelete(c->db, c->argv[1]);
@ -2929,7 +2867,7 @@ void hgetexCommand(client *c) {
}
void hdelCommand(client *c) {
robj *o;
kvobj *o;
int j, deleted = 0, keyremoved = 0;
if ((o = lookupKeyWriteOrReply(c,c->argv[1],shared.czero)) == NULL ||
@ -2974,7 +2912,7 @@ void hdelCommand(client *c) {
}
void hlenCommand(client *c) {
robj *o;
kvobj *o;
if ((o = lookupKeyReadOrReply(c,c->argv[1],shared.czero)) == NULL ||
checkType(c,o,OBJ_HASH)) return;
@ -2983,7 +2921,7 @@ void hlenCommand(client *c) {
}
void hstrlenCommand(client *c) {
robj *o;
kvobj *o;
unsigned char *vstr = NULL;
unsigned int vlen = UINT_MAX;
long long vll = LLONG_MAX;
@ -3027,7 +2965,7 @@ static void addHashIteratorCursorToReply(client *c, hashTypeIterator *hi, int wh
}
void genericHgetallCommand(client *c, int flags) {
robj *o;
kvobj *o;
hashTypeIterator *hi;
int length, count = 0;
@ -3082,7 +3020,7 @@ void hgetallCommand(client *c) {
}
void hexistsCommand(client *c) {
robj *o;
kvobj *o;
if ((o = lookupKeyReadOrReply(c,c->argv[1],shared.czero)) == NULL ||
checkType(c,o,OBJ_HASH)) return;
@ -3091,7 +3029,7 @@ void hexistsCommand(client *c) {
}
void hscanCommand(client *c) {
robj *o;
kvobj *o;
unsigned long long cursor;
if (parseScanCursorOrReply(c,c->argv[2],&cursor) == C_ERR) return;
@ -3131,7 +3069,7 @@ static void hrandfieldReplyWithListpack(client *c, unsigned int count, listpackE
void hrandfieldWithCountCommand(client *c, long l, int withvalues) {
unsigned long count, size;
int uniq = 1;
robj *hash;
kvobj *hash;
if ((hash = lookupKeyReadOrReply(c,c->argv[1],shared.emptyarray))
== NULL || checkType(c,hash,OBJ_HASH)) return;
@ -3376,7 +3314,7 @@ void hrandfieldWithCountCommand(client *c, long l, int withvalues) {
void hrandfieldCommand(client *c) {
long l;
int withvalues = 0;
robj *hash;
kvobj *hash;
CommonEntry ele;
if (c->argc >= 3) {
@ -3524,13 +3462,13 @@ static void propagateHashFieldDeletion(redisDb *db, sds key, char *field, size_t
static ExpireAction onFieldExpire(eItem item, void *ctx) {
OnFieldExpireCtx *expCtx = ctx;
hfield hf = item;
dict *d = expCtx->hashObj->ptr;
dictExpireMetadata *dictExpireMeta = (dictExpireMetadata *) dictMetadata(d);
propagateHashFieldDeletion(expCtx->db, dictExpireMeta->key, hf, hfieldlen(hf));
kvobj *kv = expCtx->hashObj;
sds key = kvobjGetKey(kv);
propagateHashFieldDeletion(expCtx->db, key, hf, hfieldlen(hf));
/* update keysizes */
unsigned long l = hashTypeLength(expCtx->hashObj, 0);
updateKeysizesHist(expCtx->db, getKeySlot(dictExpireMeta->key), OBJ_HASH, l, l - 1);
updateKeysizesHist(expCtx->db, getKeySlot(key), OBJ_HASH, l, l - 1);
serverAssert(hashTypeDelete(expCtx->hashObj, hf, 0) == 1);
server.stat_expired_subkeys++;
@ -3556,7 +3494,7 @@ static ExpireMeta *hashGetExpireMeta(const eItem hash) {
/* HTTL key <FIELDS count field [field ...]> */
static void httlGenericCommand(client *c, const char *cmd, long long basetime, int unit) {
UNUSED(cmd);
robj *hashObj;
kvobj *hashObj;
long numFields = 0, numFieldsAt = 3;
/* Read the hash object */
@ -3715,10 +3653,10 @@ static void hexpireGenericCommand(client *c, long long basetime, int unit) {
long long expire; /* unix time in msec */
int fieldAt, fieldsNotSet = 0, expireSetCond = 0, updated = 0, deleted = 0;
int64_t oldlen, newlen;
robj *hashObj, *keyArg = c->argv[1], *expireArg = c->argv[2];
robj *keyArg = c->argv[1], *expireArg = c->argv[2];
/* Read the hash object */
hashObj = lookupKeyWrite(c->db, keyArg);
kvobj *hashObj = lookupKeyWrite(c->db, keyArg);
if (checkType(c, hashObj, OBJ_HASH))
return;
@ -3880,12 +3818,11 @@ void hpexpiretimeCommand(client *c) {
/* HPERSIST key FIELDS numfields <field [field ...]> */
void hpersistCommand(client *c) {
robj *hashObj;
long numFields = 0, numFieldsAt = 3;
int changed = 0; /* Used to determine whether to send a notification. */
/* Read the hash object */
hashObj = lookupKeyWrite(c->db, c->argv[1]);
kvobj *hashObj = lookupKeyWrite(c->db, c->argv[1]);
if (checkType(c, hashObj, OBJ_HASH))
return;

59
src/t_list.c
View File

@ -465,9 +465,10 @@ void listTypeDelRange(robj *subject, long start, long count) {
* 'xx': push if key exists. */
void pushGenericCommand(client *c, int where, int xx) {
unsigned long llen;
dictEntryLink link;
int j;
robj *lobj = lookupKeyWrite(c->db, c->argv[1]);
kvobj *lobj = lookupKeyWriteWithLink(c->db, c->argv[1], &link);
if (checkType(c,lobj,OBJ_LIST)) return;
if (!lobj) {
if (xx) {
@ -476,7 +477,7 @@ void pushGenericCommand(client *c, int where, int xx) {
}
lobj = createListListpackObject();
dbAdd(c->db,c->argv[1],lobj);
dbAddByLink(c->db, c->argv[1], &lobj, &link);
}
listTypeTryConversionAppend(lobj,c->argv,2,c->argc-1,NULL,NULL);
@ -517,7 +518,7 @@ void rpushxCommand(client *c) {
/* LINSERT <key> (BEFORE|AFTER) <pivot> <element> */
void linsertCommand(client *c) {
int where;
robj *subject;
kvobj *subject;
listTypeIterator *iter;
listTypeEntry entry;
int inserted = 0;
@ -571,14 +572,14 @@ void linsertCommand(client *c) {
/* LLEN <key> */
void llenCommand(client *c) {
robj *o = lookupKeyReadOrReply(c,c->argv[1],shared.czero);
if (o == NULL || checkType(c,o,OBJ_LIST)) return;
addReplyLongLong(c,listTypeLength(o));
kvobj *kv = lookupKeyReadOrReply(c,c->argv[1],shared.czero);
if (kv == NULL || checkType(c,kv,OBJ_LIST)) return;
addReplyLongLong(c,listTypeLength(kv));
}
/* LINDEX <key> <index> */
void lindexCommand(client *c) {
robj *o = lookupKeyReadOrReply(c,c->argv[1],shared.null[c->resp]);
kvobj *o = lookupKeyReadOrReply(c,c->argv[1],shared.null[c->resp]);
if (o == NULL || checkType(c,o,OBJ_LIST)) return;
long index;
@ -607,7 +608,7 @@ void lindexCommand(client *c) {
/* LSET <key> <index> <element> */
void lsetCommand(client *c) {
robj *o = lookupKeyWriteOrReply(c,c->argv[1],shared.nokeyerr);
kvobj *o = lookupKeyWriteOrReply(c, c->argv[1], shared.nokeyerr);
if (o == NULL || checkType(c,o,OBJ_LIST)) return;
long index;
robj *value = c->argv[3];
@ -775,7 +776,7 @@ void popGenericCommand(client *c, int where) {
return;
}
robj *o = lookupKeyWriteOrReply(c, c->argv[1], hascount ? shared.nullarray[c->resp]: shared.null[c->resp]);
kvobj *o = lookupKeyWriteOrReply(c, c->argv[1], hascount ? shared.nullarray[c->resp] : shared.null[c->resp]);
if (o == NULL || checkType(c, o, OBJ_LIST))
return;
@ -861,7 +862,7 @@ void rpopCommand(client *c) {
/* LRANGE <key> <start> <stop> */
void lrangeCommand(client *c) {
robj *o;
kvobj *o;
long start, end;
if ((getLongFromObjectOrReply(c, c->argv[2], &start, NULL) != C_OK) ||
@ -875,7 +876,7 @@ void lrangeCommand(client *c) {
/* LTRIM <key> <start> <stop> */
void ltrimCommand(client *c) {
robj *o;
kvobj *o;
long start, end, llen, ltrim, rtrim, llenNew;
if ((getLongFromObjectOrReply(c, c->argv[2], &start, NULL) != C_OK) ||
@ -945,7 +946,7 @@ void ltrimCommand(client *c) {
* The returned elements indexes are always referring to what LINDEX
* would return. So first element from head is 0, and so forth. */
void lposCommand(client *c) {
robj *o, *ele;
robj *ele;
ele = c->argv[2];
int direction = LIST_TAIL;
long rank = 1, count = -1, maxlen = 0; /* Count -1: option not given. */
@ -989,7 +990,8 @@ void lposCommand(client *c) {
/* We return NULL or an empty array if there is no such key (or
* if we find no matches, depending on the presence of the COUNT option. */
if ((o = lookupKeyRead(c->db,c->argv[1])) == NULL) {
kvobj *o = lookupKeyRead(c->db,c->argv[1]);
if (o == NULL) {
if (count != -1)
addReply(c,shared.emptyarray);
else
@ -1042,7 +1044,7 @@ void lposCommand(client *c) {
/* LREM <key> <count> <element> */
void lremCommand(client *c) {
robj *subject, *obj;
robj *obj;
obj = c->argv[3];
long toremove;
long removed = 0;
@ -1050,7 +1052,7 @@ void lremCommand(client *c) {
if (getRangeLongFromObjectOrReply(c, c->argv[2], -LONG_MAX, LONG_MAX, &toremove, NULL) != C_OK)
return;
subject = lookupKeyWriteOrReply(c,c->argv[1],shared.czero);
kvobj *subject = lookupKeyWriteOrReply(c, c->argv[1], shared.czero);
if (subject == NULL || checkType(c,subject,OBJ_LIST)) return;
listTypeIterator *li;
@ -1095,7 +1097,7 @@ void lmoveHandlePush(client *c, robj *dstkey, robj *dstobj, robj *value,
/* Create the list if the key does not exist */
if (!dstobj) {
dstobj = createListListpackObject();
dbAdd(c->db,dstkey,dstobj);
dbAdd(c->db, dstkey, &dstobj);
}
listTypeTryConversionAppend(dstobj,&value,0,0,NULL,NULL);
listTypePush(dstobj,value,where);
@ -1131,32 +1133,31 @@ robj *getStringObjectFromListPosition(int position) {
}
void lmoveGenericCommand(client *c, int wherefrom, int whereto) {
robj *sobj, *value;
if ((sobj = lookupKeyWriteOrReply(c,c->argv[1],shared.null[c->resp]))
== NULL || checkType(c,sobj,OBJ_LIST)) return;
kvobj *kvsrc = lookupKeyWriteOrReply(c,c->argv[1],shared.null[c->resp]);
if (kvsrc == NULL || checkType(c,kvsrc,OBJ_LIST)) return;
if (listTypeLength(sobj) == 0) {
if (listTypeLength(kvsrc) == 0) {
/* This may only happen after loading very old RDB files. Recent
* versions of Redis delete keys of empty lists. */
addReplyNull(c);
} else {
robj *dobj, *skey = c->argv[1];
robj *kvdst, *skey = c->argv[1];
int64_t oldlen = 0, newlen = 1; /* init lengths assuming new dst object */
if ((dobj = lookupKeyWrite(c->db,c->argv[2])) != NULL) {
if (checkType(c,dobj,OBJ_LIST)) return;
if ((kvdst = lookupKeyWrite(c->db,c->argv[2])) != NULL) {
if (checkType(c,kvdst,OBJ_LIST)) return;
/* dst object exists */
oldlen = (int64_t) listTypeLength(dobj);
oldlen = (int64_t) listTypeLength(kvdst);
newlen = oldlen + 1;
}
value = listTypePop(sobj,wherefrom);
robj *value = listTypePop(kvsrc, wherefrom);
serverAssert(value); /* assertion for valgrind (avoid NPD) */
lmoveHandlePush(c,c->argv[2],dobj,value,whereto);
lmoveHandlePush(c, c->argv[2], kvdst, value, whereto);
/* Update dst obj cardinality in KEYSIZES */
updateKeysizesHist(c->db, getKeySlot(c->argv[2]->ptr), OBJ_LIST, oldlen, newlen);
/* Update src obj cardinality in KEYSIZES by listElementsRemoved() */
listElementsRemoved(c,skey,wherefrom,sobj,1,1,NULL);
listElementsRemoved(c, skey, wherefrom, kvsrc, 1, 1, NULL);
/* listTypePop returns an object with its refcount incremented */
decrRefCount(value);

104
src/t_set.c
View File

@ -127,16 +127,17 @@ int setTypeAddAux(robj *set, char *str, size_t len, int64_t llval, int str_is_sd
/* Avoid duping the string if it is an sds string. */
sds sdsval = str_is_sds ? (sds)str : sdsnewlen(str, len);
dict *ht = set->ptr;
void *position = dictFindPositionForInsert(ht, sdsval, NULL);
if (position) {
dictEntryLink bucket, link = dictFindLink(ht, sdsval, &bucket);
if (link == NULL) {
/* Key doesn't already exist in the set. Add it but dup the key. */
if (sdsval == str) sdsval = sdsdup(sdsval);
dictInsertAtPosition(ht, sdsval, position);
dictSetKeyAtLink(ht, sdsval, &bucket, 1);
return 1;
} else if (sdsval != str) {
/* String is already a member. Free our temporary sds copy. */
sdsfree(sdsval);
return 0;
}
return (position != NULL);
} else if (set->encoding == OBJ_ENCODING_LISTPACK) {
unsigned char *lp = set->ptr;
unsigned char *p = lpFirst(lp);
@ -587,15 +588,16 @@ robj *setTypeDup(robj *o) {
}
void saddCommand(client *c) {
robj *set;
kvobj *set;
int j, added = 0;
dictEntryLink link;
set = lookupKeyWrite(c->db,c->argv[1]);
set = lookupKeyWriteWithLink(c->db,c->argv[1], &link);
if (checkType(c,set,OBJ_SET)) return;
if (set == NULL) {
set = setTypeCreate(c->argv[2]->ptr, c->argc - 2);
dbAdd(c->db,c->argv[1],set);
robj *o = setTypeCreate(c->argv[2]->ptr, c->argc - 2);
set = dbAddByLink(c->db, c->argv[1], &o, &link);
} else {
setTypeMaybeConvert(set, c->argc - 2);
}
@ -614,11 +616,11 @@ void saddCommand(client *c) {
}
void sremCommand(client *c) {
robj *set;
int j, deleted = 0, keyremoved = 0;
if ((set = lookupKeyWriteOrReply(c,c->argv[1],shared.czero)) == NULL ||
checkType(c,set,OBJ_SET)) return;
kvobj *set = lookupKeyWriteOrReply(c, c->argv[1], shared.czero);
if (set == NULL || checkType(c, set, OBJ_SET))
return;
unsigned long oldSize = setTypeSize(set);
@ -633,8 +635,8 @@ void sremCommand(client *c) {
}
}
if (deleted) {
int64_t newSize = oldSize - deleted;
int64_t newSize = oldSize - deleted;
signalModifiedKey(c,c->db,c->argv[1]);
notifyKeyspaceEvent(NOTIFY_SET,"srem",c->argv[1],c->db->id);
if (keyremoved) {
@ -693,7 +695,7 @@ void smoveCommand(client *c) {
/* Create the destination set when it doesn't exist */
if (!dstset) {
dstset = setTypeCreate(ele->ptr, 1);
dbAdd(c->db,c->argv[2],dstset);
dbAdd(c->db, c->argv[2], &dstset);
}
signalModifiedKey(c,c->db,c->argv[1]);
@ -711,7 +713,7 @@ void smoveCommand(client *c) {
}
void sismemberCommand(client *c) {
robj *set;
kvobj *set;
if ((set = lookupKeyReadOrReply(c,c->argv[1],shared.czero)) == NULL ||
checkType(c,set,OBJ_SET)) return;
@ -723,17 +725,14 @@ void sismemberCommand(client *c) {
}
void smismemberCommand(client *c) {
robj *set;
int j;
/* Don't abort when the key cannot be found. Non-existing keys are empty
* sets, where SMISMEMBER should respond with a series of zeros. */
set = lookupKeyRead(c->db,c->argv[1]);
kvobj *set = lookupKeyRead(c->db, c->argv[1]);
if (set && checkType(c,set,OBJ_SET)) return;
addReplyArrayLen(c,c->argc - 2);
for (j = 2; j < c->argc; j++) {
for (int j = 2; j < c->argc; j++) {
if (set && setTypeIsMember(set,c->argv[j]->ptr))
addReply(c,shared.cone);
else
@ -742,12 +741,12 @@ void smismemberCommand(client *c) {
}
void scardCommand(client *c) {
robj *o;
kvobj *kv;
if ((o = lookupKeyReadOrReply(c,c->argv[1],shared.czero)) == NULL ||
checkType(c,o,OBJ_SET)) return;
if ((kv = lookupKeyReadOrReply(c,c->argv[1],shared.czero)) == NULL ||
checkType(c,kv,OBJ_SET)) return;
addReplyLongLong(c,setTypeSize(o));
addReplyLongLong(c,setTypeSize(kv));
}
/* Handle the "SPOP key <count>" variant. The normal version of the
@ -761,16 +760,15 @@ void scardCommand(client *c) {
void spopWithCountCommand(client *c) {
long l;
unsigned long count, size, toRemove;
robj *set;
/* Get the count argument */
if (getPositiveLongFromObjectOrReply(c,c->argv[2],&l,NULL) != C_OK) return;
count = (unsigned long) l;
/* Make sure a key with the name inputted exists, and that it's type is
* indeed a set. Otherwise, return nil */
if ((set = lookupKeyWriteOrReply(c,c->argv[1],shared.emptyset[c->resp]))
== NULL || checkType(c,set,OBJ_SET)) return;
* indeed a kv. Otherwise, return nil */
robj *set = lookupKeyWriteOrReply(c, c->argv[1], shared.emptyset[c->resp]);
if (set == NULL || checkType(c, set, OBJ_SET)) return;
/* If count is zero, serve an empty set ASAP to avoid special
* cases later. */
@ -785,7 +783,7 @@ void spopWithCountCommand(client *c) {
/* Generate an SPOP keyspace notification */
notifyKeyspaceEvent(NOTIFY_SET,"spop",c->argv[1],c->db->id);
server.dirty += toRemove;
/* CASE 1:
* The number of requested elements is greater than or equal to
* the number of elements inside the set: simply return the whole set. */
@ -947,7 +945,7 @@ void spopWithCountCommand(client *c) {
setTypeReleaseIterator(si);
/* Assign the new set as the key value (Also update KEYSIZES histogram) */
dbReplaceValue(c->db,c->argv[1],newset);
dbReplaceValue(c->db, c->argv[1], &newset);
}
/* Replicate/AOF the remaining elements as an SREM operation */
@ -970,7 +968,7 @@ void spopWithCountCommand(client *c) {
void spopCommand(client *c) {
unsigned long size;
robj *set, *ele;
robj *ele;
if (c->argc == 3) {
spopWithCountCommand(c);
@ -981,15 +979,15 @@ void spopCommand(client *c) {
}
/* Make sure a key with the name inputted exists, and that it's type is
* indeed a set */
if ((set = lookupKeyWriteOrReply(c,c->argv[1],shared.null[c->resp]))
== NULL || checkType(c,set,OBJ_SET)) return;
* indeed a kv */
kvobj *kv = lookupKeyWriteOrReply(c, c->argv[1], shared.null[c->resp]);
if (kv == NULL || checkType(c, kv, OBJ_SET)) return;
size = setTypeSize(set);
size = setTypeSize(kv);
updateKeysizesHist(c->db, getKeySlot(c->argv[1]->ptr), OBJ_SET, size, size-1);
/* Pop a random element from the set */
ele = setTypePopRandom(set);
/* Pop a random element from the kv */
ele = setTypePopRandom(kv);
notifyKeyspaceEvent(NOTIFY_SET,"spop",c->argv[1],c->db->id);
@ -1000,8 +998,8 @@ void spopCommand(client *c) {
addReplyBulk(c, ele);
decrRefCount(ele);
/* Delete the set if it's empty */
if (setTypeSize(set) == 0) {
/* Delete the kv if it's empty */
if (setTypeSize(kv) == 0) {
dbDelete(c->db,c->argv[1]);
notifyKeyspaceEvent(NOTIFY_GENERIC,"del",c->argv[1],c->db->id);
}
@ -1028,7 +1026,7 @@ void srandmemberWithCountCommand(client *c) {
long l;
unsigned long count, size;
int uniq = 1;
robj *set;
kvobj *set;
char *str;
size_t len = 0;
int64_t llele = 0;
@ -1230,7 +1228,7 @@ void srandmemberWithCountCommand(client *c) {
/* SRANDMEMBER <key> [<count>] */
void srandmemberCommand(client *c) {
robj *set;
kvobj *set;
char *str;
size_t len = 0;
int64_t llele = 0;
@ -1284,7 +1282,7 @@ int qsortCompareSetsByRevCardinality(const void *s1, const void *s2) {
void sinterGenericCommand(client *c, robj **setkeys,
unsigned long setnum, robj *dstkey,
int cardinality_only, unsigned long limit) {
robj **sets = zmalloc(sizeof(robj*)*setnum);
kvobj **sets = zmalloc(sizeof(robj*)*setnum);
setTypeIterator *si;
robj *dstset = NULL;
char *str;
@ -1295,18 +1293,18 @@ void sinterGenericCommand(client *c, robj **setkeys,
int encoding, empty = 0;
for (j = 0; j < setnum; j++) {
robj *setobj = lookupKeyRead(c->db, setkeys[j]);
if (!setobj) {
kvobj *kv = lookupKeyRead(c->db, setkeys[j]);
if (!kv) {
/* A NULL is considered an empty set */
empty += 1;
sets[j] = NULL;
continue;
}
if (checkType(c,setobj,OBJ_SET)) {
if (checkType(c, kv, OBJ_SET)) {
zfree(sets);
return;
}
sets[j] = setobj;
sets[j] = kv;
}
/* Set intersection with an empty set always results in an empty set.
@ -1424,7 +1422,7 @@ void sinterGenericCommand(client *c, robj **setkeys,
* frequent reallocs. Therefore, we shrink it now. */
dstset->ptr = lpShrinkToFit(dstset->ptr);
}
setKey(c,c->db,dstkey,dstset,0);
setKey(c, c->db, dstkey, &dstset, 0);
addReplyLongLong(c,setTypeSize(dstset));
notifyKeyspaceEvent(NOTIFY_SET,"sinterstore",
dstkey,c->db->id);
@ -1436,8 +1434,8 @@ void sinterGenericCommand(client *c, robj **setkeys,
signalModifiedKey(c,c->db,dstkey);
notifyKeyspaceEvent(NOTIFY_GENERIC,"del",dstkey,c->db->id);
}
decrRefCount(dstset);
}
decrRefCount(dstset);
} else {
setDeferredSetLen(c,replylen,cardinality);
}
@ -1455,7 +1453,7 @@ void smembersCommand(client *c) {
char *str;
size_t len = 0;
int64_t intobj = 0;
robj *setobj = lookupKeyRead(c->db, c->argv[1]);
kvobj *setobj = lookupKeyRead(c->db, c->argv[1]);
if (checkType(c,setobj,OBJ_SET)) return;
if (!setobj) {
addReply(c, shared.emptyset[c->resp]);
@ -1531,7 +1529,7 @@ void sunionDiffGenericCommand(client *c, robj **setkeys, int setnum,
int sameset = 0;
for (j = 0; j < setnum; j++) {
robj *setobj = lookupKeyRead(c->db, setkeys[j]);
kvobj *setobj = lookupKeyRead(c->db, setkeys[j]);
if (!setobj) {
sets[j] = NULL;
continue;
@ -1689,7 +1687,7 @@ void sunionDiffGenericCommand(client *c, robj **setkeys, int setnum,
/* If we have a target key where to store the resulting set
* create this key with the result set inside */
if (setTypeSize(dstset) > 0) {
setKey(c,c->db,dstkey,dstset,0);
setKey(c, c->db, dstkey, &dstset, 0);
addReplyLongLong(c,setTypeSize(dstset));
notifyKeyspaceEvent(NOTIFY_SET,
op == SET_OP_UNION ? "sunionstore" : "sdiffstore",
@ -1702,8 +1700,8 @@ void sunionDiffGenericCommand(client *c, robj **setkeys, int setnum,
signalModifiedKey(c,c->db,dstkey);
notifyKeyspaceEvent(NOTIFY_GENERIC,"del",dstkey,c->db->id);
}
decrRefCount(dstset);
}
decrRefCount(dstset);
}
zfree(sets);
}
@ -1729,7 +1727,7 @@ void sdiffstoreCommand(client *c) {
}
void sscanCommand(client *c) {
robj *set;
kvobj *set;
unsigned long long cursor;
if (parseScanCursorOrReply(c,c->argv[2],&cursor) == C_ERR) return;

99
src/t_stream.c
View File

@ -1855,17 +1855,18 @@ size_t streamReplyWithRangeFromConsumerPEL(client *c, stream *s, streamID *start
/* Look the stream at 'key' and return the corresponding stream object.
* The function creates a key setting it to an empty stream if needed. */
robj *streamTypeLookupWriteOrCreate(client *c, robj *key, int no_create) {
robj *o = lookupKeyWrite(c->db,key);
if (checkType(c,o,OBJ_STREAM)) return NULL;
if (o == NULL) {
if (no_create) {
addReplyNull(c);
return NULL;
}
o = createStreamObject();
dbAdd(c->db,key,o);
kvobj *streamTypeLookupWriteOrCreate(client *c, robj *key, int no_create) {
dictEntryLink link;
kvobj *kv = lookupKeyWriteWithLink(c->db,key, &link);
if (checkType(c, kv, OBJ_STREAM)) return NULL;
if (kv != NULL) return kv;
if (no_create) {
addReplyNull(c);
return NULL;
}
robj *o = createStreamObject();
dbAddByLink(c->db, key, &o, &link);
return o;
}
@ -2023,10 +2024,10 @@ void xaddCommand(client *c) {
}
/* Lookup the stream at key. */
robj *o;
kvobj *kv;
stream *s;
if ((o = streamTypeLookupWriteOrCreate(c,c->argv[1],parsed_args.no_mkstream)) == NULL) return;
s = o->ptr;
if ((kv = streamTypeLookupWriteOrCreate(c,c->argv[1],parsed_args.no_mkstream)) == NULL) return;
s = kv->ptr;
/* Return ASAP if the stream has reached the last possible ID */
if (s->last_id.ms == UINT64_MAX && s->last_id.seq == UINT64_MAX) {
@ -2096,7 +2097,7 @@ void xaddCommand(client *c) {
* will match anything from 1-1 and 1-UINT64_MAX.
*/
void xrangeGenericCommand(client *c, int rev) {
robj *o;
kvobj *kv;
stream *s;
streamID startid, endid;
long long count = -1;
@ -2135,10 +2136,10 @@ void xrangeGenericCommand(client *c, int rev) {
}
/* Return the specified range to the user. */
if ((o = lookupKeyReadOrReply(c,c->argv[1],shared.emptyarray)) == NULL ||
checkType(c,o,OBJ_STREAM)) return;
if ((kv = lookupKeyReadOrReply(c, c->argv[1], shared.emptyarray)) == NULL ||
checkType(c, kv, OBJ_STREAM)) return;
s = o->ptr;
s = kv->ptr;
if (count == 0) {
addReplyNullArray(c);
@ -2160,10 +2161,10 @@ void xrevrangeCommand(client *c) {
/* XLEN key*/
void xlenCommand(client *c) {
robj *o;
if ((o = lookupKeyReadOrReply(c,c->argv[1],shared.czero)) == NULL
|| checkType(c,o,OBJ_STREAM)) return;
stream *s = o->ptr;
kvobj *kv;
if ((kv = lookupKeyReadOrReply(c, c->argv[1], shared.czero)) == NULL
|| checkType(c, kv, OBJ_STREAM)) return;
stream *s = kv->ptr;
addReplyLongLong(c,s->length);
}
@ -2259,7 +2260,7 @@ void xreadCommand(client *c) {
* starting from now. */
int id_idx = i - streams_arg - streams_count;
robj *key = c->argv[i-streams_count];
robj *o = lookupKeyRead(c->db,key);
kvobj *o = lookupKeyRead(c->db, key);
if (checkType(c,o,OBJ_STREAM)) goto cleanup;
streamCG *group = NULL;
@ -2338,7 +2339,7 @@ void xreadCommand(client *c) {
size_t arraylen = 0;
void *arraylen_ptr = NULL;
for (int i = 0; i < streams_count; i++) {
robj *o = lookupKeyRead(c->db,c->argv[streams_arg+i]);
kvobj *o = lookupKeyRead(c->db, c->argv[streams_arg + i]);
if (o == NULL) continue;
stream *s = o->ptr;
streamID *gt = ids+i; /* ID must be greater than this. */
@ -2706,7 +2707,7 @@ NULL
if (s == NULL) {
serverAssert(mkstream);
o = createStreamObject();
dbAdd(c->db,c->argv[2],o);
dbAdd(c->db, c->argv[2], &o);
s = o->ptr;
signalModifiedKey(c,c->db,c->argv[2]);
}
@ -2804,9 +2805,9 @@ void xsetidCommand(client *c) {
}
}
robj *o = lookupKeyWriteOrReply(c,c->argv[1],shared.nokeyerr);
if (o == NULL || checkType(c,o,OBJ_STREAM)) return;
stream *s = o->ptr;
kvobj *kv = lookupKeyWriteOrReply(c, c->argv[1], shared.nokeyerr);
if (kv == NULL || checkType(c, kv, OBJ_STREAM)) return;
stream *s = kv->ptr;
if (streamCompareID(&id,&s->max_deleted_entry_id) < 0) {
addReplyError(c,"The ID specified in XSETID is smaller than current max_deleted_entry_id");
@ -2852,14 +2853,14 @@ void xsetidCommand(client *c) {
*/
void xackCommand(client *c) {
streamCG *group = NULL;
robj *o = lookupKeyRead(c->db,c->argv[1]);
if (o) {
if (checkType(c,o,OBJ_STREAM)) return; /* Type error. */
group = streamLookupCG(o->ptr,c->argv[2]->ptr);
kvobj *kv = lookupKeyRead(c->db, c->argv[1]);
if (kv) {
if (checkType(c, kv, OBJ_STREAM)) return; /* Type error. */
group = streamLookupCG(kv->ptr, c->argv[2]->ptr);
}
/* No key or group? Nothing to ack. */
if (o == NULL || group == NULL) {
if (kv == NULL || group == NULL) {
addReply(c,shared.czero);
return;
}
@ -2969,12 +2970,12 @@ void xpendingCommand(client *c) {
}
/* Lookup the key and the group inside the stream. */
robj *o = lookupKeyRead(c->db,c->argv[1]);
kvobj *kv = lookupKeyRead(c->db, c->argv[1]);
streamCG *group;
if (checkType(c,o,OBJ_STREAM)) return;
if (o == NULL ||
(group = streamLookupCG(o->ptr,groupname->ptr)) == NULL)
if (checkType(c, kv, OBJ_STREAM)) return;
if (kv == NULL ||
(group = streamLookupCG(kv->ptr, groupname->ptr)) == NULL)
{
addReplyErrorFormat(c, "-NOGROUP No such key '%s' or consumer "
"group '%s'",
@ -3152,7 +3153,7 @@ void xpendingCommand(client *c) {
* what messages it is now in charge of. */
void xclaimCommand(client *c) {
streamCG *group = NULL;
robj *o = lookupKeyRead(c->db,c->argv[1]);
kvobj *o = lookupKeyRead(c->db,c->argv[1]);
long long minidle; /* Minimum idle time argument. */
long long retrycount = -1; /* -1 means RETRYCOUNT option not given. */
mstime_t deliverytime = -1; /* -1 means IDLE/TIME options not given. */
@ -3373,7 +3374,7 @@ cleanup:
* what messages it is now in charge of. */
void xautoclaimCommand(client *c) {
streamCG *group = NULL;
robj *o = lookupKeyRead(c->db,c->argv[1]);
kvobj *o = lookupKeyRead(c->db,c->argv[1]);
long long minidle; /* Minimum idle time argument, in milliseconds. */
long count = 100; /* Maximum entries to claim. */
const unsigned attempts_factor = 10;
@ -3551,11 +3552,9 @@ void xautoclaimCommand(client *c) {
* of items actually deleted, that may be different from the number
* of IDs passed in case certain IDs do not exist. */
void xdelCommand(client *c) {
robj *o;
if ((o = lookupKeyWriteOrReply(c,c->argv[1],shared.czero)) == NULL
|| checkType(c,o,OBJ_STREAM)) return;
stream *s = o->ptr;
kvobj *kv = lookupKeyWriteOrReply(c, c->argv[1], shared.czero);
if (kv == NULL || checkType(c, kv, OBJ_STREAM)) return;
stream *s = kv->ptr;
/* We need to sanity check the IDs passed to start. Even if not
* a big issue, it is not great that the command is only partially
@ -3633,8 +3632,6 @@ cleanup:
* Has meaning only if `~` was provided.
*/
void xtrimCommand(client *c) {
robj *o;
/* Argument parsing. */
streamAddTrimArgs parsed_args;
if (streamParseAddOrTrimArgsOrReply(c, &parsed_args, 0) < 0)
@ -3642,9 +3639,9 @@ void xtrimCommand(client *c) {
/* If the key does not exist, we are ok returning zero, that is, the
* number of elements removed from the stream. */
if ((o = lookupKeyWriteOrReply(c,c->argv[1],shared.czero)) == NULL
|| checkType(c,o,OBJ_STREAM)) return;
stream *s = o->ptr;
kvobj *kv = lookupKeyWriteOrReply(c, c->argv[1], shared.czero);
if (kv == NULL || checkType(c, kv, OBJ_STREAM)) return;
stream *s = kv->ptr;
/* Perform the trimming. */
int64_t deleted = streamTrim(s, &parsed_args);
@ -3904,9 +3901,9 @@ NULL
key = c->argv[2];
/* Lookup the key now, this is common for all the subcommands but HELP. */
robj *o = lookupKeyReadOrReply(c,key,shared.nokeyerr);
if (o == NULL || checkType(c,o,OBJ_STREAM)) return;
s = o->ptr;
kvobj *kv = lookupKeyReadOrReply(c, key, shared.nokeyerr);
if (kv == NULL || checkType(c, kv, OBJ_STREAM)) return;
s = kv->ptr;
/* Dispatch the different subcommands. */
if (!strcasecmp(opt,"CONSUMERS") && c->argc == 4) {

130
src/t_string.c
View File

@ -61,7 +61,18 @@ static int checkStringLength(client *c, long long size, long long append) {
/* Forward declaration */
static int getExpireMillisecondsOrReply(client *c, robj *expire, int flags, int unit, long long *milliseconds);
void setGenericCommand(client *c, int flags, robj *key, robj *val, robj *expire, int unit, robj *ok_reply, robj *abort_reply) {
/* Generic SET command family (SET, SETEX, PSETEX, SETNX)
*
* Arguments:
* valref: A pointer to the robj to be set. This argument may be updated by the function.
* The object is expected to have a refcount of 1, allowing its ownership to be
* transferred directly to the database to avoid making a copy. If needed, the
* function will replace *valref with a new allocation and increment its refcount
* so that both the database and the caller maintain valid references.
*/
void setGenericCommand(client *c, int flags, robj *key, robj **valref, robj *expire,
int unit, robj *ok_reply, robj *abort_reply)
{
long long milliseconds = 0; /* initialized to avoid any harmless warning */
int found = 0;
int setkey_flags = 0;
@ -74,8 +85,8 @@ void setGenericCommand(client *c, int flags, robj *key, robj *val, robj *expire,
if (getGenericCommand(c) == C_ERR) return;
}
dictEntry *de = NULL;
found = (lookupKeyWriteWithDictEntry(c->db,key,&de) != NULL);
dictEntryLink link = NULL;
found = (lookupKeyWriteWithLink(c->db,key,&link) != NULL);
if ((flags & OBJ_SET_NX && found) ||
(flags & OBJ_SET_XX && !found))
@ -90,17 +101,24 @@ void setGenericCommand(client *c, int flags, robj *key, robj *val, robj *expire,
setkey_flags |= ((flags & OBJ_KEEPTTL) || expire) ? SETKEY_KEEPTTL : 0;
setkey_flags |= found ? SETKEY_ALREADY_EXIST : SETKEY_DOESNT_EXIST;
setKeyWithDictEntry(c,c->db,key,val,setkey_flags,de);
setKeyByLink(c, c->db, key, valref, setkey_flags, &link);
/* If there's an expiration, setExpireByLink may reallocate the object.
* We must update valref to reflect the new object if that happens. */
if (expire) *valref = setExpireByLink(c, c->db, key->ptr, milliseconds, link);
/* The client still holds a reference to the original object via c->argv[i],
* and will call decrRefCount() at the end of call(). We increment the refcount
* from 1 to 2 to ensure both DB and client have valid references. */
incrRefCount(*valref); /* 1->2 */
server.dirty++;
notifyKeyspaceEvent(NOTIFY_STRING,"set",key,c->db->id);
if (expire) {
setExpireWithDictEntry(c,c->db,key,milliseconds,de);
/* Propagate as SET Key Value PXAT millisecond-timestamp if there is
* EX/PX/EXAT flag. */
if (!(flags & OBJ_PXAT)) {
robj *milliseconds_obj = createStringObjectFromLongLong(milliseconds);
rewriteClientCommandVector(c, 5, shared.set, key, val, shared.pxat, milliseconds_obj);
rewriteClientCommandVector(c, 5, shared.set, key, *valref, shared.pxat, milliseconds_obj);
decrRefCount(milliseconds_obj);
}
notifyKeyspaceEvent(NOTIFY_GENERIC,"expire",key,c->db->id);
@ -283,28 +301,28 @@ void setCommand(client *c) {
}
c->argv[2] = tryObjectEncoding(c->argv[2]);
setGenericCommand(c,flags,c->argv[1],c->argv[2],expire,unit,NULL,NULL);
setGenericCommand(c,flags,c->argv[1],&(c->argv[2]),expire,unit,NULL,NULL);
}
void setnxCommand(client *c) {
c->argv[2] = tryObjectEncoding(c->argv[2]);
setGenericCommand(c,OBJ_SET_NX,c->argv[1],c->argv[2],NULL,0,shared.cone,shared.czero);
setGenericCommand(c, OBJ_SET_NX, c->argv[1], &(c->argv[2]), NULL, 0, shared.cone, shared.czero);
}
void setexCommand(client *c) {
c->argv[3] = tryObjectEncoding(c->argv[3]);
setGenericCommand(c,OBJ_EX,c->argv[1],c->argv[3],c->argv[2],UNIT_SECONDS,NULL,NULL);
setGenericCommand(c, OBJ_EX, c->argv[1], &(c->argv[3]), c->argv[2], UNIT_SECONDS, NULL, NULL);
}
void psetexCommand(client *c) {
c->argv[3] = tryObjectEncoding(c->argv[3]);
setGenericCommand(c,OBJ_PX,c->argv[1],c->argv[3],c->argv[2],UNIT_MILLISECONDS,NULL,NULL);
setGenericCommand(c, OBJ_PX, c->argv[1], &(c->argv[3]), c->argv[2], UNIT_MILLISECONDS, NULL, NULL);
}
int getGenericCommand(client *c) {
robj *o;
kvobj *o;
if ((o = lookupKeyReadOrReply(c,c->argv[1],shared.null[c->resp])) == NULL)
if ((o = lookupKeyReadOrReply(c, c->argv[1], shared.null[c->resp])) == NULL)
return C_OK;
if (checkType(c,o,OBJ_STRING)) {
@ -348,7 +366,7 @@ void getexCommand(client *c) {
return;
}
robj *o;
kvobj *o;
if ((o = lookupKeyReadOrReply(c,c->argv[1],shared.null[c->resp])) == NULL)
return;
@ -412,7 +430,8 @@ void getdelCommand(client *c) {
void getsetCommand(client *c) {
if (getGenericCommand(c) == C_ERR) return;
c->argv[2] = tryObjectEncoding(c->argv[2]);
setKey(c,c->db,c->argv[1],c->argv[2],0);
setKey(c, c->db, c->argv[1], &c->argv[2], 0);
incrRefCount(c->argv[2]);
notifyKeyspaceEvent(NOTIFY_STRING,"set",c->argv[1],c->db->id);
server.dirty++;
@ -422,7 +441,6 @@ void getsetCommand(client *c) {
void setrangeCommand(client *c) {
int64_t oldLen = -1, newLen;
robj *o;
long offset;
sds value = c->argv[3]->ptr;
const size_t value_len = sdslen(value);
@ -435,9 +453,9 @@ void setrangeCommand(client *c) {
return;
}
dictEntry *de;
o = lookupKeyWriteWithDictEntry(c->db,c->argv[1],&de);
if (o == NULL) {
dictEntryLink link;
kvobj *kv = lookupKeyWriteWithLink(c->db, c->argv[1], &link);
if (kv == NULL) {
/* Return 0 when setting nothing on a non-existing string */
if (value_len == 0) {
addReply(c,shared.czero);
@ -448,15 +466,15 @@ void setrangeCommand(client *c) {
if (checkStringLength(c,offset,value_len) != C_OK)
return;
o = createObject(OBJ_STRING,sdsnewlen(NULL, offset+value_len));
dbAdd(c->db,c->argv[1],o);
robj *o = createObject(OBJ_STRING,sdsnewlen(NULL, offset+value_len));
kv = dbAddByLink(c->db, c->argv[1], &o, &link);
} else {
/* Key exists, check type */
if (checkType(c,o,OBJ_STRING))
if (checkType(c,kv,OBJ_STRING))
return;
/* Return existing string length when setting nothing */
oldLen = stringObjectLen(o);
oldLen = stringObjectLen(kv);
if (value_len == 0) {
addReplyLongLong(c, oldLen);
return;
@ -467,25 +485,25 @@ void setrangeCommand(client *c) {
return;
/* Create a copy when the object is shared or encoded. */
o = dbUnshareStringValueWithDictEntry(c->db,c->argv[1],o,de);
kv = dbUnshareStringValueByLink(c->db, c->argv[1], kv, link);
}
if (value_len > 0) {
o->ptr = sdsgrowzero(o->ptr,offset+value_len);
memcpy((char*)o->ptr+offset,value,value_len);
kv->ptr = sdsgrowzero(kv->ptr,offset+value_len);
memcpy((char*)kv->ptr+offset,value,value_len);
signalModifiedKey(c,c->db,c->argv[1]);
notifyKeyspaceEvent(NOTIFY_STRING,
"setrange",c->argv[1],c->db->id);
server.dirty++;
}
newLen = sdslen(o->ptr);
newLen = sdslen(kv->ptr);
updateKeysizesHist(c->db,getKeySlot(c->argv[1]->ptr),OBJ_STRING,oldLen,newLen);
addReplyLongLong(c,newLen);
}
void getrangeCommand(client *c) {
robj *o;
kvobj *o;
long long start, end;
char *str, llbuf[32];
size_t strlen;
@ -530,7 +548,7 @@ void mgetCommand(client *c) {
addReplyArrayLen(c,c->argc-1);
for (j = 1; j < c->argc; j++) {
robj *o = lookupKeyRead(c->db,c->argv[j]);
kvobj *o = lookupKeyRead(c->db, c->argv[j]);
if (o == NULL) {
addReplyNull(c);
} else {
@ -562,14 +580,12 @@ void msetGenericCommand(client *c, int nx) {
}
}
int setkey_flags = nx ? SETKEY_DOESNT_EXIST : 0;
for (j = 1; j < c->argc; j += 2) {
c->argv[j+1] = tryObjectEncoding(c->argv[j+1]);
setKey(c, c->db, c->argv[j], c->argv[j + 1], setkey_flags);
/* if 'NX', no need set flags SETKEY_DOESNT_EXIST. Already verified earlier! */
setKey(c, c->db, c->argv[j], &(c->argv[j+1]) , 0 /*flags*/);
incrRefCount(c->argv[j+1]); /* refcnt not incr by setKey() */
notifyKeyspaceEvent(NOTIFY_STRING,"set",c->argv[j],c->db->id);
/* In MSETNX, It could be that we're overriding the same key, we can't be sure it doesn't exist. */
if (nx)
setkey_flags = SETKEY_ADD_OR_UPDATE;
}
server.dirty += (c->argc-1)/2;
addReply(c, nx ? shared.cone : shared.ok);
@ -585,9 +601,9 @@ void msetnxCommand(client *c) {
void incrDecrCommand(client *c, long long incr) {
long long value, oldvalue;
robj *o, *new;
dictEntry *de;
o = lookupKeyWriteWithDictEntry(c->db,c->argv[1],&de);
robj *new;
dictEntryLink link;
kvobj *o = lookupKeyWriteWithLink(c->db, c->argv[1], &link);
if (checkType(c,o,OBJ_STRING)) return;
if (getLongLongFromObjectOrReply(c,o,&value,NULL) != C_OK) return;
@ -600,7 +616,6 @@ void incrDecrCommand(client *c, long long incr) {
value += incr;
if (o && o->refcount == 1 && o->encoding == OBJ_ENCODING_INT &&
(value < 0 || value >= OBJ_SHARED_INTEGERS) &&
value >= LONG_MIN && value <= LONG_MAX)
{
new = o;
@ -613,10 +628,10 @@ void incrDecrCommand(client *c, long long incr) {
new = createStringObjectFromLongLongForValue(value);
if (o) {
/* replace value in db and also update keysizes hist */
dbReplaceValueWithDictEntry(c->db,c->argv[1],new,de);
dbReplaceValueWithLink(c->db, c->argv[1], &new, link);
} else {
/* Add new key to db and also update keysizes hist */
dbAdd(c->db,c->argv[1],new);
dbAddByLink(c->db, c->argv[1], &new, &link);
}
}
addReplyLongLongFromStr(c,new);
@ -654,10 +669,9 @@ void decrbyCommand(client *c) {
void incrbyfloatCommand(client *c) {
long double incr, value;
robj *o, *new;
dictEntry *de;
o = lookupKeyWriteWithDictEntry(c->db,c->argv[1],&de);
dictEntryLink link;
kvobj *o = lookupKeyWriteWithLink(c->db,c->argv[1],&link);
if (checkType(c,o,OBJ_STRING)) return;
if (getLongDoubleFromObjectOrReply(c,o,&value,NULL) != C_OK ||
getLongDoubleFromObjectOrReply(c,c->argv[2],&incr,NULL) != C_OK)
@ -668,11 +682,11 @@ void incrbyfloatCommand(client *c) {
addReplyError(c,"increment would produce NaN or Infinity");
return;
}
new = createStringObjectFromLongDouble(value,1);
robj *new = createStringObjectFromLongDouble(value,1);
if (o)
dbReplaceValueWithDictEntry(c->db,c->argv[1],new,de);
dbReplaceValueWithLink(c->db, c->argv[1], &new, link);
else
dbAdd(c->db,c->argv[1],new);
dbAddByLink(c->db, c->argv[1], &new, &link);
signalModifiedKey(c,c->db,c->argv[1]);
notifyKeyspaceEvent(NOTIFY_STRING,"incrbyfloat",c->argv[1],c->db->id);
server.dirty++;
@ -688,14 +702,15 @@ void incrbyfloatCommand(client *c) {
void appendCommand(client *c) {
size_t totlen;
robj *o, *append;
robj *append;
kvobj *o;
dictEntry *de;
o = lookupKeyWriteWithDictEntry(c->db,c->argv[1],&de);
dictEntryLink link;
o = lookupKeyWriteWithLink(c->db,c->argv[1],&link);
if (o == NULL) {
/* Create the key */
c->argv[2] = tryObjectEncoding(c->argv[2]);
dbAdd(c->db,c->argv[1],c->argv[2]);
dbAddByLink(c->db, c->argv[1], &c->argv[2], &link);
incrRefCount(c->argv[2]);
totlen = stringObjectLen(c->argv[2]);
} else {
@ -710,7 +725,7 @@ void appendCommand(client *c) {
return;
/* Append the value */
o = dbUnshareStringValueWithDictEntry(c->db,c->argv[1],o,de);
o = dbUnshareStringValueByLink(c->db,c->argv[1],o,link);
o->ptr = sdscatlen(o->ptr,append->ptr,append_len);
totlen = sdslen(o->ptr);
int64_t oldlen = totlen - append_len;
@ -719,15 +734,15 @@ void appendCommand(client *c) {
signalModifiedKey(c,c->db,c->argv[1]);
notifyKeyspaceEvent(NOTIFY_STRING,"append",c->argv[1],c->db->id);
server.dirty++;
addReplyLongLong(c,totlen);
}
void strlenCommand(client *c) {
robj *o;
if ((o = lookupKeyReadOrReply(c,c->argv[1],shared.czero)) == NULL ||
checkType(c,o,OBJ_STRING)) return;
addReplyLongLong(c,stringObjectLen(o));
kvobj *kv;
if ((kv = lookupKeyReadOrReply(c, c->argv[1], shared.czero)) == NULL ||
checkType(c, kv, OBJ_STRING)) return;
addReplyLongLong(c,stringObjectLen(kv));
}
/* LCS key1 key2 [LEN] [IDX] [MINMATCHLEN <len>] [WITHMATCHLEN] */
@ -736,10 +751,9 @@ void lcsCommand(client *c) {
long long minmatchlen = 0;
sds a = NULL, b = NULL;
int getlen = 0, getidx = 0, withmatchlen = 0;
robj *obja = NULL, *objb = NULL;
obja = lookupKeyRead(c->db,c->argv[1]);
objb = lookupKeyRead(c->db,c->argv[2]);
kvobj *obja = lookupKeyRead(c->db, c->argv[1]);
kvobj *objb = lookupKeyRead(c->db, c->argv[2]);
if ((obja && obja->type != OBJ_STRING) ||
(objb && objb->type != OBJ_STRING))
{

48
src/t_zset.c
View File

@ -1838,8 +1838,8 @@ void zaddGenericCommand(client *c, int flags) {
if (checkType(c,zobj,OBJ_ZSET)) goto cleanup;
if (zobj == NULL) {
if (xx) goto reply_to_client; /* No key + XX option: nothing to do. */
zobj = zsetTypeCreate(elements, sdslen(c->argv[scoreidx+1]->ptr));
dbAdd(c->db,key,zobj);
robj *o = zsetTypeCreate(elements, sdslen(c->argv[scoreidx + 1]->ptr));
zobj = dbAdd(c->db,key,&o);
} else {
zsetTypeMaybeConvert(zobj, elements);
}
@ -1893,15 +1893,14 @@ void zincrbyCommand(client *c) {
void zremCommand(client *c) {
robj *key = c->argv[1];
robj *zobj;
int deleted = 0, keyremoved = 0, j;
if ((zobj = lookupKeyWriteOrReply(c,key,shared.czero)) == NULL ||
checkType(c,zobj,OBJ_ZSET)) return;
kvobj *zobj = lookupKeyWriteOrReply(c, key, shared.czero);
if (zobj == NULL || checkType(c,zobj,OBJ_ZSET)) return;
int64_t oldlen = (int64_t) zsetLength(zobj);
for (j = 2; j < c->argc; j++) {
if (zsetDel(zobj,c->argv[j]->ptr)) deleted++;
if (zsetDel(zobj, c->argv[j]->ptr)) deleted++;
if (zsetLength(zobj) == 0) {
/* Del key but don't update KEYSIZES. Else it will decr wrong bin in histogram */
dbDeleteSkipKeysizesUpdate(c->db, key);
@ -1935,7 +1934,6 @@ typedef enum {
/* Implements ZREMRANGEBYRANK, ZREMRANGEBYSCORE, ZREMRANGEBYLEX commands. */
void zremrangeGenericCommand(client *c, zrange_type rangetype) {
robj *key = c->argv[1];
robj *zobj;
int keyremoved = 0;
unsigned long deleted = 0;
zrangespec range;
@ -1966,8 +1964,8 @@ void zremrangeGenericCommand(client *c, zrange_type rangetype) {
}
/* Step 2: Lookup & range sanity checks if needed. */
if ((zobj = lookupKeyWriteOrReply(c,key,shared.czero)) == NULL ||
checkType(c,zobj,OBJ_ZSET)) goto cleanup;
kvobj *zobj = lookupKeyWriteOrReply(c, key, shared.czero);
if (zobj == NULL || checkType(c, zobj, OBJ_ZSET)) goto cleanup;
if (rangetype == ZRANGE_RANK) {
/* Sanitize indexes. */
@ -2666,7 +2664,7 @@ void zunionInterDiffGenericCommand(client *c, robj *dstkey, int numkeysIndex, in
/* read keys to be used for input */
for (i = 0, j = numkeysIndex+1; i < setnum; i++, j++) {
robj *obj = lookupKeyRead(c->db, c->argv[j]);
kvobj *obj = lookupKeyRead(c->db, c->argv[j]);
if (obj != NULL) {
if (obj->type != OBJ_ZSET && obj->type != OBJ_SET) {
zfree(src);
@ -2875,7 +2873,7 @@ void zunionInterDiffGenericCommand(client *c, robj *dstkey, int numkeysIndex, in
if (dstkey) {
if (dstzset->zsl->length) {
zsetConvertToListpackIfNeeded(dstobj, maxelelen, totelelen);
setKey(c, c->db, dstkey, dstobj, 0);
setKey(c, c->db, dstkey, &dstobj, 0);
addReplyLongLong(c, zsetLength(dstobj));
notifyKeyspaceEvent(NOTIFY_ZSET,
(op == SET_OP_UNION) ? "zunionstore" :
@ -2889,8 +2887,8 @@ void zunionInterDiffGenericCommand(client *c, robj *dstkey, int numkeysIndex, in
notifyKeyspaceEvent(NOTIFY_GENERIC, "del", dstkey, c->db->id);
server.dirty++;
}
decrRefCount(dstobj);
}
decrRefCount(dstobj);
} else if (cardinality_only) {
addReplyLongLong(c, cardinality);
} else {
@ -3087,7 +3085,7 @@ static void zrangeResultEmitLongLongForStore(zrange_result_handler *handler,
static void zrangeResultFinalizeStore(zrange_result_handler *handler, size_t result_count)
{
if (result_count) {
setKey(handler->client, handler->client->db, handler->dstkey, handler->dstobj, 0);
setKey(handler->client, handler->client->db, handler->dstkey, &handler->dstobj, 0);
addReplyLongLong(handler->client, result_count);
notifyKeyspaceEvent(NOTIFY_ZSET, "zrangestore", handler->dstkey, handler->client->db->id);
server.dirty++;
@ -3098,8 +3096,8 @@ static void zrangeResultFinalizeStore(zrange_result_handler *handler, size_t res
notifyKeyspaceEvent(NOTIFY_GENERIC, "del", handler->dstkey, handler->client->db->id);
server.dirty++;
}
decrRefCount(handler->dstobj);
}
decrRefCount(handler->dstobj);
}
/* Initialize the consumer interface type with the requested type. */
@ -3373,7 +3371,7 @@ void zrevrangebyscoreCommand(client *c) {
void zcountCommand(client *c) {
robj *key = c->argv[1];
robj *zobj;
kvobj *zobj;
zrangespec range;
unsigned long count = 0;
@ -3450,7 +3448,7 @@ void zcountCommand(client *c) {
void zlexcountCommand(client *c) {
robj *key = c->argv[1];
robj *zobj;
kvobj *zobj;
zlexrangespec range;
unsigned long count = 0;
@ -3657,7 +3655,6 @@ void zrangeGenericCommand(zrange_result_handler *handler, int argc_start, int st
{
client *c = handler->client;
robj *key = c->argv[argc_start];
robj *zobj;
zrangespec range;
zlexrangespec lexrange;
int minidx = argc_start + 1;
@ -3759,7 +3756,7 @@ void zrangeGenericCommand(zrange_result_handler *handler, int argc_start, int st
}
/* Step 3: Lookup the key and get the range. */
zobj = lookupKeyRead(c->db, key);
kvobj *zobj = lookupKeyRead(c->db, key);
if (zobj == NULL) {
if (store) {
handler->beginResultEmission(handler, -1);
@ -3802,7 +3799,7 @@ cleanup:
void zcardCommand(client *c) {
robj *key = c->argv[1];
robj *zobj;
kvobj *zobj;
if ((zobj = lookupKeyReadOrReply(c,key,shared.czero)) == NULL ||
checkType(c,zobj,OBJ_ZSET)) return;
@ -3812,7 +3809,7 @@ void zcardCommand(client *c) {
void zscoreCommand(client *c) {
robj *key = c->argv[1];
robj *zobj;
kvobj *zobj;
double score;
if ((zobj = lookupKeyReadOrReply(c,key,shared.null[c->resp])) == NULL ||
@ -3827,9 +3824,8 @@ void zscoreCommand(client *c) {
void zmscoreCommand(client *c) {
robj *key = c->argv[1];
robj *zobj;
double score;
zobj = lookupKeyRead(c->db,key);
kvobj *zobj = lookupKeyRead(c->db, key);
if (checkType(c,zobj,OBJ_ZSET)) return;
addReplyArrayLen(c,c->argc - 2);
@ -3846,7 +3842,7 @@ void zmscoreCommand(client *c) {
void zrankGenericCommand(client *c, int reverse) {
robj *key = c->argv[1];
robj *ele = c->argv[2];
robj *zobj;
kvobj *zobj;
robj* reply;
long rank;
int opt_withscore = 0;
@ -3896,7 +3892,7 @@ void zrevrankCommand(client *c) {
}
void zscanCommand(client *c) {
robj *o;
kvobj *o;
unsigned long long cursor;
if (parseScanCursorOrReply(c,c->argv[2],&cursor) == C_ERR) return;
@ -4201,7 +4197,7 @@ static void zrandmemberReplyWithListpack(client *c, unsigned int count, listpack
void zrandmemberWithCountCommand(client *c, long l, int withscores) {
unsigned long count, size;
int uniq = 1;
robj *zsetobj;
kvobj *zsetobj;
if ((zsetobj = lookupKeyReadOrReply(c, c->argv[1], shared.emptyarray))
== NULL || checkType(c, zsetobj, OBJ_ZSET)) return;
@ -4406,7 +4402,7 @@ void zrandmemberWithCountCommand(client *c, long l, int withscores) {
void zrandmemberCommand(client *c) {
long l;
int withscores = 0;
robj *zset;
kvobj *zset;
listpackEntry ele;
if (c->argc >= 3) {

15
tests/integration/replication-rdbchannel.tcl
View File

@ -296,12 +296,15 @@ start_server {tags {"repl external:skip"}} {
puts "peak_master_slave_buf_size $peak_master_slave_buf_size"
puts "peak_replica_buf_size $peak_replica_buf_size"
}
# memory on the master is less than 1mb
assert_lessthan [expr $peak_master_used_mem - $prev_used - $backlog_size] 1000000
assert_lessthan $peak_master_rpl_buf [expr {$backlog_size + 1000000}]
assert_lessthan $peak_master_slave_buf_size 1000000
# buffers in the replica are more than 10mb
assert_morethan $peak_replica_buf_size 10000000
# Valgrind contribute to flakiness of this test
if {!$::valgrind} {
# memory on the master is less than 3mb (TODO: Fine tune thresholds)
assert_lessthan [expr $peak_master_used_mem - $prev_used - $backlog_size] 3000000
assert_lessthan $peak_master_rpl_buf [expr {$backlog_size + 1000000}]
assert_lessthan $peak_master_slave_buf_size 1000000
# buffers in the replica are more than 10mb
assert_morethan $peak_replica_buf_size 10000000
}
stop_write_load $load_handle
}

48
tests/unit/info.tcl
View File

@ -504,35 +504,45 @@ start_server {tags {"info" "external:skip"}} {
start_server {tags {"info" "external:skip"}} {
test {memory: database and pubsub overhead and rehashing dict count} {
r flushall
# Better not set ht0_size to 4 since there is a probability that all
# keys will end up in the same bucket and rehashing will ended instantly.
set ht0_size [expr 1 << 3]
# ht1 size is twice the size of ht0
set ht1_size [expr $ht0_size << 1]
populate [expr $ht0_size - 1]
# Verify rehashing is not ongoing
wait_for_condition 100 10 {
[dict get [r memory stats] db.dict.rehashing.count] == 0
} else {
fail "Rehashing did not finish in time"
}
# Verify the info reflects steady state
set info_mem [r info memory]
set mem_stats [r memory stats]
assert_equal [getInfoProperty $info_mem mem_overhead_db_hashtable_rehashing] {0}
assert_equal [dict get $mem_stats overhead.db.hashtable.lut] {0}
assert_equal [dict get $mem_stats overhead.db.hashtable.lut] [expr $ht0_size * 8]
assert_equal [dict get $mem_stats overhead.db.hashtable.rehashing] {0}
assert_equal [dict get $mem_stats db.dict.rehashing.count] {0}
# Initial dict expand is not rehashing
r set a b
set info_mem [r info memory]
set mem_stats [r memory stats]
assert_equal [getInfoProperty $info_mem mem_overhead_db_hashtable_rehashing] {0}
assert_range [dict get $mem_stats overhead.db.hashtable.lut] 1 64
assert_equal [dict get $mem_stats overhead.db.hashtable.rehashing] {0}
assert_equal [dict get $mem_stats db.dict.rehashing.count] {0}
# set 4 more keys to trigger rehashing
# Set 2 more keys to trigger rehashing
# get the info within a transaction to make sure the rehashing is not completed
r multi
r set b c
r set c d
r set d e
r set e f
r set this_will_reach_max_load_factor 1
r set this_must_be_rehashed 1
r info memory
r memory stats
set res [r exec]
set info_mem [lindex $res 4]
set mem_stats [lindex $res 5]
assert_range [getInfoProperty $info_mem mem_overhead_db_hashtable_rehashing] 1 64
assert_range [dict get $mem_stats overhead.db.hashtable.lut] 1 192
assert_range [dict get $mem_stats overhead.db.hashtable.rehashing] 1 64
set info_mem [lindex $res 2]
set mem_stats [lindex $res 3]
# Verify the info reflects rehashing state
assert_range [getInfoProperty $info_mem mem_overhead_db_hashtable_rehashing] 1 [expr $ht0_size * 8]
assert_equal [dict get $mem_stats overhead.db.hashtable.lut] [expr ($ht0_size + $ht1_size) * 8 ]
assert_equal [dict get $mem_stats overhead.db.hashtable.rehashing] [expr $ht0_size * 8]
assert_equal [dict get $mem_stats db.dict.rehashing.count] {1}
}
}

23
tests/unit/maxmemory.tcl
View File

@ -148,29 +148,6 @@ start_server {tags {"maxmemory" "external:skip"}} {
}
start_server {tags {"maxmemory external:skip"}} {
test "Without maxmemory small integers are shared" {
r config set maxmemory 0
r set a 1
assert_refcount_morethan a 1
}
test "With maxmemory and non-LRU policy integers are still shared" {
r config set maxmemory 1073741824
r config set maxmemory-policy allkeys-random
r set a 1
assert_refcount_morethan a 1
}
test "With maxmemory and LRU policy integers are not shared" {
r config set maxmemory 1073741824
r config set maxmemory-policy allkeys-lru
r set a 1
r config set maxmemory-policy volatile-lru
r set b 1
assert_refcount 1 a
assert_refcount 1 b
r config set maxmemory 0
}
foreach policy {
allkeys-random allkeys-lru allkeys-lfu volatile-lru volatile-lfu volatile-random volatile-ttl

4
tests/unit/memefficiency.tcl
View File

@ -327,7 +327,7 @@ run_solo {defrag} {
r set "{bigstream}smallitem" val
set expected_frag 1.5
set expected_frag 1.49
if {$::accurate} {
# scale the hash to 1m fields in order to have a measurable the latency
for {set j 10000} {$j < 1000000} {incr j} {
@ -683,7 +683,7 @@ run_solo {defrag} {
puts "frag [s allocator_frag_ratio]"
puts "frag_bytes [s allocator_frag_bytes]"
}
assert_morethan [s allocator_frag_ratio] 1.4
assert_morethan [s allocator_frag_ratio] 1.35
catch {r config set activedefrag yes} e
if {[r config get activedefrag] eq "activedefrag yes"} {

30
tests/unit/type/hash-field-expire.tcl
View File

@ -1446,6 +1446,36 @@ start_server {tags {"external:skip needs:debug"}} {
r config set hash-max-listpack-value 64
}
test {Test HEXPIRE coexists with EXPIRE} {
# Verify HEXPIRE & EXPIRE coexists. When setting EXPIRE a new kvobj might be
# created whereas the old one can be ref by hash field expiration DS.
# Take care to set hexpire before expire. Verify all combinations of
# which expired first.
# Another point to verify is that whether hexpire deletes the last field
# and in turn the key (See f2).
foreach etime {10 1000} htime {10 1000} f2 {0 1} {
r del myhash
r hset myhash f1 v1
if {$f2} { r hset myhash f2 v2 }
r hpexpire myhash $etime FIELDS 1 f1
r pexpire myhash $htime
after 20
# If EXPIRE is shorter, it should delete the key.
if {$etime == 10} {
assert_equal [r httl myhash FIELDS 1 f1] $T_NO_FIELD
assert_equal [r exists myhash] 0
} else {
if {$htime == 10} {
assert_equal [r httl myhash FIELDS 1 f1] $T_NO_FIELD
assert_range [r pttl myhash] 500 1000
} else {
assert_range [r httl myhash FIELDS 1 f1] 1 1000
assert_range [r pttl myhash] 500 1000
}
}
}
}
}
start_server {tags {"external:skip needs:debug"}} {

6
tests/unit/type/incr.tcl
View File

@ -75,13 +75,13 @@ start_server {tags {"incr"}} {
assert_equal {-1} [r decrby key_not_exist 1]
}
test {INCR uses shared objects in the 0-9999 range} {
test {INCR does not use shared objects} {
r set foo -1
r incr foo
assert_refcount_morethan foo 1
assert_refcount 1 foo
r set foo 9998
r incr foo
assert_refcount_morethan foo 1
assert_refcount 1 foo
r incr foo
assert_refcount 1 foo
}

33
tests/unit/type/string.tcl
View File

@ -671,4 +671,37 @@ if {[string match {*jemalloc*} [s mem_allocator]]} {
assert_encoding "int" bar
lappend res [r get bar]
} {12 12}
if {[string match {*jemalloc*} [s mem_allocator]]} {
test {Check MEMORY USAGE for embedded key strings with jemalloc} {
proc expected_mem {key val with_expire exp_mem_usage exp_debug_sdslen} {
r del $key
r set $key $val
if {$with_expire} { r expire $key 5678315 }
assert_equal $exp_mem_usage [r memory usage $key]
assert_equal $exp_debug_sdslen [r debug sdslen $key]
}
if {[s arch_bits] == 64} {
# 16 (kvobj) + 1 (key-hdr-size) + 1 (sdshdr5) + 4 (key) + 1 (\0) + 3 (sdshdr8) + 5 (val) + 1 (\0) = 32bytes
expected_mem x234 y2345 0 32 "key_sds_len:4, key_sds_avail:0, key_zmalloc: 32, val_sds_len:5, val_sds_avail:0, val_zmalloc: 0"
# 16 (kvobj) + 1 (key-hdr-size) + 1 (sdshdr5) + 4 (key) + 1 (\0) + 3 (sdshdr8) + 6 (val) + 1 (\0) = 33bytes
expected_mem x234 y23456 0 40 "key_sds_len:4, key_sds_avail:0, key_zmalloc: 40, val_sds_len:6, val_sds_avail:7, val_zmalloc: 0"
# 16 (kvobj) + 1 (key-hdr-size) + 1 (sdshdr5) + 4 (key) + 1 (\0) + 3 (sdshdr8) + 13 (val) + 1 (\0) = 40bytes
expected_mem x234 y234561234567 0 40 "key_sds_len:4, key_sds_avail:0, key_zmalloc: 40, val_sds_len:13, val_sds_avail:0, val_zmalloc: 0"
# 16 (kvobj) + 8 (expiry) + 1 (key-hdr-size) + 1 (sdshdr5) + 4 (key) + 1 (\0) + 3 (sdshdr8) + 13 (val) + 1 (\0) = 48bytes
expected_mem x234 y234561234567 1 48 "key_sds_len:4, key_sds_avail:0, key_zmalloc: 48, val_sds_len:13, val_sds_avail:0, val_zmalloc: 0"
} else {
# 12 (kvobj) + 1 (key-hdr-size) + 1 (sdshdr5) + 4 (key) + 1 (\0) + 3 (sdshdr8) + 9 (val) + 1 (\0) = 32bytes
expected_mem x234 y23456789 0 32 "key_sds_len:4, key_sds_avail:0, key_zmalloc: 32, val_sds_len:9, val_sds_avail:0, val_zmalloc: 0"
# 12 (kvobj) + 1 (key-hdr-size) + 1 (sdshdr5) + 4 (key) + 1 (\0) + 3 (sdshdr8) + 10 (val) + 1 (\0) = 33bytes
expected_mem x234 y234567890 0 40 "key_sds_len:4, key_sds_avail:0, key_zmalloc: 40, val_sds_len:10, val_sds_avail:7, val_zmalloc: 0"
# 12 (kvobj) + 1 (key-hdr-size) + 1 (sdshdr5) + 4 (key) + 1 (\0) + 3 (sdshdr8) + 17 (val) + 1 (\0) = 40bytes
expected_mem x234 y2345678901234567 0 40 "key_sds_len:4, key_sds_avail:0, key_zmalloc: 40, val_sds_len:17, val_sds_avail:0, val_zmalloc: 0"
# 12 (kvobj) + 8 (expiry) + 1 (key-hdr-size) + 1 (sdshdr5) + 4 (key) + 1 (\0) + 3 (sdshdr8) + 17 (val) + 1 (\0) = 48bytes
expected_mem x234 y2345678901234567 1 48 "key_sds_len:4, key_sds_avail:0, key_zmalloc: 48, val_sds_len:17, val_sds_avail:0, val_zmalloc: 0"
}
} {} {needs:debug}
}
}