redis/tests/unit
Meir Shpilraien (Spielrein) 4db4b43417
Function Flags support (no-writes, no-cluster, allow-state, allow-oom) (#10066)
# Redis Functions Flags

Following the discussion on #10025 Added Functions Flags support.
The PR is divided to 2 sections:
* Add named argument support to `redis.register_function` API.
* Add support for function flags

## `redis.register_function` named argument support

The first part of the PR adds support for named argument on `redis.register_function`, example:
```
redis.register_function{
    function_name='f1',
    callback=function()
        return 'hello'
    end,
    description='some desc'
}
```

The positional arguments is also kept, which means that it still possible to write:
```
redis.register_function('f1', function() return 'hello' end)
```

But notice that it is no longer possible to pass the optional description argument on the positional
argument version. Positional argument was change to allow passing only the mandatory arguments
(function name and callback). To pass more arguments the user must use the named argument version.

As with positional arguments, the `function_name` and `callback` is mandatory and an error will be
raise if those are missing. Also, an error will be raise if an unknown argument name is given or the
arguments type is wrong.

Tests was added to verify the new syntax.

## Functions Flags

The second part of the PR is adding functions flags support.
Flags are given to Redis when the engine calls `functionLibCreateFunction`, supported flags are:

* `no-writes` - indicating the function perform no writes which means that it is OK to run it on:
   * read-only replica
   * Using FCALL_RO
   * If disk error detected
   
   It will not be possible to run a function in those situations unless the function turns on the `no-writes` flag

* `allow-oom` - indicate that its OK to run the function even if Redis is in OOM state, if the function will
  not turn on this flag it will not be possible to run it if OOM reached (even if the function declares `no-writes`
  and even if `fcall_ro` is used). If this flag is set, any command will be allow on OOM (even those that is
  marked with CMD_DENYOOM). The assumption is that this flag is for advance users that knows its
  meaning and understand what they are doing, and Redis trust them to not increase the memory usage.
  (e.g. it could be an INCR or a modification on an existing key, or a DEL command)

* `allow-state` - indicate that its OK to run the function on stale replica, in this case we will also make
  sure the function is only perform `stale` commands and raise an error if not.

* `no-cluster` - indicate to disallow running the function if cluster is enabled.

Default behaviure of functions (if no flags is given):
1. Allow functions to read and write
2. Do not run functions on OOM
3. Do not run functions on stale replica
4. Allow functions on cluster

### Lua API for functions flags

On Lua engine, it is possible to give functions flags as `flags` named argument:

```
redis.register_function{function_name='f1', callback=function() return 1 end, flags={'no-writes', 'allow-oom'}, description='description'}
```

The function flags argument must be a Lua table that contains all the requested flags, The following
will result in an error:
* Unknown flag
* Wrong flag type

Default behaviour is the same as if no flags are used.

Tests were added to verify all flags functionality

## Additional changes
* mark FCALL and FCALL_RO with CMD_STALE flag (unlike EVAL), so that they can run if the function was
  registered with the `allow-stale` flag.
* Verify `CMD_STALE` on `scriptCall` (`redis.call`), so it will not be possible to call commands from script while
  stale unless the command is marked with the `CMD_STALE` flags. so that even if the function is allowed while
  stale we do not allow it to bypass the `CMD_STALE` flag of commands.
* Flags section was added to `FUNCTION LIST` command to provide the set of flags for each function:
```
> FUNCTION list withcode
1)  1) "library_name"
    2) "test"
    3) "engine"
    4) "LUA"
    5) "description"
    6) (nil)
    7) "functions"
    8) 1) 1) "name"
          2) "f1"
          3) "description"
          4) (nil)
          5) "flags"
          6) (empty array)
    9) "library_code"
   10) "redis.register_function{function_name='f1', callback=function() return 1 end}"
```
* Added API to get Redis version from within a script, The redis version can be provided using:
   1. `redis.REDIS_VERSION` - string representation of the redis version in the format of MAJOR.MINOR.PATH
   2. `redis.REDIS_VERSION_NUM` - number representation of the redis version in the format of `0x00MMmmpp`
      (`MM` - major, `mm` - minor,  `pp` - patch). The number version can be used to check if version is greater or less 
      another version. The string version can be used to return to the user or print as logs.

   This new API is provided to eval scripts and functions, it also possible to use this API during functions loading phase.
2022-01-14 14:02:02 +02:00
..
moduleapi Added RM_MonotonicMicroseconds() API to provide monotonic time function (#10101) 2022-01-13 11:36:03 +02:00
type Move doc metadata from COMMAND to COMMAND DOCS (#10056) 2022-01-11 17:16:16 +02:00
acl.tcl Show subcommand full name in error log / ACL LOG (#10105) 2022-01-12 20:05:14 +02:00
aofrw.tcl Fix kill aof rewrite child test (#10107) 2022-01-13 12:38:41 +02:00
auth.tcl Prevent unauthenticated client from easily consuming lots of memory (CVE-2021-32675) (#9588) 2021-10-04 12:10:31 +03:00
bitfield.tcl Improve test suite to handle external servers better. (#9033) 2021-06-09 15:13:24 +03:00
bitops.tcl Change lzf to handle values larger than UINT32_MAX (#9776) 2021-11-16 13:12:25 +02:00
client-eviction.tcl Client eviction ci issues (#9549) 2021-09-26 17:45:02 +03:00
cluster.tcl Function Flags support (no-writes, no-cluster, allow-state, allow-oom) (#10066) 2022-01-14 14:02:02 +02:00
dump.tcl Improve test suite to handle external servers better. (#9033) 2021-06-09 15:13:24 +03:00
expire.tcl Implement Multi Part AOF mechanism to avoid AOFRW overheads. (#9788) 2022-01-03 19:14:13 +02:00
functions.tcl Function Flags support (no-writes, no-cluster, allow-state, allow-oom) (#10066) 2022-01-14 14:02:02 +02:00
geo.tcl GEO* STORE with empty src key delete the dest key and return 0, not empty array (#9271) 2021-08-01 19:32:24 +03:00
hyperloglog.tcl Improve test suite to handle external servers better. (#9033) 2021-06-09 15:13:24 +03:00
info.tcl Changed latency histogram output to omit trailing 0s and periods (#10075) 2022-01-09 17:04:18 -08:00
introspection-2.tcl Fix COMMAND GETKEYS on LCS (#9852) 2021-11-28 09:02:38 +02:00
introspection.tcl Show the elapsed time of single test and speed up some tests (#10058) 2022-01-05 13:49:01 +02:00
keyspace.tcl Add external test that runs without debug command (#9964) 2021-12-19 17:41:51 +02:00
latency-monitor.tcl Added INFO LATENCYSTATS section: latency by percentile distribution/latency by cumulative distribution of latencies (#9462) 2022-01-05 14:01:05 +02:00
lazyfree.tcl attempt to fix tracking test issue with external tests due to lazy free (#9722) 2021-11-02 16:42:53 +02:00
limits.tcl Improve test suite to handle external servers better. (#9033) 2021-06-09 15:13:24 +03:00
maxmemory.tcl Added INFO LATENCYSTATS section: latency by percentile distribution/latency by cumulative distribution of latencies (#9462) 2022-01-05 14:01:05 +02:00
memefficiency.tcl Add external test that runs without debug command (#9964) 2021-12-19 17:41:51 +02:00
multi.tcl Fix typo in multi test (#10054) 2022-01-05 10:16:04 +02:00
networking.tcl Protected configs and sensitive commands (#9920) 2021-12-19 10:46:16 +02:00
obuf-limits.tcl Added INFO LATENCYSTATS section: latency by percentile distribution/latency by cumulative distribution of latencies (#9462) 2022-01-05 14:01:05 +02:00
oom-score-adj.tcl Don't write oom score adj to proc unless we're managing it. (#9904) 2021-12-07 16:05:51 +02:00
other.tcl Fix crash when error [sub]command name contains | (#10082) 2022-01-09 13:06:51 +02:00
pause.tcl Shorten timeouts of CLIENT PAUSE to avoid hanging when tests fail. (#9975) 2021-12-22 12:06:29 +02:00
pendingquerybuf.tcl Introduce memory management on cluster link buffers (#9774) 2021-12-16 21:56:59 -08:00
printver.tcl Print version info before running the test 2011-05-20 11:44:54 +02:00
protocol.tcl Tests: add a few missing needs:debug tags. (#9806) 2021-11-18 23:01:56 +02:00
pubsub.tcl Connection leak in external tests. (#9777) 2021-11-15 11:07:43 +02:00
pubsubshard.tcl Sharded pubsub implementation (#8621) 2022-01-02 16:54:47 -08:00
querybuf.tcl Ignore resize threshold on idle qbuf resizing (#9322) 2021-08-06 20:50:34 +03:00
quit.tcl Add tests for OK on QUIT 2010-10-15 12:54:53 +02:00
scan.tcl Replace all usage of ziplist with listpack for t_zset (#9366) 2021-09-09 18:18:53 +03:00
scripting.tcl Function Flags support (no-writes, no-cluster, allow-state, allow-oom) (#10066) 2022-01-14 14:02:02 +02:00
shutdown.tcl Wait for replicas when shutting down (#9872) 2022-01-02 09:50:15 +02:00
slowlog.tcl Redact ACL SETUSER arguments if the user has spaces (#9935) 2021-12-13 08:39:04 -08:00
sort.tcl Add SORT_RO command (#9299) 2021-08-09 09:40:29 +03:00
tls.tcl Add support for reading encrypted keyfiles. (#8644) 2021-03-22 13:27:46 +02:00
tracking.tcl Solve issues with tracking test in external mode (#9726) 2021-11-02 16:07:51 -07:00
violations.tcl Fix possible int overflow when hashing an sds. (#9916) 2021-12-13 21:16:25 +02:00
wait.tcl Improve test suite to handle external servers better. (#9033) 2021-06-09 15:13:24 +03:00