platform_system_core/logd
Mark Salyzyn c9e5f37166 liblog: add android_lookupEventTagNum
android_lookupEventTagNum added.  Adds support for creating a new
log tag at runtime, registered to the logd service.

Tested on Hikey, all services stopped, shell only access, CPUs not
locked (there is enough repeatability on this platform).

$ /data/nativetest64/liblog-benchmarks/liblog-benchmarks BM_lookupEventTagNum

                          iterations      ns/op
Precharge: start
Precharge: stop 231
NB: only Tag matching, linear lookup (as reference, before unordered_map)
BM_lookupEventTagNum         1000000       1017
NB: unordered_map with full Tag & Format lookup, but with Tag hashing
BM_lookupEventTagNum         2000000        683
NB: with full Tag & Format hash and lookup for matching
BM_lookupEventTagNum         2000000        814
NB: only Tag matching (Hail Mary path)
BM_lookupEventTagNum         5000000        471

Because the database can now be dynamic, we added reader/writer locks
which adds a 65ns (uncontended) premium on lookups, and switch to
check for an allocation adds 25ns (either open code, or using
string_view, no difference) which means our overall speed takes 90%
as long as the requests did before we switched to unordered_map.
Faster than before where we originally utilized binary lookup on
static content, but not by much.  Dynamic updates that are not cached
locally take the following times to acquire long path to logd to
generate.

BM_lookupEventTag           20000000         139
BM_lookupEventTag_NOT       20000000         87
BM_lookupEventFormat        20000000         139
BM_lookupEventTagNum_logd_new   5000         335936
BM_lookupEventTagNum_logd_existing 10000     249226

The long path pickups are mitigated by the built-in caching, and
the public mapping in /dev/event-log-tags.

SideEffects: Event tags and signal handlers do not mix
Test: liblog benchmarks
Bug: 31456426
Change-Id: I69e6489d899cf35cdccffcee0d8d7cad469ada0a
2017-02-02 15:21:15 -08:00
..
tests liblog: add android_lookupEventTagNum 2017-02-02 15:21:15 -08:00
Android.mk logd: add getEventTag command and service 2017-02-02 14:24:18 -08:00
CommandListener.cpp logd: add getEventTag command and service 2017-02-02 14:24:18 -08:00
CommandListener.h logd: add getEventTag command and service 2017-02-02 14:24:18 -08:00
FlushCommand.cpp logd: Allow (some) headers to be individually importable 2016-02-25 12:42:31 -08:00
FlushCommand.h liblog: logd: logcat: Split out log/logger.h into public and private. 2016-10-24 11:12:49 -07:00
LogAudit.cpp logd: sepolicy dynamic rate limiting 2017-01-04 14:46:58 -08:00
LogAudit.h logd: sepolicy dynamic rate limiting 2017-01-04 14:46:58 -08:00
LogBuffer.cpp logd: add getEventTag command and service 2017-02-02 14:24:18 -08:00
LogBuffer.h logd: add getEventTag command and service 2017-02-02 14:24:18 -08:00
LogBufferElement.cpp logd: multiple identical do not report expired 2017-01-24 11:02:18 -08:00
LogBufferElement.h logd: multiple identical do not report expired 2017-01-24 11:02:18 -08:00
LogCommand.cpp logd: liblog: logcat: Add LOG_ID_SECURITY 2015-12-08 16:46:29 -08:00
LogCommand.h Fix google-explicit-constructor warnings in system/core. 2016-07-26 11:26:01 -07:00
LogKlog.cpp logd: record multiple duplicate messages as chatty 2016-12-15 16:31:51 -08:00
LogKlog.h liblog: logd: logcat: Split out log/logger.h into public and private. 2016-10-24 22:53:11 +00:00
LogListener.cpp system/core: preparation to pull back interfaces from android/log.h 2016-10-20 08:11:39 -07:00
LogListener.h logd: initial checkin. 2014-02-26 09:52:35 -08:00
LogReader.cpp utils: Add FastStrcmp.h 2016-12-09 12:40:17 -08:00
LogReader.h Fix google-explicit-constructor warnings in system/core. 2016-07-26 11:26:01 -07:00
LogStatistics.cpp logd: add getEventTag command and service 2017-02-02 14:24:18 -08:00
LogStatistics.h logd: add getEventTag command and service 2017-02-02 14:24:18 -08:00
LogTags.cpp logd: add getEventTag command and service 2017-02-02 14:24:18 -08:00
LogTags.h logd: add getEventTag command and service 2017-02-02 14:24:18 -08:00
LogTimes.cpp logd: clarify release_Locked() for static analyzer 2016-03-02 11:37:21 -08:00
LogTimes.h liblog: logd: logcat: Split out log/logger.h into public and private. 2016-10-24 11:12:49 -07:00
LogUtils.h logd: add getEventTag command and service 2017-02-02 14:24:18 -08:00
LogWhiteBlackList.cpp logd: Add worst pid of system filter 2015-12-29 09:32:35 -08:00
LogWhiteBlackList.h logd: Add worst pid of system filter 2015-12-29 09:32:35 -08:00
README.auditd logd: selinux auditd initial commit 2014-04-07 10:51:00 -07:00
README.property logd: Add support for ro.logd.auditd.[main|events] 2017-01-03 09:44:42 -08:00
event.logtags logd: add getEventTag command and service 2017-02-02 14:24:18 -08:00
libaudit.c logd: sepolicy dynamic rate limiting 2017-01-04 14:46:58 -08:00
libaudit.h logd: sepolicy dynamic rate limiting 2017-01-04 14:46:58 -08:00
logd.rc logd: add getEventTag command and service 2017-02-02 14:24:18 -08:00
logtagd.rc logd: add getEventTag command and service 2017-02-02 14:24:18 -08:00
main.cpp logd: add getEventTag command and service 2017-02-02 14:24:18 -08:00

README.property

The properties that logd and friends react to are:

name                       type default  description
ro.logd.auditd             bool   true   Enable selinux audit daemon
ro.logd.auditd.dmesg       bool   true   selinux audit messages sent to dmesg.
ro.logd.auditd.main        bool   true   selinux audit messages sent to main.
ro.logd.auditd.events      bool   true   selinux audit messages sent to events.
persist.logd.security      bool   false  Enable security buffer.
ro.device_owner            bool   false  Override persist.logd.security to false
ro.logd.kernel             bool+ svelte+ Enable klogd daemon
ro.logd.statistics         bool+ svelte+ Enable logcat -S statistics.
ro.debuggable              number        if not "1", logd.statistics &
                                         ro.logd.kernel default false.
logd.logpersistd.enable    bool   auto   Safe to start logpersist daemon service
logd.logpersistd          string persist Enable logpersist daemon, "logcatd"
                                         turns on logcat -f in logd context.
					 Responds to logcatd, clear and stop.
logd.logpersistd.buffer          persist logpersistd buffers to collect
logd.logpersistd.size            persist logpersistd size in MB
persist.logd.logpersistd   string        Enable logpersist daemon, "logcatd"
                                         turns on logcat -f in logd context.
persist.logd.logpersistd.buffer    all   logpersistd buffers to collect
persist.logd.logpersistd.size      256   logpersistd size in MB
persist.logd.size          number  ro    Global default size of the buffer for
                                         all log ids at initial startup, at
                                         runtime use: logcat -b all -G <value>
ro.logd.size               number svelte default for persist.logd.size. Larger
                                         platform default sizes than 256KB are
                                         known to not scale well under log spam
                                         pressure. Address the spam first,
                                         resist increasing the log buffer.
persist.logd.size.<buffer> number  ro    Size of the buffer for <buffer> log
ro.logd.size.<buffer>      number svelte default for persist.logd.size.<buffer>
ro.config.low_ram          bool   false  if true, logd.statistics, logd.kernel
                                         default false, logd.size 64K instead
                                         of 256K.
persist.logd.filter        string        Pruning filter to optimize content.
                                         At runtime use: logcat -P "<string>"
ro.logd.filter       string "~! ~1000/!" default for persist.logd.filter.
                                         This default means to prune the
                                         oldest entries of chattiest UID, and
                                         the chattiest PID of system
                                         (1000, or AID_SYSTEM).
persist.logd.timestamp     string  ro    The recording timestamp source.
                                         "m[onotonic]" is the only supported
                                         key character, otherwise realtime.
ro.logd.timestamp        string realtime default for persist.logd.timestamp
log.tag                   string persist The global logging level, VERBOSE,
                                         DEBUG, INFO, WARN, ERROR, ASSERT or
                                         SILENT. Only the first character is
                                         the key character.
persist.log.tag            string build  default for log.tag
log.tag.<tag>             string persist The <tag> specific logging level.
persist.log.tag.<tag>      string build  default for log.tag.<tag>

NB:
- auto - managed by /init
- bool+ - "true", "false" and comma separated list of "eng" (forced false if
  ro.debuggable is not "1") or "svelte" (forced false if ro.config.low_ram is
  true).
- svelte - see ro.config.low_ram for details.
- svelte+ - see ro.config.low_ram and ro.debuggable for details.
- ro - <base property> temporary override, ro.<base property> platform default.
- persist - <base property> override, persist.<base property> platform default.
- build - VERBOSE for native, DEBUG for jvm isLoggable, or developer option.
- number - support multipliers (K or M) for convenience. Range is limited
  to between 64K and 256M for log buffer sizes. Individual log buffer ids
  such as main, system, ... override global default.
- Pruning filter is of form of a space-separated list of [~][UID][/PID]
  references, where '~' prefix means to blacklist otherwise whitelist. For
  blacklisting, UID or PID may be a '!' to instead reference the chattiest
  client, with the restriction that the PID must be in the UID group 1000
  (system or AID_SYSTEM).