Commit Graph

339 Commits

Author SHA1 Message Date
Mark Salyzyn 0c5ab13a72 logd: worst uid record watermark
(cherry pick from commit c892ea3fa8)

Hold on to last worst uid watermark and bypass a spike to O(n*n*x)
(n=samples, x=number of spammers) wrt chatty trimming.

Bug: 23327476
Change-Id: I9f21ce95e969b67e576417a760f75c4d86acf364
2015-08-20 11:33:57 -07:00
Mark Salyzyn c2f7eee26b logd: log buffer switch to std::list
(cherry pick from commit 94a89c42fe)

Bug: 23350706
Bug: 23327476
Change-Id: Ib0530b9dd5842c6d05c84d7a66f2531c97461067
2015-08-20 11:32:25 -07:00
Mark Salyzyn dc1c936139 logd: white and black switch to std::list
(cherry pick from commit e0ed16c6db)

Bug: 23350706
Change-Id: I981d9fa63a0d87eb309485cca93cfc44fc0506cc
2015-08-20 11:31:46 -07:00
Mark Salyzyn a68a5167bc logd: logtimes switch to std::list
(cherry pick from commit 98dca2d0b1)

Bug: 23350706
Change-Id: Icc60dd06119ea20a22610644ff880d5135363aba
2015-08-20 11:30:49 -07:00
Mark Salyzyn edc6f52a89 logd: prune 10% or 256 entries max
(cherry pick from commit b39ed0c992)

Bug: 22351810
Bug: 23327476
Change-Id: I902ba6b431d8b7cee2d65ee2f76e9f7c4f30b152
2015-08-20 11:29:52 -07:00
Mark Salyzyn 8894c395bf logd: sizes > 1M prune in smaller batches
(cherry pick from commit 62ab0fd4ef)

Switch to 1% batch sizes from 10% when individual buffer size > 1M

Bug: 22351810
Change-Id: Ifee570a54643ceb0ba767e1787e937f70cc90b72
2015-08-20 11:28:53 -07:00
Mark Salyzyn 73160acc5c logd: switch asprintf to std::string
Bug: 23350706
Change-Id: I715cdd4563a09de3680081947a3439f0cac623be
2015-08-20 10:32:16 -07:00
Mark Salyzyn decbcd9c41 logd: statistics switch to std::string
Bug: 23350706
Change-Id: I5564898c4f67b8fcc43cee64604855f789409482
2015-08-20 10:25:57 -07:00
Mark Salyzyn b332f1c427 logd: white and black switch to std::string
Bug: 23350706
Change-Id: I92f21aee0a9702f63e8465851d0f35007b0469a7
2015-08-20 10:25:57 -07:00
Mark Salyzyn c892ea3fa8 logd: worst uid record watermark
Hold on to last worst uid watermark and bypass a spike to O(n*n*x)
(n=samples, x=number of spammers) wrt chatty trimming.

Bug: 23327476
Change-Id: I9f21ce95e969b67e576417a760f75c4d86acf364
2015-08-20 10:25:57 -07:00
Mark Salyzyn 94a89c42fe logd: log buffer switch to std::list
Bug: 23350706
Bug: 23327476
Change-Id: Ib0530b9dd5842c6d05c84d7a66f2531c97461067
2015-08-20 08:36:13 -07:00
Mark Salyzyn e0ed16c6db logd: white and black switch to std::list
Bug: 23350706
Change-Id: I981d9fa63a0d87eb309485cca93cfc44fc0506cc
2015-08-20 08:36:08 -07:00
Mark Salyzyn 98dca2d0b1 logd: logtimes switch to std::list
Bug: 23350706
Change-Id: Icc60dd06119ea20a22610644ff880d5135363aba
2015-08-20 08:36:03 -07:00
Mark Salyzyn b39ed0c992 logd: prune 10% or 256 entries max
Bug: 22351810
Bug: 23327476
Change-Id: I902ba6b431d8b7cee2d65ee2f76e9f7c4f30b152
2015-08-20 08:35:45 -07:00
Mark Salyzyn 1763d1aa78 am e49aba54: am 95e7cb5b: Merge "healthd: logd: add timestamp to kernel logged battery messages"
* commit 'e49aba5413419bbc4f87889edd9a12956a932ef8':
  healthd: logd: add timestamp to kernel logged battery messages
2015-08-12 14:29:01 +00:00
Mark Salyzyn 95e7cb5b8e Merge "healthd: logd: add timestamp to kernel logged battery messages" 2015-08-12 14:17:38 +00:00
Mark Salyzyn 5cfda541dd am c5482138: am ea56594c: Merge "logd: sizes > 1M prune in smaller batches"
* commit 'c54821388d685eff70a466cb01b2e8bb3e6c98fe':
  logd: sizes > 1M prune in smaller batches
2015-08-12 00:04:27 +00:00
Mark Salyzyn 62ab0fd4ef logd: sizes > 1M prune in smaller batches
Switch to 1% batch sizes from 10% when individual buffer size > 1M

Bug: 22351810
Change-Id: Ifee570a54643ceb0ba767e1787e937f70cc90b72
2015-08-11 16:20:54 -07:00
Mark Salyzyn acb1ddf56c healthd: logd: add timestamp to kernel logged battery messages
Aid monotonic to realtime logging synchronization correction in
the Android ecosystem by providing a periodic notification.  We
now have the following messages in the kernel logs:

- PM: suspend entry %Y-%m-%d %H:%M:%S.%09q UTC
- PM: suspend exit %Y-%m-%d %H:%M:%S.%09q UTC
- Suspended for %s.%03q seconds
- healthd: battery l=100 ... %Y-%m-%d %H:%M:%S.%09q UTC

Alter klogd to resynchronize on healthd messages as well.

NB: Time using strftime format, %q is a reference to fractional
second as introduced into log_time strptime method.

Bug: 21868540
Change-Id: I854afc0a07dff9c7f26d2b2f68990e52bf90e300
2015-07-28 16:52:58 -07:00
Mark Salyzyn 6e629e46a1 am f2e86153: am fac3bf35: Merge changes Id65ed2e8,Idbf8fae9
* commit 'f2e86153eda03144f14f75899a5743ec54e667f9':
  liblog: deprecate TARGET_USES_LOGD
  logd: deprecate TARGET_USES_LOGD
2015-07-28 18:31:40 +00:00
Mark Salyzyn cb41e3673b logd: deprecate TARGET_USES_LOGD
This is not the kernel logger you are looking for

Bug: 22787659
Change-Id: Idbf8fae908287ae9a96007b6353ec56d3c49f5bf
2015-07-28 09:37:07 -07:00
Andreas Gampe ba6def6805 am 0c61a650: am 2e132ef6: Merge "Logd: Handle unused variable and fields"
* commit '0c61a650ce8638e6fd7bbbb1ef7ed42e75167194':
  Logd: Handle unused variable and fields
2015-07-28 00:17:05 +00:00
Andreas Gampe d75564f9b8 Logd: Handle unused variable and fields
For build-system CFLAGS clean-up.

Bug: 18632512
Change-Id: If81d6705b44e9a29f64c44c56ea633c031e831b7
2015-07-27 14:17:33 -07:00
Jeff Vander Stoep c200e9564b am 3f62a020: logd: allow logd to write to /dev/cpuset files
* commit '3f62a020c48d5d812fb2898759b93a59dc24d310':
  logd: allow logd to write to /dev/cpuset files
2015-07-24 23:31:53 +00:00
Jeff Vander Stoep 3f62a020c4 logd: allow logd to write to /dev/cpuset files
Required by logd on devices with USE_CPUSETS defined.

Make /dev/cpuset/background, /dev/cpuset/foreground and
/dev/cpuset/task writeable by system gid. Add logd to system
group for writing to cpuset files and to root group to avoid
regressions. When dropping privs, also drop supplementary groups.

Bug: 22699101
Change-Id: Icc01769b18b5e1f1649623da8325a8bfabc3a3f0
2015-07-24 16:22:06 +00:00
Mark Salyzyn 618d0dec50 logd: refine is_prio
- Heuristics associated with translation of kernel messages to
  Android user space logs.
- Limit is_prio to 4 characters, we got false positives on hex
  values like <register contents> with no alpha chars.
- x11 and other register definitions are not valid tags, en0 is
- fix some Android coding standard issues

Change-Id: Idc3dcc53a2cb75ac38628c8ef7a5d5b53f12587a
2015-07-22 07:36:03 -07:00
Mark Salyzyn 5803b79528 logd: serialize accesses to stats helpers
(cherry pick from commit ed777e9eec)

Quick low-risk to resolve possible hash table corruption.
Resolved an unlikely path memory leak.

ToDo: replace lock with nested lock so no lock
      helpers are required.

Bug: 22068332
Change-Id: I303ab06608502c7d61d42f111a9c43366f184d0c
2015-06-25 08:34:06 -07:00
Mark Salyzyn ed777e9eec logd: serialize accesses to stats helpers
Quick low-risk to resolve possible hash table corruption.
Resolved an unlikely path memory leak.

ToDo: replace lock with nested lock so no lock
      helpers are required.

Bug: 22068332
Change-Id: I303ab06608502c7d61d42f111a9c43366f184d0c
2015-06-25 07:39:24 -07:00
Mark Salyzyn d5600fd40f logd: missing klogd content
(cherry pick from commit ee49c6a670)

- regression in log_strtok_r (part deux) In commit
      'logd: fix kernel logline stutter'
  2c3b300fd8 we introduced log_strtok_r.
  as a replacement for strtok_r that dealt with a problem with
  some kernel log messages. Fix is to refine definition of
  is_timestamp to not match on patterns like [0], requiring
  a single period. Another fix is to refine definition of
  is_prio to properly escape non-digit content.
- Missing content because SYSLOG_ACTION_SIZE_BUFFER with added logging
  is too short for full read of SYSLOG_ACTION_READ_ALL dropping
  initial content. Add a margin for additional 1024 bytes.
- Absolute _first_ log entry has sequence number of 1, which is
  specifically dropped, start sequence count at 1 rather than 0.
- Remove trailing space for efficiency.
- If tag exists but no content, trick into kernel logging.

Bug: 21851884
Change-Id: I0867a555a3bca09bbf18d18e75e41dffffe57a23
2015-06-15 14:22:02 -07:00
Mark Salyzyn ee49c6a670 logd: missing klogd content
- regression in log_strtok_r (part deux) In commit
      'logd: fix kernel logline stutter'
  2c3b300fd8 we introduced log_strtok_r.
  as a replacement for strtok_r that dealt with a problem with
  some kernel log messages. Fix is to refine definition of
  is_timestamp to not match on patterns like [0], requiring
  a single period. Another fix is to refine definition of
  is_prio to properly escape non-digit content.
- Missing content because SYSLOG_ACTION_SIZE_BUFFER with added logging
  is too short for full read of SYSLOG_ACTION_READ_ALL dropping
  initial content. Add a margin for additional 1024 bytes.
- Absolute _first_ log entry has sequence number of 1, which is
  specifically dropped, start sequence count at 1 rather than 0.
- Remove trailing space for efficiency.
- If tag exists but no content, trick into kernel logging.

Bug: 21851884
Change-Id: I0867a555a3bca09bbf18d18e75e41dffffe57a23
2015-06-15 21:19:10 +00:00
Mark Salyzyn bd1ef90448 logd: filter on __android_log_is_loggable
(cherry pick from commit e59c469fa8)

- Default level when not specified is ANDROID_LOG_VERBOSE
  which is inert.

Bug: 20416721
Bug: 19544788
Bug: 17760225
Change-Id: Icc098e53dc47ceaaeb24ec42eb6f61d6430ec2f6
2015-06-15 07:41:03 -07:00
Mark Salyzyn e59c469fa8 logd: filter on __android_log_is_loggable
- Default level when not specified is ANDROID_LOG_VERBOSE
  which is inert.

Bug: 20416721
Bug: 19544788
Bug: 17760225
Change-Id: Icc098e53dc47ceaaeb24ec42eb6f61d6430ec2f6
2015-06-12 10:35:09 -07:00
Mark Salyzyn 3a1ff5ab65 Merge "Lower the priority of the threads in logd/logcat." into mnc-dev 2015-06-09 19:42:45 +00:00
Riley Andrews aede9897df Lower the priority of the threads in logd/logcat.
(cherry pick from commit d98f4e8af5)

sched_batch implies only a penalty to latency in scheduling, but
does not imply that the process will be given less cpu time. Increase
the nice level to 10 to prioritize it below ui threads.

Bug: 21696721
Change-Id: I075af059dc755402f7df9b0d7a66cca921ff04b2
2015-06-09 12:40:20 -07:00
Riley Andrews d98f4e8af5 Lower the priority of the threads in logd/logcat.
sched_batch implies only a penalty to latency in scheduling, but
does not imply that the process will be given less cpu time. Increase
the nice level to 10 to prioritize it below ui threads.

Bug: 21696721
Change-Id: I075af059dc755402f7df9b0d7a66cca921ff04b2
2015-06-09 12:36:45 -07:00
Mark Salyzyn 03196c9c75 logd: build breakage
(cherry pick from 3e21de2915)

OPEN_BRACKET_SPACE comparison always false

Change-Id: I1ff4288b4b79a49702727d3a8b8c8f179f500951
2015-06-08 14:57:07 -07:00
Mark Salyzyn 3e21de2915 logd: build breakage
OPEN_BRACKET_SPACE comparison always false

Change-Id: I1ff4288b4b79a49702727d3a8b8c8f179f500951
2015-06-08 14:52:14 -07:00
Mark Salyzyn df5902c4f3 logd: fix kernel logline stutter
(cherry pick from commit 2c3b300fd8)

- look for cases where one log line contains two without a newline.
- rare condition, occurs when a printk does not have
  a terminating newline under certain race conditions.
- the newline may be performed broken up as a second call
- the timestamps can be reversed (showing the race effects).
- driver(s) should really have the newline in their log messages.

Change-Id: Ibfb56b32047da3d6513db059ca6edad0f0105168
2015-06-08 13:55:14 -07:00
Mark Salyzyn 2c3b300fd8 logd: fix kernel logline stutter
- look for cases where one log line contains two without a newline.
- rare condition, occurs when a printk does not have
  a terminating newline under certain race conditions.
- the newline may be performed broken up as a second call
- the timestamps can be reversed (showing the race effects).
- driver(s) should really have the newline in there log messages.

Change-Id: Ibfb56b32047da3d6513db059ca6edad0f0105168
2015-06-08 13:10:31 -07:00
Mark Salyzyn c1e9d6dada logd: filters remove leading expire messages and rate
(cherry pick from commit 047cc0729f)

- Cleanup resulting from experience and feedback
- When filtering inside logd, drop any leading expire messages, they
  are cluttering up leading edge of tombstones (which filter by pid)
- Increase and introduce EXPIRE_RATELIMIT from 1 to 10 seconds
- Increase EXPIRE_THRESHOLD from 4 to 10 count
- Improve the expire messages from:
   logd : uid=1000(system) too chatty comm=com.google.android.phone,
                                                   expire 2800 lines
  change tag to be more descriptive, and reduce accusatory tone to:
   chatty : uid=1000(system) com.google.android.phone expire 2800
                                                               lines
- if the UID name forms a prefix for comm name, then drop UID name

Change-Id: Ied7cc04c0ab3ae02167649a0b97378e44ef7b588
2015-06-05 10:20:55 -07:00
Mark Salyzyn 047cc0729f logd: filters remove leading expire messages and rate
- Cleanup resulting from experience and feedback
- When filtering inside logd, drop any leading expire messages, they
  are cluttering up leading edge of tombstones (which filter by pid)
- Increase and introduce EXPIRE_RATELIMIT from 1 to 10 seconds
- Increase EXPIRE_THRESHOLD from 4 to 10 count
- Improve the expire messages from:
   logd : uid=1000(system) too chatty comm=com.google.android.phone,
                                                   expire 2800 lines
  change tag to be more descriptive, and reduce accusatory tone to:
   chatty : uid=1000(system) com.google.android.phone expire 2800
                                                               lines
- if the UID name forms a prefix for comm name, then drop UID name

Change-Id: Ied7cc04c0ab3ae02167649a0b97378e44ef7b588
2015-06-05 08:05:05 -07:00
Mark Salyzyn 6f1457adc5 logd: switch to unordered_map from BasicHashtable
(charry pick from commit 511338dd57)

BasicHashtable is relatively untested, move over to
a C++ template library that has more bake time.

Bug: 20419786
Bug: 21590652
Bug: 20500228
Change-Id: I926aaecdc8345eca75c08fdd561b0473504c5d95
2015-06-04 11:03:37 -07:00
Mark Salyzyn 511338dd57 logd: switch to unordered_map from BasicHashtable
BasicHashtable is relatively untested, move over to
a C++ template library that has more bake time.

Bug: 20419786
Bug: 21590652
Bug: 20500228
Change-Id: I926aaecdc8345eca75c08fdd561b0473504c5d95
2015-06-03 13:03:07 -07:00
Mark Salyzyn 49ed105fd9 init.rc: logd: Add logpersistd (nee logcatd)
(cherry pick from commit 100658c303)

- logpersistd is defined as a thread or process in the context of the
  logd domain. Here we define logpersistd as logcat -f in logd domain
  and call it logcatd to represent its service mechanics.
- Use logcatd to manage content in /data/misc/logd/ directory.
- Only turn on for persist.logd.logpersistd = logcatd.
- Add logpersist.start, logpersist.stop and logpersist.cat debug
  class executables, thus only in the eng and userdebug builds.

ToDo: Wish to add Developer Options menu to turn this feature on or
off, complicated by the fact that user builds have no tools with
access rights to /data/misc/logd.

Bug: 19608716
Change-Id: I57ad757f121c473d04f9fabe9d4820a0eca06f31
2015-06-02 15:28:36 -07:00
Mark Salyzyn 100658c303 init.rc: logd: Add logpersistd (nee logcatd)
- logpersistd is defined as a thread or process in the context of the
  logd domain. Here we define logpersistd as logcat -f in logd domain
  and call it logcatd to represent its service mechanics.
- Use logcatd to manage content in /data/misc/logd/ directory.
- Only turn on for persist.logd.logpersistd = logcatd.
- Add logpersist.start, logpersist.stop and logpersist.cat debug
  class executables, thus only in the eng and userdebug builds.

ToDo: Wish to add Developer Options menu to turn this feature on or
off, complicated by the fact that user builds have no tools with
access rights to /data/misc/logd.

Bug: 19608716
Change-Id: I57ad757f121c473d04f9fabe9d4820a0eca06f31
2015-06-02 15:17:59 -07:00
Mark Salyzyn 42ac34f740 logd: test modernization
(cherry pick from commit 62d6a2a921)

Bug: 19603976
Change-Id: Ie920c128e7e6a436fea7a96c7d68bc39e13a2ad4
2015-06-01 14:27:02 -07:00
Mark Salyzyn 222f8c3ac7 logd: whitelist should not preserve expire messages
(cherry pick from commit c5bf3b8304)

Change-Id: I56275c73191b96aa21e7b4049d401e1f44211f9b
2015-06-01 14:27:02 -07:00
Mark Salyzyn fbf96b55d9 logd: KISS & fix preserve a day
(cherry pick from commit 5921276a16)

Code in 833a9b1e38 was borken,
simpler approach is to simply check last entry (to save a
syscall) minus EXPIRE_HOUR_THRESHOLD. This does make longer logs
less likely to call upon the spam detection than the algorithm
being replaced, but sadly we ended up with a log entry in the
future at the beginning of the logs confounding the previous
algorithm.

Bug: 21555259
Change-Id: I04fad67e95c8496521dbabb73b5f32c19d6a16c2
2015-06-01 14:27:02 -07:00
Mark Salyzyn d4bdb7df9b logd: deal with sloppy leading expire messages
(cherry pick from commit 5392aac95d)

The odds of chatty content also leading the logs is pretty high eg:

 1799 12017 I logd: uid=10007 chatty comm=Binder_B, expire 4 lines
 1799  1829 I logd: uid=10007 chatty comm=Binder_2, expire 4 lines
 1919 20637 I logd: uid=10007 chatty comm=m.sersistent, expire 1 line
 1919 20638 I logd: uid=10007 chatty comm=s.persistent, expire 1 line
 1919  2316 I logd: uid=10007 chatty comm=UlrDispatch, expire 4 lines
19379 20634 I logd: uid=10045 chatty, expire 14 lines
19379 19388 I logd: uid=10045 chatty comm=lizerDaemon, expire 4 lines
  591  4396 I logd: uid=1000 chatty comm=Thread-220, expire 5 lines
  591  1377 I logd: uid=1000 chatty comm=Thread-92, expire 4 lines
 1919  2267 I logd: uid=10007 chatty comm=WifiScanner, expire 4 lines
  591  4397 I logd: uid=1000 chatty comm=DhcpClient, expire 4 lines
  591  4398 I logd: uid=1000 chatty comm=Thread-222, expire 4 lines
  226   580 D CommandListener: Setting iface cfg

Change-Id: I5ab24bc7bf5d2690bae7e789831b07f23ff8bcc6
2015-06-01 14:27:02 -07:00
Mark Salyzyn a8b6131ddf Merge "logd: test modernization" 2015-06-01 21:16:58 +00:00
Mark Salyzyn 10a124d342 Merge "logd: whitelist should not preserve expire messages" 2015-06-01 21:16:57 +00:00
Mark Salyzyn 5921276a16 logd: KISS & fix preserve a day
Code in 833a9b1e38 was borken,
simpler approach is to simply check last entry (to save a
syscall) minus EXPIRE_HOUR_THRESHOLD. This does make longer logs
less likely to call upon the spam detection than the algorithm
being replaced, but sadly we ended up with a log entry in the
future at the beginning of the logs confounding the previous
algorithm.

Bug: 21555259
Change-Id: I04fad67e95c8496521dbabb73b5f32c19d6a16c2
2015-06-01 13:06:35 -07:00
Mark Salyzyn 5392aac95d logd: deal with sloppy leading expire messages
The odds of chatty content also leading the logs is pretty high eg:

 1799 12017 I logd: uid=10007 chatty comm=Binder_B, expire 4 lines
 1799  1829 I logd: uid=10007 chatty comm=Binder_2, expire 4 lines
 1919 20637 I logd: uid=10007 chatty comm=m.sersistent, expire 1 line
 1919 20638 I logd: uid=10007 chatty comm=s.persistent, expire 1 line
 1919  2316 I logd: uid=10007 chatty comm=UlrDispatch, expire 4 lines
19379 20634 I logd: uid=10045 chatty, expire 14 lines
19379 19388 I logd: uid=10045 chatty comm=lizerDaemon, expire 4 lines
  591  4396 I logd: uid=1000 chatty comm=Thread-220, expire 5 lines
  591  1377 I logd: uid=1000 chatty comm=Thread-92, expire 4 lines
 1919  2267 I logd: uid=10007 chatty comm=WifiScanner, expire 4 lines
  591  4397 I logd: uid=1000 chatty comm=DhcpClient, expire 4 lines
  591  4398 I logd: uid=1000 chatty comm=Thread-222, expire 4 lines
  226   580 D CommandListener: Setting iface cfg

Change-Id: I5ab24bc7bf5d2690bae7e789831b07f23ff8bcc6
2015-06-01 13:04:09 -07:00
Mark Salyzyn 62d6a2a921 logd: test modernization
Bug: 19603976
Change-Id: Ie920c128e7e6a436fea7a96c7d68bc39e13a2ad4
2015-05-26 12:24:51 -07:00
Mark Salyzyn c5bf3b8304 logd: whitelist should not preserve expire messages
Change-Id: I56275c73191b96aa21e7b4049d401e1f44211f9b
2015-05-21 13:01:56 -07:00
Mark Salyzyn 94a811ab19 logd: worst-UID only to preserve a day
(cherry pick from commit 833a9b1e38)

Do not invoke worst-UID pruning in the face of other
UIDs logs that are more than a day old, switch to
pruning oldest only.

Change-Id: Icf988b8d5458400a660d0f8e9d2df3f9d9a4c2d9
2015-05-20 10:03:11 -07:00
Mark Salyzyn 833a9b1e38 logd: worst-UID only to preserve a day
Do not invoke worst-UID pruning in the face of other
UIDs logs that are more than a day old, switch to
pruning oldest only.

Change-Id: Icf988b8d5458400a660d0f8e9d2df3f9d9a4c2d9
2015-05-20 09:47:54 -07:00
Mark Salyzyn 654904f0f9 logd: Add TID statistics
(cherry pick from commit 17ed6797df)

Bug: 19608965
Change-Id: Ifbf0b00c48ef12b5970b9f9f217bd1dd8f587f2c
2015-05-13 09:23:01 -07:00
Mark Salyzyn 66091f11f4 logd: Cleanup
(cherry pick from commit 7718778793)

- Android Coding Standard for Constructors
- Side effects NONE

Change-Id: I2cda9dd73f3ac3ab58f394015cb810820093d47b
2015-05-13 09:22:56 -07:00
Mark Salyzyn a1aacb71f3 logd: Add klogd
(cherry pick from commit ae4d928d81)

- Add a klogd to collect the kernel logs and place them into a
  new kernel log buffer
- Parse priority, tag and message from the kernel log messages.
- Turn off pruning for worst UID for the kernel log buffer
- Sniff for 'PM: suspend exit', 'PM: suspend enter' and
  'Suspended for' messages and correct the internal definition
  time correction against monotonic dynamically.
- Discern if we have monotonic or real time (delineation 1980) in
  audit messages.
- perform appropriate math to correct the timestamp to be real time
- filter out any external sources of kernel logging

Change-Id: I8d4c7c5ac19f1f3218079ee3a05a50e2ca55f60d
2015-05-13 09:22:45 -07:00
Mark Salyzyn d1371a8b8b Merge "logd: Add TID statistics" 2015-05-13 16:17:37 +00:00
Mark Salyzyn 7718778793 logd: Cleanup
- Android Coding Standard for Constructors
- Side effects NONE

Change-Id: I2cda9dd73f3ac3ab58f394015cb810820093d47b
2015-05-12 15:51:46 -07:00
Mark Salyzyn ae4d928d81 logd: Add klogd
- Add a klogd to collect the kernel logs and place them into a
  new kernel log buffer
- Parse priority, tag and message from the kernel log messages.
- Turn off pruning for worst UID for the kernel log buffer
- Sniff for 'PM: suspend exit', 'PM: suspend enter' and
  'Suspended for' messages and correct the internal definition
  time correction against monotonic dynamically.
- Discern if we have monotonic or real time (delineation 1980) in
  audit messages.
- perform appropriate math to correct the timestamp to be real time
- filter out any external sources of kernel logging

Change-Id: I8d4c7c5ac19f1f3218079ee3a05a50e2ca55f60d
2015-05-12 15:51:46 -07:00
Mark Salyzyn ff32f3c774 logd: Add Tag statistics
(cherry pick from commit 344bff4391)

- Optional class of statistics for events Tags
- export tagToName from LogBuffer (located in main.cp to address
  https://android-review.googlesource.com/#/c/110204)
- Can not handle dropped because getTag() can not work,
  will need to be fixed if we start filtering.

Bug: 19608965
Change-Id: I7b90607ca588bf37bab71f19b1570a290e772776
2015-05-12 15:34:56 -07:00
Mark Salyzyn 4fba28671b logd: pidToName deficiency
(Cherry pick from commit aa43ae2268)

any logging in zygote permanently sets the pidToName

Change-Id: I0b8fdc96fb5965a04d4dc4293c39815beef5eb8e
2015-05-12 13:06:25 -07:00
Mark Salyzyn 17ed6797df logd: Add TID statistics
Bug: 19608965
Change-Id: Ifbf0b00c48ef12b5970b9f9f217bd1dd8f587f2c
2015-05-12 12:57:25 -07:00
Mark Salyzyn 344bff4391 logd: Add Tag statistics
- Optional class of statistics for events Tags
- export tagToName from LogBuffer (located in main.cp to address
  https://android-review.googlesource.com/#/c/110204)
- Can not handle dropped because getTag() can not work,
  will need to be fixed if we start filtering.

Bug: 19608965
Change-Id: I7b90607ca588bf37bab71f19b1570a290e772776
2015-05-12 12:53:27 -07:00
Mark Salyzyn 7297278ade Merge "logd: pidToName deficiency" 2015-05-12 19:38:40 +00:00
Mark Salyzyn 9bdc064720 logd: uidToName add dex2oat UID
(cherry pick from commit 023f51f360)

- When searching for uidToName from /data/system/packages.list
  one must search for the uid % AID_USER;
- If uid % AID_USER is greater than AID_SHARED_GID_START - AID_APP,
  then subract it and try again to pick up uidToName.

Change-Id: I24df42957ff7e746fffa5d39484637b1b726b9c0
2015-05-12 09:55:02 -07:00
Mark Salyzyn 80b129f9f0 Merge "logd: uidToName add dex2oat UID" 2015-05-11 23:51:32 +00:00
Nick Kralevich 6de7a06afb Revert "libaudit: limit to 5 selinux denials per sec"
The shamu instabilities continued even after throttling SELinux denials
to 5/second. 5 denials per second is too low when doing device bringup,
and there have been some complaints about lost SELinux denials. See,
for example, http://comments.gmane.org/gmane.comp.security.selinux/21941

Bring the limit back up to 20/second to prevent dropping too many
denials on the floor.

This reverts commit a15db51bbf.

(cherrypick of commit 9667a66019)

Change-Id: I05e85cce0a792d05aa557fcc614c0fc019c15014
2015-05-11 15:43:25 -07:00
Nick Kralevich 9667a66019 Revert "libaudit: limit to 5 selinux denials per sec"
The shamu instabilities continued even after throttling SELinux denials
to 5/second. 5 denials per second is too low when doing device bringup,
and there have been some complaints about lost SELinux denials. See,
for example, http://comments.gmane.org/gmane.comp.security.selinux/21941

Bring the limit back up to 20/second to prevent dropping too many
denials on the floor.

This reverts commit a15db51bbf.

Change-Id: I05e85cce0a792d05aa557fcc614c0fc019c15014
2015-05-09 12:41:41 -07:00
Mark Salyzyn 8b22c293a0 logd: class hierarcy for Uid and Pid statistics.
(Cherry pick from commit 81b3eabc49)

Add EntryBase and EntryBaseDropped base classes for statistical
entries to inherit from. Abstract add(), subtract() and drop()
methods to common LogBufferElement in each for entry policy decisions.
Some move of details out of LogStatistics.cpp and place them into
LogStatistics.h. Add statistical add(), subtract() and
drop() methods to hash table to call entries for policy.

Bug: 19608965
Change-Id: Ib8a33a8fe28871ef165d1632c6546a5c606231e8
2015-05-07 10:01:39 -07:00
Mark Salyzyn 023f51f360 logd: uidToName add dex2oat UID
- When searching for uidToName from /data/system/packages.list
  one must search for the uid % AID_USER;
- If uid % AID_USER is greater than AID_SHARED_GID_START - AID_APP,
  then subract it and try again to pick up uidToName.

Change-Id: I24df42957ff7e746fffa5d39484637b1b726b9c0
2015-05-07 09:16:02 -07:00
Mark Salyzyn aa43ae2268 logd: pidToName deficiency
any logging in zygote permanently sets the pidToName

Change-Id: I0b8fdc96fb5965a04d4dc4293c39815beef5eb8e
2015-05-05 12:34:16 -07:00
Mark Salyzyn aaf9c1b3b4 Merge "logd: class hierarcy for Uid and Pid statistics." 2015-05-05 16:47:03 +00:00
Mark Salyzyn 95108f1844 logd: improve details on chatty records
(Cherry pick from commit 21fb7e0b75)

- Report applications UID, TID/PID by name.
- change wording to have an accurate connotation
- drop privilege check since filtered upstream

Bug: 19608965
Bug: 20334069
Bug: 20370119
Change-Id: I2b1c26580b4c2de293874214ff5ae745546f3cca
2015-04-30 15:25:12 -07:00
Mark Salyzyn 5a9d33ee1c logd: reduce chance of dropped 1 messages
- do not time out at 1 second if drop has count of less than 4

(Cherry picked from commit 35173a9ac5)

Bug: 20334069
Bug: 20370119
Change-Id: I787cb553dfab5ed71abd6ed72b63de675f834e0c
2015-04-30 13:24:59 -07:00
Mark Salyzyn 21fb7e0b75 logd: improve details on chatty records
- Report applications UID, TID/PID by name.
- change wording to have an accurate connotation
- drop privilege check since filtered upstream

Bug: 19608965
Bug: 20334069
Bug: 20370119
Change-Id: I2b1c26580b4c2de293874214ff5ae745546f3cca
2015-04-30 13:14:51 -07:00
Mark Salyzyn 35173a9ac5 logd: reduce chance of dropped 1 messages
- do not time out at 1 second if drop has count of less than 4

Bug: 20334069
Bug: 20370119
Change-Id: I787cb553dfab5ed71abd6ed72b63de675f834e0c
2015-04-30 13:14:51 -07:00
Mark Salyzyn 653f9581bf Merge "logd: per UID less aggressive 12.5% threshold" 2015-04-23 14:41:48 +00:00
Mark Salyzyn e06a6e0f26 logd: ratelimit drop messages to 1/sec
Bug: 20334069
Bug: 20370119
Change-Id: I6f850aec46c4df1c99a5b1f28db75d071e134ad5
2015-04-22 11:58:37 -07:00
Mark Salyzyn d717d805d4 logd: per UID less aggressive 12.5% threshold
Per-UID quota has a threshold of 12.5% of the total log size.
If less than that space is taken by the UID, then we
will not engage the pruning based on worst UID.

Change-Id: I9f15c9a26938f1115eb75e9c28ddb073e7680e06
2015-04-22 11:53:29 -07:00
Mark Salyzyn 81b3eabc49 logd: class hierarcy for Uid and Pid statistics.
Add EntryBase and EntryBaseDropped base classes for statistical
entries to inherit from. Abstract add(), subtract() and drop()
methods to common LogBufferElement in each for entry policy decisions.
Some move of details out of LogStatistics.cpp and place them into
LogStatistics.h. Add statistical add(), subtract() and
drop() methods to hash table to call entries for policy.

Bug: 19608965
Change-Id: Ib8a33a8fe28871ef165d1632c6546a5c606231e8
2015-04-21 10:54:34 -07:00
Mark Salyzyn 5720d2c168 logd: Statistics headers
Answer to the question "Bytes or Lines?"

Change-Id: I352737265aeac63ea8d89e778d4222a9123bdf07
2015-04-21 10:54:34 -07:00
Mark Salyzyn 2c9d909a00 logd: better drop message merging
- Former algorithm anlo coalesced adjacent records
- New algorithm maintains a hash list of all drop
  records and coalesces them all.

Bug: 20334069
Bug: 20370119
Change-Id: Idc15ce31fc1087c2cfa39da60c62feade8b88761
2015-04-20 12:58:54 -07:00
Mark Salyzyn 202e153f94 logd: propagate ::log status
Add a return value for the ::log() methods, this allows
us to optimize the wakeup for the readers to only occur
when the log message is actually placed.

This is for a future where we may dedupe identical log
messages, filter out log messages, and certainly if we
filter the messages out with an internal logd check of
__android_log_is_loggable().

Change-Id: I763b2a7c29502ab7fa0a5d5022c7b60244fcfde4
2015-04-15 19:43:28 -07:00
Mark Salyzyn 56c6575da7 Merge "logd: syscall optimization" 2015-04-15 14:45:24 +00:00
Mark Salyzyn c32afdf913 logd: pidToUid incorrect
Change-Id: I9b0e655780924db125568179771e57b01e80aaae
2015-04-14 13:07:29 -07:00
Mark Salyzyn ae76923839 logd: disable worst uid prune for events buffer
There is some usage statistics that would be hurt by pruning by UID,
since _all_ usage statistics come from system_server. In other words
we expect it to be chatty. Until we formulate and evaluate a better
(eg: per-tag?) filtration mechanism, lets hold off pruning by UID.

Bug: 19608965
Change-Id: Iddd45a671e13bdcf3394c20919ad1f2e4ef36616
2015-04-10 15:45:10 -07:00
Mark Salyzyn 1a01f963ff Revert: "logd: default off by-UID spam filter"
With "logd: inject first UID by log buffer message" we can
remove the FUD introduced by having the UID spam filter on

This reverts commit 4141cb2391

Bug: 19608965
Bug: 14469172
Change-Id: Ifdc20b099e0e426546525b11c1dfe8cc0f830a02
2015-04-10 15:45:10 -07:00
Mark Salyzyn ab0dcf6828 logd: annotate worst-UID pruned entries
- internal dropped entries are associated by prune by worst UID
  and are applied by UID and by PID
- track dropped entries by rewriting them in place
- merge similar dropped entries together for same UID(implied),
  PID and TID so that blame can more clearly be placed
- allow aging of dropped entries by the general backgound pruning
- report individual dropped entries formatted to reader
- add statistics to track dropped entries by UID, the combination
  of statistics and dropped logging can track over-the-top Chattiest
  clients.

Bug: 19608965
Change-Id: Ibc68480df0c69c55703270cd70c6b26aea165853
2015-04-10 15:45:08 -07:00
Mark Salyzyn c6637859c2 logd: Build Breakage
a change slipped in from a rebase conflict, repairing.

Change-Id: Ib6479f88fb044f37b6721035c2f7cc75577c0411
2015-04-08 16:09:28 -07:00
Mark Salyzyn 08739ba71f logd: uidToName improvement
- read packages.list to acquire package names
- hijack the reinit handler as a privileged worker

Bug: 19608965
Change-Id: If7becb34354d6415e5c387ecea7d4109a15259c9
2015-04-08 14:47:54 -07:00
Mark Salyzyn 720f6d1d55 logd: Add Pid statistics
- Optional class of statistics for PID
- Enhance pidToName
- Enhanced uidToName
- Enhance pidToUid
- template sort and iteration

Bug: 19608965
Change-Id: I04a1f02e9851b62987f9b176908134e455f22d1d
2015-04-08 14:37:32 -07:00
Nick Kralevich 58ba58a97c logd: Don't embed a flexible array member within another struct
C (but not C++) has a concept of a flexible array member, which
is documented at https://gcc.gnu.org/onlinedocs/gcc/Zero-Length.html .
Using a flexible array member indicates that the structure is
really a header for a variable length object.

In logd's case, the variable length structure android_event_string_t
was embedded within another structure called
android_log_event_string_t. This makes gcc's __builtin_object_size()
function really confused. When compiling with C++,
__builtin_object_size(android_log_event_string_t.payload.data, 1)
would return 0, whereas if you compiled the code with C, the same
call would (properly) return -1.

Code which does automatic bounds checking, such as the proposed
patch at https://android-review.googlesource.com/145411 , will
cause problems for logd if this syntax is used.

Don't try to embed a variable length structure within another
structure. This doesn't appear to be valid C nor C++, and
while it's worked, it seems problematic.

Instead, inline the structure so it's one big happy structure.

Change-Id: I8ac02b7142a4f6560f5f80df2effcf720f9896fc
2015-04-07 10:12:20 -07:00
Elliott Hughes bcc2b5f44a Remove LOCAL_ADDITIONAL_DEPENDENCIES in cases where it's not needed.
Change-Id: I720b8ef1050da45a7833adef8219b6acb2cf3a38
2015-04-02 14:31:07 -07:00
Mark Salyzyn e3aeeeeccc logd: syscall optimization
- prset(PR_SET_NAME) call once
- No need to call getuid(), should be AID_LOGD

Change-Id: I4dde0b178bc84e711b355cd7677b0dbf905a0634
2015-04-01 19:41:59 +00:00
Mark Salyzyn 801bcecbb5 logd: missing include for string.h
LogCommand.cpp gets string.h inherited from
private/android_filesystem_config.h it should
not rely on this in the future. The intent is
to move fs_config function into libcutils and
thus deprecate any need for string.h in this
include file.

Bug: 19908228
Change-Id: Iaf3a77298b56efd8004300b17e9a1faafad5b08b
2015-04-01 11:18:42 -07:00
Nick Kralevich a15db51bbf libaudit: limit to 5 selinux denials per sec
watchdog is triggering on shamu. This may be due to an excessive
number of SELinux denials. Drop the limit from 20/sec to 5/sec.

Bug: 19950451
Bug: 19949988
Change-Id: I979f11e17c241ff2ebda4dec9694ef441dc5d0ed
2015-03-26 14:04:13 -07:00
Mark Salyzyn 317843decf logd: statistics: missing comma
regression from statistics rewrite series resulted in build breakage

Change-Id: I71b532b23437b9b5f3af47b18c9110f3d7dda48a
2015-03-20 13:46:00 -07:00
Mark Salyzyn 97c1c2beee logd: optimize statistics
- Go back to basic requirements
- Simplify
- use hash tables to minimize memory impact

Bug: 19608965
Change-Id: If7becb34354d6415e5c387ecea7d4109a15259c8
2015-03-20 12:31:23 -07:00
Mark Salyzyn f7c0f75275 logd: replace internal CLOCK_MONOTONIC use with sequence numbers
- switch to simpler and faster internal sequence number, drops
  a syscall overhead on 32-bit platforms.
- add ability to break-out of flushTo loop with filter return -1
  allowing in reduction in reader overhead.

Change-Id: Ic5cb2b9afa4d9470153971fc9197b07279e2b79d
2015-03-18 12:43:23 -07:00
Mark Salyzyn ccbadc6be0 logd: report reinit
Bug: 19681572
Change-Id: I343b9d108f064f87df79512a0fdf1b35513c3136
2015-03-12 20:38:53 +00:00
Mark Salyzyn 11e55cb9c1 logd: add reinit command
- respond to SIGHUP
- respond to logd command "reinit"
- respond to logd --reinit
- reopens files on /data, eg: re-read of persistent properties

Bug: 19681572
Change-Id: Iadac58e6653f027cb7355497bd675eef376ce0a8
2015-03-12 12:24:47 -07:00
Mark Salyzyn 95b467888c logd: build cleanup
- Hard code the "auditd" event tag

Change-Id: I5f76fd286628be5c0ca819e8ed775648c0d2fa44
2015-03-09 10:02:51 -07:00
Mark Salyzyn 4ed16b4381 Revert "logd: Add minimum time bucket statistics"
This forward port reverts
    commit e457b74ce6

No longer as necessary once we add
    liblog: Instrument logging of logd write drops
Although this provided an indication of how close statistically we
were to overloading logd it is simpler to understand fails thus to
hunt and peck a corrected value for /proc/sys/net/unix/max_dgram_qlen

Change-Id: I2b30e0fc30625a48fd11a12c2d2cc6a41f26226f
2015-03-04 13:21:41 -08:00
Mark Salyzyn 29eb57066c logd: use <endian.h>
Change-Id: Iba843c054ea4fbe1a26c7821b5613fdb5e8001a1
2015-03-04 13:21:41 -08:00
Mark Salyzyn 56ba4b5b77 logd: logd placed into background cgroup
Change-Id: I38d7be05ab77fc944a9dbef2b6575d4caa920d08
2015-02-04 01:38:34 +00:00
Elliott Hughes a744b05984 Add missing <malloc.h> and <string.h> includes.
Change-Id: Ia41756e607663d056e7d2fdd7ecbec7e5841a913
2015-01-28 11:37:57 -08:00
Dragoslav Mitrinovic 8e8e8db549 logd: fix persistent blocking reader performance
logd suffers major performance degradation when persistent (blocking)
client reader connects to it (e.g. logcat). The root cause of the
degradation is that each time when reader is notified of the arrival
of new log entries, the reader commences its search for the new entries
from the beginning of the linked list (oldest entries first).

This commit alters the search to start from the end of the linked list
and work backwards. This dramatically decreases logd CPU consumption
when blocking reader is connected, and increases the maximum logging
throughput (before the logs start getting lost) by a factor ~ 20.

Change-Id: Ib60955ce05544e52a8b24acc3dcf5863e1e39c5c
2015-01-24 00:10:34 +00:00
Mark Salyzyn b5f6e45d6b logd: create private/android_logger.h
- create a structure to depict the private header
  expected at logd end of socket.
- utilize this new structure instead of unscalable
  byte stream technique used to unpack in logd.

Change-Id: I2d0e5c3531c279f2dc1fbd74807210ff8d804de0
2015-01-23 23:32:11 +00:00
TraianX Schiau da6495d06f logd: Fix pruning
In a scenario in which an on-line (blocking) client is running and
a clean is attempted (logcat -c), the following can be observed:

1) the on-line logger seems to freeze
2) any other clear attempt will have no effect

What is actually happening:

In this case prune function will "instruct" the oldest timeEntry
to skip a huge number (very close to ULONG_MAX) of messages, this
being the cause of 1.

Since the consumer thread will skip all the log entries, mStart
updating will also be skipped. So a new cleaning attempt will have
the same oldest entry, nothing will be done.

Fix description:
a. keep a separated skipAhead count for individual log buffers (log_id_t)
      LogTimeEntry::LogTimeEntry
      LogTimeEntry::FilterSecondPass
      LogTimeEntry::skipAhead
      LogTimeEntry::riggerSkip_Locked

b. update LogTimeEntry::mStart even if the current message is skipped
      LogTimeEntry::FilterSecondPass

c. while pruning, only take into account the LogTimeEntrys that are monitoring
   the log_id in question, and provide a public method of checking this.
      LogTimeEntry::isWatching
      LogTimeEntry::FilterFirstPass
      LogTimeEntry::FilterSecondPass

d. Reset the skip cont befor the client thtread starts to sleep, at this point
   we should be up to date.
      LogTimeEntry::cleanSkip_Locked
      LogTimeEntry::threadStart

Change-Id: I1b369dc5b02476e633e52578266a644e37e188a5
Signed-off-by: TraianX Schiau <traianx.schiau@intel.com>
2015-01-14 19:12:47 +00:00
Mark Salyzyn eb06de716b logd: auditd remove logDmesg method
- logDmesg method consumes considerable memory
  resources (typically 128KB depending on kernel)
- In the future (eg: klogd, syslogd) there may be need to
  feed multiple logs or threads with the retrieved data.
- By moving the actions of logDmesg into the mainline that
  instantiates the thread objects, we can leverage a single
  allocation of the the kernel log allocation.
- logDmesg (private) is replaced with log (public) which
  has a more useful and descriptive purpose for the class.

Change-Id: Ie2dd0370661493c1e596a7e486904a0e8caab9ff
2014-12-05 20:56:44 +00:00
Nick Kralevich c234a1b879 logd: throttle SELinux denials to 20/sec
Impose a limit of 20 selinux denials per second. Denials beyond
that point don't add any value, and have the potential to cause
crashes or denial of service attacks.

Do some other misc cleanup while I'm here.

Bug: 18341932
Change-Id: I6125d629ae4d6ae131d2e53bfa41e1f50277d402
2014-11-19 13:35:36 -08:00
Mark Salyzyn eae155e1ce logd: insert order for first entry
Change-Id: I39e8e6d32407a1796a0c3a121f9dc4dc5402c3df
2014-10-14 23:57:13 +00:00
Mark Salyzyn f48ea7c8dc logd: LogStatistics leak
- uid = -1 in subtract operation to match add

Change-Id: I7d0b85b5334c5264fd04309cb78e0c9aec0ad261
2014-10-06 22:43:56 +00:00
Mark Salyzyn df5aa61f05 logd: kill(0,0) issue
- Recognize pid=0 as special case (kernel or pre-init sourced)
  and refrain from treating it in the general case.

Bug: 17526159
Change-Id: I74796043ac34753c6dd10018719ebc0bcd94e012
2014-10-06 22:43:46 +00:00
Mark Salyzyn f5fc509589 logd: Add control statistics enable/disable.
- ro.build.type=user turn off statistics
- ro.config.low_ram=true turn off statistics
- logd.statistics override

Bug: 17526159
Bug: 17526187
Change-Id: I74796043ac34753c6dd10018719ebc0bcd94e00f
2014-10-06 22:42:47 +00:00
Mark Salyzyn e72c6e4366 logd: cleanup
- simplify access and control exposure to class list
- indent
- compile warning
- Follow standard naming convention for variables and methods
- merge common fragments
- Side Effects: none

Change-Id: I74796043ac34753c6dd10018719ebc0bcd94e010
2014-10-06 22:42:07 +00:00
Mark Salyzyn f669acb018 logd: in nonblocking read, sched_yield() synchronization
- sched_yield and lock synchronization in reader thread
  startup to give writer thread a chance to catch up

Bug: 17512203
Change-Id: I43cf0b4e2829b22b3ab4e537fa95ce13c76a869c
2014-09-29 17:40:10 +00:00
Mark Salyzyn 7ee2aef8e0 logd: auditd: report facility LOG_AUTH
Change-Id: Ie325e1b58f52b6c728d5cfd6f6b87287fcf32e10
2014-09-28 14:42:05 -07:00
Mark Salyzyn 6bdeee0ce6 logd: auditd: kmsg priority
Change-Id: I2016fe140e2daf6c69efbd10aef205fffb931aa1
2014-09-19 11:59:42 -07:00
Chih-Hung Hsieh 634118e261 logd: fix format-extra-args warning.
Bug: 17409250
Change-Id: Id50ebb57754b12d69ed605d0e2901b8e05c607c6
2014-09-12 23:37:42 +00:00
Arseniy Antonov c3ce224c62 Fix for parenthesis error
Added parentheses as it was suggested by compiler.
error: suggest parentheses around operand of '!' or change '&' to '&&' or '!' to '~' [-Werror=parentheses]

Change-Id: Ic3746d9804488411d10d460ddfda11f1fad8efef
Signed-off-by: Arseniy Antonov <arseniy.antonov@intel.com>
2014-09-03 17:29:20 +04:00
Mark Salyzyn a16f761faa logd: persistent reader threads
(cherry picked from commit c113c5813e)

Bug: 16822776
Change-Id: I5bea468a41089b51108880044f32e2b2df1278e7
2014-08-12 18:59:35 +00:00
Elliott Hughes e5a0f2064b Fix implicit declaration of function 'prctl' in logd.
Change-Id: I0f655ad15295739adb8f04ec62c88a220413875e
2014-07-18 17:39:41 -07:00
Mark Salyzyn 89472df4cd logd: test: disable kernel LOGGER
Bug: 15384806
Change-Id: I3072d6dd0618fa9ebe151c78a85eb121327f9df3
2014-07-10 13:31:57 -07:00
Mark Salyzyn 4da93e3438 Merge "logd: Allow apps to clear their UID-specific data" 2014-06-17 21:47:03 +00:00
Mark Salyzyn 6e9aaae9ed logd: test: deal with spam filter turned off
Change-Id: I759c65e869d0c7cd81063fa7993ba3d67ef9cd06
2014-06-18 11:12:27 -07:00
Mark Salyzyn 0c57bd864d logd: test: modernize
- Correct default tense for kernel or user logger
- harden code that parses the benchmark data
- give more meaning to the spam filter test

Change-Id: I97699037ca9d56718f53f08d22be79092fb431de
2014-06-18 11:12:27 -07:00
Mark Salyzyn 1a240b4790 logd: Allow apps to clear their UID-specific data
Bug: 13501501
Change-Id: Ia72e25fc19430ce63fb359cd9b3f0523d41f5aa8
2014-06-13 08:06:14 -07:00
Mark Salyzyn 8d7656b8c1 logd: logcat unexpected EOF on slow consoles
Change-Id: I3520aa05e080de85df352ef36ae16e0406311f3c
2014-06-06 15:08:37 +00:00
Mark Salyzyn e4369d68a2 logd: logcat: debuggerd: audit logs to events and main
- auditd spawn log copy to events and main
- logcat delete events as one of the default logs
- debuggerd do not collect events.
- squish multiple spaces
- switch from strcpy to memmove for overlapping buffers

BUG: 14626551
Change-Id: I89b30273ce931ed2b25a53ea9be48e77f4c1bbf4
2014-05-27 14:29:56 -07:00
Mark Salyzyn ab4b7308ec logd: used before set warning
Change-Id: I30c078f8b22ebe4dcb48d57adf88b3e51b4fa202
2014-05-23 09:48:09 -07:00
Mark Salyzyn bc741ac212 Merge "logd: logcat: debuggerd: auditd logs to events" 2014-05-16 20:34:18 +00:00
Mark Salyzyn 989980c55d logd: logcat: debuggerd: auditd logs to events
- auditd switch to recording logs to events log id
- logcat add events as one of the default logs
- debuggerd collect events log as well.

ToDo: debuggerd & bugreport collect intermixed logs.

BUG: 14626551
Change-Id: I958f0e729b7596748be57488a38824db5645be7b
2014-05-15 08:50:50 -07:00
Mark Salyzyn 57a0af9313 init: logd: Allow Developer settings to adjust logd size
- AID_SYSTEM can set persist.logd.size
- AID_SYSTEM can issue command to /dev/socket/logd to
  change the runtime global log sizes.
- Add support for ro.logd.size.* as populated by BoardConfig.mk
- Limit size to maximum ~3% of physical memory.

Bug: 14563261
Bug: 14627052
Change-Id: I606420db2e9d9d032296e71149e4a5b20cbd1137
2014-05-13 07:42:15 -07:00
Mark Salyzyn 671e343c7d logd: logcat: Add persist.logd.size group of properties
- logd Add persist.logd.size (global), persist.logd.size.<logid>
- logcat report a more flexible multiplier in -g command.

Bug: 14563261
Bug: 14469172
Change-Id: Ie389caa14ad3ae3d4a3a3cc463425bb9dbc5e483
2014-05-06 14:58:41 -07:00
Mark Salyzyn 4141cb2391 logd: default off by-UID spam filter
Bug: 14469172
Change-Id: I37c8dbcea0490afb994cbe6f033591fea1c58bc8
2014-05-06 13:50:28 -07:00
Mark Salyzyn 6e2c0f7010 logd: build breakage in git_master-nova @ 1151709
Change-Id: Ic7920d1b0930155c20a896720a79af62f8acacc1
2014-05-02 15:47:24 -07:00
Mark Salyzyn c61895ac3a logd: turn on -Werror
- Deal with some -Wunused / -Wunused-variable issues

Change-Id: Ic86cd7b0680868ad0536198b71a34cb19134fc21
2014-05-02 14:16:00 -07:00
Mark Salyzyn 3cb5498785 logd: 64-bit compile issue
- ULONG_MAX is used against an uint32
- Add tv_sec_max and tv_nsec_max constants to log_time

Change-Id: Ic86cd7b0680868ad0536198b71a34cb19134fc22
2014-05-02 14:16:00 -07:00
Mark Salyzyn fdabe72876 logd: oneline statistics does not report PID gone
- oneline, add check for gone
- Add pidGone() method, which caches the gone status

Change-Id: I8f03daeafb58583c2b08ec6b540486c05da0872d
2014-05-01 10:20:51 -07:00
Mark Salyzyn e0fa291e89 logd: add logd.auditd property
- permit us a mechanism to disable auditd
- standardize property boolean

Bug: 14275676
Change-Id: I76f245c6aee511ed44274159e0ea55915b484dda
2014-04-29 07:25:27 -07:00
Mark Salyzyn 8daa9af02d logd: add thread setname
- permits easier determination of logd thread at
  fault in a stack trace from debuggerd.

Bug: 14275676
Change-Id: Iac2c523147e2bcce34ab7ddcecd02582c5fa7cc0
2014-04-29 00:37:37 +00:00
Mark Salyzyn e821dace41 logd: enable UID spam filter and test
Change-Id: Ie9c7a744e341e6e64afa14973d4d095717c97933
2014-04-25 11:03:58 -07:00
Mark Salyzyn c8a576c637 logd: Statistics improvements
- Drop in-place sorting.
- Add fast-track for uid sizes and elements.
- Add sort() for per-uid and per-pid statistics.

Change-Id: Ib8655d4cc0b3bd4e87534522987bcc7fc7a5251a
2014-04-25 10:46:27 -07:00
Mark Salyzyn 72031ab878 Merge "logd: liblog: ALOG silently drops excessively long messages" 2014-04-24 23:12:57 +00:00
Mark Salyzyn 8444eb81b3 logd: liblog: ALOG silently drops excessively long messages
Bug: 14280914
Change-Id: Idddd29784625094e54144953c29f64692a88eee9
2014-04-24 15:49:57 -07:00
Mark Salyzyn 7260a5c348 logd: Add README.property
Change-Id: I89da10a73470b037f1811a9890d959cae186f257
2014-04-24 15:07:54 -07:00
Mark Salyzyn 696817d352 liblog: Statistics truncated to 16384 bytes
- if network read/write broken up, reassemble the pieces.
- Use a 20ms poll to check if a new fragment has been
  sent by the other side.
- fixup logd-unit-tests to take a (simplified) fragment
  from the liblog changes.

Bug: 14164765
Change-Id: I98ff87888c119e1e8349717646d0f733e8971bc8
2014-04-21 17:07:25 -07:00
Mark Salyzyn 3ad0af6d04 Merge "logd: liblog: logcat: debuggerd: Add LOG_ID_CRASH" 2014-04-17 23:12:10 +00:00
Mark Salyzyn dfc47e8685 logd: libsysutils: logd startup outside init environment
Change-Id: I3426b6c3eebdd0c8258e966dcaaaa2825d7a23d1
2014-04-17 16:14:24 +00:00
Mark Salyzyn 99f47a9e7c logd: liblog: logcat: debuggerd: Add LOG_ID_CRASH
Change-Id: Iea453764a1a4fc1661f461f10c641c30150e4d20
2014-04-16 15:38:50 -07:00
Mark Salyzyn d3ba665020 logd: Add logd_test
* Test statistics output, all four main log id types
  are checked to be present, and output is properly
  formatted with header (ascii new-line terminated
  byte count) and trailer (newline formfeed) frames.
* On dev build test dgram_qlen, circumvent if
  liblog benchmarks executed first though.
* Test if kernel and user space loggers are
  present, and if any content has leaked to both.
* test benchmark and use it to verify worst UID pruning

Change-Id: I890c89f64825956f40108de806ed750b0170ac6c
2014-04-15 10:21:57 -07:00
Mark Salyzyn 8e72c5384b logd: Statistics: Add up to 10 chattiest clients
- embellish pid name
- >1% of the logs
- fix PID gone detection (EPERM)
- some ? : cleanup

Change-Id: Iba3b306e1fee43e53931165819a398693910514b
2014-04-10 08:59:53 -07:00
Mark Salyzyn 472e04f2b4 Merge "logd: liblog: logcat: enable prune features for user" 2014-04-09 23:33:28 +00:00
Greg Hackmann 9101878dd0 Merge "logd: fix LogStatistics::format freeing wrong pointer" 2014-04-07 23:18:58 +00:00
Greg Hackmann 239605ef64 logd: fix LogStatistics::format freeing wrong pointer
Fixes a leak and use-after-free if a client passes in an
already-allocated target buffer (currently none do)

Change-Id: Ie20c30bd5ba31e1b6db73885bd9d2ebd7e40dc27
Signed-off-by: Greg Hackmann <ghackmann@google.com>
2014-04-07 15:17:30 -07:00
Mark Salyzyn 0bb5288c0e logd: Turn off default prune by worst UID
- Algorithm places too much weight on stack dumps

Change-Id: Ibb31a59892870397b780965ec791f47a4ce87eaf
2014-04-07 12:17:19 -07:00
Mark Salyzyn e9bebd0eb1 logd: auditd: add logd.auditd.dmesg property
Change-Id: If4a579c2221eec99cf3f6acf59ead8c2d5230517
2014-04-07 10:51:00 -07:00
William Roberts 29d238d2a8 logd: selinux auditd initial commit
Initial commit for an audit daemon that writes kernel audit
messages to the Android logger. The daemon searches dmesg
for all lines that contain "audit" and writes them. Then
receiving the messages from the netlink socket.

It also formats the messages so they are compatable with
ausearch (type=<t> <m> format)

Modified: Mark Salyzyn <salyzyn@google.com>

- do not start auditd
- merge into logd, stripping unnecessary file logging.
- Convert headers and code to support C++
- Fix bugs in libaudit
- squash timestamp (replace with 0.0) due to duplication
- squash pid due to duplication
- squash comm due to duplication

Change-Id: I421bcf33e7e670d596628b1b5c7c25536ce2d3fe
2014-04-07 10:51:00 -07:00
Mark Salyzyn 4ba0387af5 logd: Add pidToUid helper
Change-Id: I23ebae1957c027bff6cbc2573a227bf0c44c08a2
2014-04-07 10:51:00 -07:00
Mark Salyzyn 9a03863e88 logd: Add pidToName helper
Change-Id: Idd8e804ab65feb8dc432150ae701464de1ad5302
2014-04-07 10:50:55 -07:00
Mark Salyzyn 765f782891 logd: transitory reader thread create should be detached
Bug: 13692914
Change-Id: Ia7306e6e844ae9f17c6f594bf6b25f3148652570
2014-04-04 13:12:56 -07:00
Mark Salyzyn 1c95047939 logd: liblog: logcat: enable prune features for user
- Enable whitelist, blacklist and logsize tuneables for user

Change-Id: Id0c283844c71c5263a8cfbebf2e550f7ac415858
2014-04-02 13:12:04 -07:00
Mark Salyzyn ca4c63d799 logd: build breakage in user
build breakage in branch: git_klp-dev-gpl @ 1101951

Change-Id: I4c9681bf4c97368ec19f5a79fef134aba937f337
2014-04-01 16:57:29 -07:00
Mark Salyzyn e457b74ce6 logd: Add minimum time bucket statistics
* Only enabled for dev builds because halves performance.
- Used to establish if /proc/sys/net/unix/max_dgram_qlen
  is appropriate for the platform
  (see logd/LogStatistics.cpp comments)
- enabled if logd.dgram_qlen.statistics is not zero/empty

Change-Id: Ib8ecfeb75b6f0f954c099a6b1032b8afb5efcbd4
2014-04-01 14:38:52 -07:00
Mark Salyzyn c66607a8aa Merge "logd: compile warning message" 2014-03-24 21:56:34 +00:00
Mark Salyzyn dbb708a0a1 logd: compile warning message
Change-Id: I7ddd010031299395cad96f1743d7a7e6a6866b4c
2014-03-21 15:48:05 -07:00
Mark Salyzyn 22e287df0d logcat: white and blacklist failure
- logcat improperly squashes out count
- logcat test enhanced to catch failure
- logd places entries in ascending sorted order

Change-Id: If39d8e2c40852b520c98e7840034ab63c2538e5d
2014-03-21 15:43:31 -07:00
Mark Salyzyn b992d0d7d3 logd: liblog: Thread IDs missing from logcat -v thread
- stuff caller's thread id into the packet.

Bug: 13568206
Change-Id: I02d0cdf9b1d9e839ff8969f591db42dfe6e4cc95
2014-03-21 10:37:44 -07:00
Mark Salyzyn b059cf53c9 logd: Logging is truncated
- stack buffer undersized

Bug: 13514601
Change-Id: I3e5dda2daa7531ddcacece33dfdecaa28dcca765
2014-03-18 16:03:34 -07:00
Mark Salyzyn a1c60cf80d logd: Find log time for arbitrary time to tail
- prototype to evaluate the increase in complexity or
  performance impact.

Change-Id: I4e815d74c023092fbb75055d260f75de57ad6522
2014-03-14 10:24:19 -07:00
Mark Salyzyn fa3716b250 logd: liblog: logcat: Arbitrary time to tail
Change-Id: I10e8d92c933e31ee11e78d2d1114261a30c4be0e
2014-03-14 10:23:51 -07:00
Mark Salyzyn dfa7a07f5b logd: liblog: logcat: Add LogWhiteBlackList
- liblog android_logger_get_log_size and android_logger_get_readable_size
  adjusted to return long instead of int because of -G flag extending range

NB: ifdef'd only for userdebug and eng builds

- liblog Add android_logger_[sg]et_prune_list and android_logger_set_log_size
- logcat Add -P, -p and -G flags
- logd Add LogWhiteBlackList and configurable log size

(cherry picked from commit 18a5432158)

Change-Id: I1572338c1b34bd968ad7867857ef708156ec3b6a
2014-03-13 14:48:07 -07:00
Mark Salyzyn 64d6fe9362 logd: prune by worst offending UID
(cherry picked from commit 3c4919e474)

Change-Id: I39965007569123ff5eebe01b5bfa555bbcb2dfe7
2014-03-13 14:48:02 -07:00
Mark Salyzyn 34facab86b logd: liblog: logcat: Add Statistics
- logd add statistical collection and formatting
- liblog add android_logger_get_statistics call
- logcat add -S flag
- logcat add -b all

(cherry picked from commit 51a29c8dc4)

Change-Id: I521753b1969ecd4590c956aeeb1557d101059d67
2014-03-13 14:47:58 -07:00
Mark Salyzyn 7e2f83c0bc logd: liblog: 64-bit issues
- structure packing
- move towards log_time from struct timespec
- extend log_time to cover differences between
  log_time and struct timespec

Change-Id: I106ed0b609917306d170044054b5b32645f2a295
2014-03-05 16:12:28 -08:00
Mark Salyzyn 581edc1b6c set /proc/sys/net/unix/max_dgram_qlen to large value
- init: set /proc/sys/net/unix/max_dgram_qlen to 300
- libsysutils: Add listen backlog argument to startListener
- logd: set listen backlog to 300

Change-Id: Id6d37d6c937ba2d221e76258d89c9516619caeec
2014-03-03 12:48:00 -08:00
Mark Salyzyn 4381d43848 logd: Always build logd
- build breakage aosp-idea133 @ 1049092

Change-Id: Iabac6009567926137f455044b33ab5d67924896d
2014-02-28 15:29:36 -08:00
Mark Salyzyn 154f4608aa liblog: enable logging to logd.
* Modify liblog to send all messages to the new syslog user
  space daemon.

Original-Change-Id: I0ce439738cd921efb2db4c1d6a289a96bdbc8bc2
Original-Change-Id: If4eb0d09409f7e9be3eb4bb7017073dc7e931ab4
Signed-off-by: Nick Kralevich <nnk@google.com>

* Add a TARGET_USES_LOGD make flag for BoardConfig.mk to manage
  whether logd is enabled for use or not.
* rename syslog to logd to avert confusion with bionic syslog
* Add fake log support back in
* prefilter for logging messages from logd
* Fill in timestamps at logging source
* update abstract log reader
* switch from using suffix for id to v3 format
* log a message when creating devices that a deprecated interface
  is being utilized.

Signed-off-by: Mark Salyzyn <salyzyn@google.com>

(cherry pick from commit 099e2c1f6f706a8600c1cef74cce9066fc315480)

Change-Id: I47929a5432977a1d7235267a435cec0a7d6bd440
2014-02-28 13:49:11 -08:00
Mark Salyzyn 1114f18065 logd: institute getGroups for socket credentials
(cherry pick from commit 4d851290fc81eb36d2fcf76e6f06213a28b877f5)

Change-Id: Ib8be84d2a3f873e91fb1495df439a498f395c137
2014-02-28 13:48:33 -08:00
Martijn Coenen 89343c1113 logd: Remove $(KERNEL_HEADERS) include.
Points to wrong kernel-headers dir in multi-arch
builds.

(cherry picked from commit 920a5629ff)

Change-Id: I5055bf5cd80119a34f0a6b0872ffb0a422a06baf
2014-02-27 07:57:05 -08:00
Mark Salyzyn 882f856668 logd: Add SCHED_BATCH to reduce priority
(cherry picked from commit 85620a1bff)

Change-Id: Id25cdef0dd7f8c5e154a0da2db2dd0554849babc
2014-02-26 09:55:03 -08:00
Mark Salyzyn c03e72cc1c logd: Adjust to match defacto coding style
(cherry picked from commit c46f77bd2a)

Change-Id: I80685cdc7116e10c5a5a77abe856fd96804f9117
2014-02-26 09:54:23 -08:00
Mark Salyzyn 740f9b4f97 logd: prune more aggressively when over the top
(cherry picked from commit 63c15d5061)

Change-Id: I929dddc7da048c032fb791c7af23f215f8856bf3
2014-02-26 09:53:55 -08:00
Mark Salyzyn 0175b0747a logd: initial checkin.
* Create a new userspace log daemon for handling logging messages.

Original-Change-Id: I75267df16359684490121e6c31cca48614d79856
Signed-off-by: Nick Kralevich <nnk@google.com>

* Merge conflicts
* rename new syslog daemon to logd to prevent confusion with bionic syslog
* replace racy getGroups call with KISS call to client->getGid()
* Timestamps are filed at logging source
* insert entries into list in timestamp order
* Added LogTimeEntry tail filtration handling
* Added region locking around LogWriter list
* separate threads for each writer
* /dev/socket/logd* permissions

Signed-off-by: Mark Salyzyn <salyzyn@google.com>

(cherry picked from commit 3e76e0a497)

Author: Nick Kralevich <nnk@google.com>
Change-Id: Ice88b1412d8f9daa7f9119b2b5aaf684a5e28098
2014-02-26 09:52:35 -08:00