Commit Graph

3528 Commits

Author SHA1 Message Date
Yurii Zubrytskyi cf77acf723 Merge "[adb] file sync performance on Windows" into rvc-dev am: 34fdb226f5
Change-Id: Ib8a77d9db54ec0565798ea1fe80044d6e7bd04bc
2020-03-24 06:02:51 +00:00
Yurii Zubrytskyi 34fdb226f5 Merge "[adb] file sync performance on Windows" into rvc-dev 2020-03-24 05:47:13 +00:00
Josh Gao a8776b4913 adb: fix sync.
adbd's file sync service doesn't handle a full socket gracefully,
immediately terminating the service as soon as it fails to write a
response. This would generally be fine if the socket's buffer were as
large as it claims (212992 by default with a 64-bit kernel), but this
buffer size is a giant lie, as each write has 576 bytes of overhead
that's used up in the send buffer. When setting the send buffer size,
the kernel helpfully doubles the value to attempt to account for the
overhead, but when writing 8 byte responses, only 2% of the buffer
actually gets used for responses, so we run out of buffer after 364
files instead of the 26624 that would be expected.

Fix this by processing the responses as they become available, and
calculate a maximum limit to how many sends we dispatch before we stop
and wait for responses to come in.

Bug: http://b/150827486
Test: manually modified adbd to respond with giant error messages, and
      modified adb to not read responses until we choose to block
Change-Id: Ieb8c935662864211e2fd16c337ffed0992990086
(cherry picked from commit 672cdfeeff)
2020-03-23 17:36:15 -07:00
Josh Gao 65396f4734 adb: extract syncmsg structs.
Make it so that we can get the sizeof a member of syncmsg without having
an instance of syncmsg or doing something awful along the lines of
sizeof(reinterpret_cast<syncmsg*>(nullptr)->status).

Bug: http://b/150827486
Test: m adb adbd
Change-Id: I4830e7f90033c7706ff52cdd8d13e9cf40c73628
(cherry picked from commit eddae92928)
2020-03-23 17:36:06 -07:00
Anna Trostanetski 1c1d2bb7e0 Merge "Revert "Set apex_available property"" am: 447a1ead21 am: d731f3ed3b
Change-Id: I2352a891e0d1c762280b4b981f695189417b89e8
2020-03-23 15:00:32 +00:00
Anna Trostanetski 447a1ead21 Merge "Revert "Set apex_available property"" 2020-03-23 14:14:32 +00:00
Anna Trostanetski 56a712f6c9 Revert "Set apex_available property"
This reverts commit d25bb60e05.

Reason for revert: Breaks build_test b/152190149
Bug: 152190149

Change-Id: I6b7cb4aa4fc07fae648bcf4d9e8f60ea1c62fd9e
2020-03-23 12:02:53 +00:00
Jiyong Park 3ee525ee32 Merge "Set apex_available property" am: fb0b0d24f2 am: d0d864deef
Change-Id: I34d549ca369020acf0939cdbf1cd64fb59a8f5ad
2020-03-23 11:21:44 +00:00
Jiyong Park fb0b0d24f2 Merge "Set apex_available property" 2020-03-23 10:51:28 +00:00
Jiyong Park d25bb60e05 Set apex_available property
The marked library(ies) were available to the adbd APEX via the
hand-written whitelist in build/soong/apex/apex.go. Trying to remove the
whitelist by adding apex_available property to the Android.bp of the
libraries.

Exempt-From-Owner-Approval: already +2'ed by the owner (enh)

Bug: 150999716
Bug: 151398197
Test: m
Change-Id: I8b572e3c4e76bd10c0443a6c08b72e9519243ab5
2020-03-23 10:50:12 +00:00
Treehugger Robot 4b437076dc Merge "adb: fix sync." am: 4abafe2230 am: 6e9effd885
Change-Id: I8a405cf3f731c347bc8cb9f660624108785ec979
2020-03-23 06:36:48 +00:00
Treehugger Robot 141aa49eea Merge "adb: extract syncmsg structs." am: c549b122f4 am: a651084255
Change-Id: I492b747e5d49d8b537a834bc731edb36579b79d9
2020-03-23 06:36:29 +00:00
Treehugger Robot 4abafe2230 Merge "adb: fix sync." 2020-03-23 06:06:25 +00:00
Treehugger Robot c549b122f4 Merge "adb: extract syncmsg structs." 2020-03-23 06:01:26 +00:00
Josh Gao 672cdfeeff adb: fix sync.
adbd's file sync service doesn't handle a full socket gracefully,
immediately terminating the service as soon as it fails to write a
response. This would generally be fine if the socket's buffer were as
large as it claims (212992 by default with a 64-bit kernel), but this
buffer size is a giant lie, as each write has 576 bytes of overhead
that's used up in the send buffer. When setting the send buffer size,
the kernel helpfully doubles the value to attempt to account for the
overhead, but when writing 8 byte responses, only 2% of the buffer
actually gets used for responses, so we run out of buffer after 364
files instead of the 26624 that would be expected.

Fix this by processing the responses as they become available, and
calculate a maximum limit to how many sends we dispatch before we stop
and wait for responses to come in.

Test: manually modified adbd to respond with giant error messages, and
      modified adb to not read responses until we choose to block
Change-Id: Ieb8c935662864211e2fd16c337ffed0992990086
2020-03-21 20:06:39 -07:00
Yurii Zubrytskyi 05227d99e0 [adb] file sync performance on Windows
Print not more often than once a 100ms - it is smooth enough
and speeds up transfer even more on Windows, where a single
line output may take up to 5ms.
An added benefit is getting rid of some extra heap allocation
and string formatting when in the end the identical message
filtering would've dropped the line anyway. This is also
significantly more expensive on Windows.

Bug: 151900478
Test: manual, push/pull a file and a directory

Change-Id: I9038729e8a01d5f93fd301beaeb8a086f5039b77
2020-03-20 22:48:10 -07:00
Yurii Zubrytskyi b959a788b2 Merge "Adding block kind to protocol for future streaming of a tree." into rvc-dev am: c0a5aff0df
Change-Id: Ie9f752720280473a4634c184448cbc6b3a57812b
2020-03-20 22:27:09 +00:00
Yurii Zubrytskyi c0a5aff0df Merge "Adding block kind to protocol for future streaming of a tree." into rvc-dev 2020-03-20 22:20:57 +00:00
Alex Buynytskyy e7817e42d1 Adding block kind to protocol for future streaming of a tree.
Bug: b/152050621
Test: adb install --incremental megacity.apk
Change-Id: I0c977080475088bf046a521a85f78595aac994f0
2020-03-20 11:18:33 -07:00
Josh Gao 6e9bbfcc0e Merge "adbd: add apex_available and visibility for internal libs." am: 448519736b am: b802fd9b1c
Change-Id: I4f3ba750255be96a1f0a0b3a18d16b2266ccf9fa
2020-03-20 10:29:17 +00:00
TreeHugger Robot 97046b951c Merge "[adb] Print fewer progress messages for push/pull" into rvc-dev am: 934135d576
Change-Id: I3a05ae90ff85a979a1d4efc661e8c8a2f2aa13e7
2020-03-20 10:13:15 +00:00
TreeHugger Robot 934135d576 Merge "[adb] Print fewer progress messages for push/pull" into rvc-dev 2020-03-20 09:59:55 +00:00
TreeHugger Robot 50821ab80b Merge "[adb] Optimize fdevent machinery" into rvc-dev am: fd3b80c99c
Change-Id: I80e4f2baaacb88ec78a6fc0a557fe7d748536e0d
2020-03-20 06:47:42 +00:00
TreeHugger Robot fd3b80c99c Merge "[adb] Optimize fdevent machinery" into rvc-dev 2020-03-20 06:33:47 +00:00
Josh Gao eddae92928 adb: extract syncmsg structs.
Make it so that we can get the sizeof a member of syncmsg without having
an instance of syncmsg or doing something awful along the lines of
sizeof(reinterpret_cast<syncmsg*>(nullptr)->status).

Test: m adb adbd
Change-Id: I4830e7f90033c7706ff52cdd8d13e9cf40c73628
2020-03-19 17:24:15 -07:00
TreeHugger Robot 8d4383e728 Merge "[adb] Fix incremental installation on Windows" into rvc-dev am: ea1103cb1d
Change-Id: Id4fc8cd1430efd567f86acc22664c3f452e6fc6f
2020-03-19 22:19:48 +00:00
TreeHugger Robot ea1103cb1d Merge "[adb] Fix incremental installation on Windows" into rvc-dev 2020-03-19 22:05:53 +00:00
TreeHugger Robot 80afce0b66 Merge "[adb data server] Initialize variable" into rvc-dev am: 126a1c2dd0
Change-Id: If01b50552b08160da93b4b6b3880133d17b3e2ed
2020-03-19 17:32:33 +00:00
Yurii Zubrytskyi 67caf0ceef [adb] Print fewer progress messages for push/pull
Windows console IO is terribly slow. Reducing the number of
printed progress messages speeds up the transfer rate
from 80 to 130 MB/s on Windows laptop

Bug: 151900478
Test: adb push/pull
Change-Id: I223284c8a662bd8f2b8ba280cdcc8c930d3e5205
2020-03-19 00:21:08 -07:00
Yurii Zubrytskyi 76820308c5 [adb] Optimize fdevent machinery
- Use one fewer heap allocation per fdevent object
- Lazy-init the fdevent context

Bug: 151239696
Test: various adb commands on Win/Linux

Change-Id: Ic7de207b30495e618f187e097c0276ad42c34005
2020-03-18 23:20:00 -07:00
Yurii Zubrytskyi c3ac339daf [adb] Fix incremental installation on Windows
Use only the syscalls that work with the wrapped ADB fds, or
extract the native handles for the case when need to call one
not wrapped.

Bug: 151239696
Test: adb install --incremental <apk> on Windows
Change-Id: Ia6de620171ab696b8136dcb60a2b63af6f86419f
2020-03-18 22:28:01 -07:00
Josh Gao b567303330 adbd: add apex_available and visibility for internal libs.
abb links against libadbd_core for the shell protocol, and must be
on the system image because it links against cmd, so let's just
expose it to abb for now.

Bug: http://b/151398197
Test: treehugger
Change-Id: Id926bc4324d3259def21ea19d3bd72320311a6e3
2020-03-18 18:36:01 -07:00
Greg Kaiser 6a2c18a88f [adb data server] Initialize variable
servingComplete_ was left uninitialized and only set to 'true'
in the code.  We initialize it to the 'false' state to avoid
uninitialized references in SkipToRequest().

Bug: 150865433
Test: TreeHugger
Change-Id: Ia8a4d7135c432eb657543c5498fc9dbe8f4718b6
2020-03-18 06:18:10 -07:00
Yurii Zubrytskyi 4e7bede912 Merge changes Ifda7de48,Ie33505f9 into rvc-dev am: c2872d831d
Change-Id: I632830b0a15bdada757230c2a1d6ec4f48ce51ee
2020-03-18 03:38:50 +00:00
Songchun Fan 8cdefd4f95 [adb incremental] send priority blocks first
Before this change, "Success" is returned after all data is streamed,
around 7.5 seconds for Megacity.

After this change, "Success" is returned in about 1.5 seconds, before
streaming finishes.

BUG: 151676293
Test: manual
Change-Id: Ifda7de48da8e82623c99ae0194f70cb162fd72fa
2020-03-17 20:19:49 -07:00
Songchun Fan c4a9d16ef1 [adb data server] wait for installation results before terminates
Currently the server often quits before installation finishes. As a
result, there is no difference in the commandline output between a
successful installation and a failed one.

Let adb client wait till installation fails or succeeds by parsing the
output from the inc-server process.

Test: $ adb install --incremental ~/Downloads/base.apk
Test: Performing Incremental Install
Test: Serving...
Test: All files should be loaded. Notifying the device.
Test: Failure [INSTALL_PARSE_FAILED_NOT_APK: Failed to parse /data/app/vmdl749343150.tmp/base.apk: Failed to load asset path /data/app/vmdl749343150.tmp/base.apk]
Test: Install command complete (ms: 91 total, 0 apk prep, 91 install)

BUG: b/150865433
Change-Id: Ie33505f9cc08fc6d60ad4a5d709526e7aa9a0ad1
2020-03-17 20:19:47 -07:00
Automerger Merge Worker 6a0d14ac15 Merge "adbd: shrink libadbd.so, libadbd_services.so." am: cb76bf0558 am: e7d5ae30c6
Change-Id: I55dd7c315e2222584972ded39571d845a3f7fffa
2020-03-16 21:38:02 +00:00
Josh Gao c151a1ba28 adbd: shrink libadbd.so, libadbd_services.so.
We were ending up with multiple copies of the proto runtime in the
recovery image, when we need zero.

Before:
    784K	recovery/root/system/lib64/libadbd_services.so
    832K	recovery/root/system/lib64/libadbd.so

After:
    360K	recovery/root/system/lib64/libadbd_services.so
    344K	recovery/root/system/lib64/libadbd.so

Bug: http://b/150317254
Test: treehugger
Change-Id: I39fbb3959128994f0de2ae0ea47dbc0800c516fe
2020-03-16 11:30:09 -07:00
Automerger Merge Worker cde607bfed Merge "Implement the new v4 signing scheme in adb" into rvc-dev am: 742ef23ee9
Change-Id: I1f30ddb248fca223a47a532ad8d02e5dcc6a0c59
2020-03-14 02:35:08 +00:00
TreeHugger Robot 742ef23ee9 Merge "Implement the new v4 signing scheme in adb" into rvc-dev 2020-03-14 02:28:40 +00:00
Alex Buynytskyy 33ac1b05f9 Implement the new v4 signing scheme in adb
To be submitted along with changes in apksigner tool and the framework.
Merged to AOSP after that.

Test: adb install --incremental <apk>

go/apk-v4-signature-format

Bug: b/151241461
Change-Id: I26e187f8e389e31e2759037057b96fc6c9cb1e94
2020-03-13 11:04:17 -07:00
Automerger Merge Worker 1939280381 Merge "adb: support wait-for- with multiple states." am: 4fe694a9c1 am: a13edaac4e
Change-Id: Ia1b1aaf24eb18adf9d3905d6e43ec40367af366a
2020-03-13 15:43:21 +00:00
Elliott Hughes 4fe694a9c1 Merge "adb: support wait-for- with multiple states." 2020-03-13 15:19:34 +00:00
Jiyong Park a94047be70 Merge "Make libselinux a stub library" into rvc-dev 2020-03-13 05:16:49 +00:00
Jiyong Park 8ce4a3067c Make libselinux a stub library
libselinux is currently being copied to APEXes. This is risky because
the library is not designed to be portable; part of it is tied to the
specific version of the Android that it was developed for.

This change fixes the problem by declaring that the library supports
a stub with the list of C APIs that are included in the stub. Then there
is only one copy of libselinux in /system/lib and other APEXes use the
copy by dynamically linking to it.

Also, adbd no longer statically links to it, because doing so brings
libselinux in it.

Bug: 151053366
Test: m com.android.adbd. It doesn't include libselinux in it.
Test: m com.android.adbd-deps-info. then inspect
out/soong/com.android.adbd-deps-info.txt. The dependency to libselinux
is shown as '(external)'.

Exempt-From-Owner-Approval: cherry-pick from AOSP

Merged-In: If418cbe3abdeacb759d59052e6dca4c2067678dd
(cherry picked from commit 3ffdad0cb5)
Change-Id: If418cbe3abdeacb759d59052e6dca4c2067678dd
2020-03-13 05:16:28 +00:00
Jiyong Park c2a354be68 Merge "don't include liblog to APEXes" into rvc-dev 2020-03-12 04:10:15 +00:00
Automerger Merge Worker 18ee520e93 Merge "Make libselinux a stub library" am: b407502c98 am: 9281e97450
Change-Id: Ica4f4de8425f84eb6efb1321df1f5f78e3393464
2020-03-12 03:31:03 +00:00
Treehugger Robot b407502c98 Merge "Make libselinux a stub library" 2020-03-12 02:57:14 +00:00
Automerger Merge Worker 5dc6290c24 Merge "adb: Avoid a couple std::string constructions" am: 2442e64416 am: c2d303231d
Change-Id: I768760f4439b5403025bb3a91a8acedcf6fd4d47
2020-03-12 00:41:33 +00:00
Jiyong Park 3ffdad0cb5 Make libselinux a stub library
libselinux is currently being copied to APEXes. This is risky because
the library is not designed to be portable; part of it is tied to the
specific version of the Android that it was developed for.

This change fixes the problem by declaring that the library supports
a stub with the list of C APIs that are included in the stub. Then there
is only one copy of libselinux in /system/lib and other APEXes use the
copy by dynamically linking to it.

Also, adbd no longer statically links to it, because doing so brings
libselinux in it.

Bug: 151053366
Test: m com.android.adbd. It doesn't include libselinux in it.
Test: m com.android.adbd-deps-info. then inspect
out/soong/com.android.adbd-deps-info.txt. The dependency to libselinux
is shown as '(external)'.

Change-Id: If418cbe3abdeacb759d59052e6dca4c2067678dd
2020-03-12 00:02:18 +00:00