Commit Graph

340 Commits

Author SHA1 Message Date
Josh Gao a227c19ef2 Merge changes Id8d9fa6c,I47d3ad50,I7cebbf75,Id10e320a
* changes:
  adbd: avoid compiling more code in the daemon.
  adb: don't run all of the tests again over TCP in coverage.
  adbd: respect ADB_TRACE on host adbd.
  adb: mark kMaxProcessNameLength as constexpr.
2020-05-12 05:32:48 +00:00
Treehugger Robot 8987016b1e Merge changes I24e70952,I652560ad
* changes:
  adb: use transport-id to make coverage script faster.
  adb: add `adb transport-id` command.
2020-05-12 02:22:27 +00:00
Josh Gao 3441f232ab adb: add `adb transport-id` command.
Make it possible to safely script waiting for a device to go away after
root/unroot.

Test: manual
Change-Id: I652560ad1d88a5c6a1e8ea5787c3cf165c82aa63
2020-05-05 14:43:53 -07:00
Joshua Duong 077ac11106 [adb client] Fix mdns discovery service registry.
We were getting stale service ip addresses because we weren't
destroying the sdref correctly.

Also, we were leaking the ResolvedServices when removing it from the
ServiceRegistry. Converted them to smart pointers to fix that.

Bug: 153343580

Test: test_adb.py
Change-Id: Ib7c1dbf54937d4ac6d9885cb5f7289bef616d12e
2020-05-05 00:21:12 -07:00
Josh Gao 9f3064f26f adbd: avoid compiling more code in the daemon.
Strip out more code that has no meaning on device, to "improve"
coverage.

Test: test_device.py over TCP
Change-Id: Id8d9fa6cc6c6c30773f67303bcc89e6d60824700
2020-04-27 14:38:28 -07:00
Joshua Duong 3eea62432d Merge "[adb] Disable _adb._tcp. service auto-connect by default." 2020-04-24 05:38:22 +00:00
Treehugger Robot abc40c7c5d Merge "Using ABB for install-multi." 2020-04-24 01:15:49 +00:00
Joshua Duong 2eedc2303b [adb] Disable _adb._tcp. service auto-connect by default.
Added ADB_MDNS_AUTO_CONNECT envionment variable to control which
services to allow auto-connect. By default, only _adb-tls-connect
services can auto-connect, since these services only auto-connect once
paired. _adb services will try to auto-connect to every service found.

Bug: 152636135

Test: ADB_TRACE=1 adb server nodaemon | grep whitelist
Test: ADB_TRACE=1 ADB_MDNS_AUTO_CONNECT=adb adb server nodaemon | grep whitelist
Test: ADB_TRACE=1 ADB_MDNS_AUTO_CONNECT=adb,adb-tls-connect adb server nodaemon | grep whitelist
Test: ADB_TRACE=1 ADB_MDNS_AUTO_CONNECT=adb-tls-connect adb server nodaemon | grep whitelist
Test: ADB_TRACE=1 ADB_MDNS_AUTO_CONNECT=0 adb server nodaemon | grep whitelist
Test: ADB_TRACE=1 ADB_MDNS_AUTO_CONNECT=1 adb server nodaemon | grep whitelist

Change-Id: Ie562ea24fea3d6d96e67b376a0523b09e2778eb7
2020-04-23 16:55:52 -07:00
Alex Buynytskyy a59e9b4e70 Using ABB for install-multi.
ABB uses single shared CMD for all operations which improves
reliability.

Bug: b/153486595
Test: atest adb_test adbd_test fastdeploy_test
Change-Id: I1e3da63882c980811ed2e9f5556732b24a041ce5
2020-04-23 16:30:49 -07:00
Josh Gao 8a9277a243 adb: split transport_local.cpp into client/daemon.
Improve coverage some more.

Test: ./coverage/gen_coverage.sh && ./coverage/report.sh
Change-Id: I014d7ad25b2793d5d836ce5e526d657d46687ea4
2020-04-22 23:15:02 -07:00
Joshua Duong 13c639e0bb [adb client] Add "mdns services" command.
This command list all discovered mdns services, so we
can connect via service name later on.

Bug: 152521166

Test: 'adb mdns services'
Test: test_adb.py
Change-Id: I23d42a7933e67a65bd0c9924afd6abe5915c0a11
2020-04-22 08:43:15 -07:00
Joshua Duong 504d393176 [adb client] Add "adb mdns check" command.
This command will check if the mdns daemon is available on the host
machine.

Bug: 152510294

Test: pkill -9 mdnsd; adb mdns check; mdnsd; adb mdns check;
Test: test_adb.py
Change-Id: If644678a339763817a8a7adcbdc545626d161aba
2020-04-22 08:43:15 -07:00
Greg Kaiser 89fe376d13 [adb data server] Initialize variable
servingComplete_ was left uninitialized and only set to 'true'
in the code.  We initialize it to the 'false' state to avoid
uninitialized references in SkipToRequest().

Bug: 150865433
Test: TreeHugger
Change-Id: Ia8a4d7135c432eb657543c5498fc9dbe8f4718b6
Merged-In: Ia8a4d7135c432eb657543c5498fc9dbe8f4718b6
2020-04-20 11:06:13 -07:00
Treehugger Robot 22cc277bfe Merge "adb: change mdns tls service names (RFC 6763)." 2020-04-18 01:37:41 +00:00
Joshua Duong 81f2db4d6a adb: change mdns tls service names (RFC 6763).
Even though mdnsresponder seems to allow us to use _adb_secure_connect
and _adb_secure_pairing as service names, these names violate the syntax
outlined in RFC6763, and may not be compatible with other dns-sd
implementations.

Also address some comments from a previous CL.

Bug: 154268895

Test: atest adbd_test
Change-Id: Ia872e976fc4276587b500a827a41d46d9dc755dd
2020-04-17 16:38:34 -07:00
Josh Gao fb4616b200 adb: fix feature sets on devices that don't have them.
Previously, we weren't distinguishing between nonexistent devices and
devices that don't support features, resulting in lots of stuff being
broken for pre-N devices.

Bug: http://b/154272158
Test: adb shell on a JB emulator
Change-Id: I0058b2a60678e1ad0503f5abac30157be976c9c4
2020-04-16 19:39:05 -07:00
Elliott Hughes 7d9a7a3b6e macOS build fix: no off64_t on darwin.
Darwin has always had a 64-bit off_t, and never added an off64_t.

Test: builds on mac
Change-Id: I4b244dda94c67da76c9e0f47b333c2734f5a94af
2020-04-14 15:34:01 -07:00
Alex Buynytskyy 82c1c9711d [incfs] Stream the hash tree for incremental installation
Instead of sending the whole tree upfront use the same
streaming interface for it as for the data blocks

This improves installation speed by almost 200ms,
650-800ms -> 500-600ms

Bug: 153696423
Test: manual, adb install in various configurations

Change-Id: Ia83de2af54ca0b1969397514ea5d761719af9055
Merged-In: Ia83de2af54ca0b1969397514ea5d761719af9055
2020-04-13 13:00:01 -07:00
Yurii Zubrytskyi 6fc26dff7c [adb] Don't copy features set on each get()
The features are already cached in a static and don't change.
Let's return the cached set instead of copying it every time.

Bug: 150183149
Test: manual
Change-Id: Ifdca852cc3b32e09e50ea4771f7878987c46cce8
2020-04-09 22:26:48 -07:00
Josh Gao 89cce05891 Merge changes from topic "adbd_lz4"
* changes:
  adb: add dry-run option to push/sync.
  adb: implement LZ4 compression.
  adb: fix use of wrong union variant.
  adb: fix front_size, front_data.
  adb: add interfaces for Encoder/Decoder.
2020-04-07 21:30:30 +00:00
Josh Gao ba6d1da8ea adb: add option to disable kill-server.
In the post-apocalypse, it's increasingly common for people to use ssh
port forwarding to use an adb client with a remote adb server. This
results in `adb kill-server` being catastrophic, because the client has
no way of restarting the server on the other end. Android Studio in
particular has an unforunate habit of trying to manage adb's life cycle
such that starting or exiting Studio will result in a kill-server.

Add the ADB_REJECT_KILL_SERVER environment variable which ignores
kill-server, to make this scenario a bit better.

Bug: http://b/152251952
Test: ADB_REJECT_KILL_SERVER=1 adb start-server; adb kill-server
Change-Id: I5533a6dcbdb9220526a6fcf9ca31d9fcef1cec17
2020-04-06 14:33:14 -07:00
Josh Gao 5949fccc42 adb: add dry-run option to push/sync.
Make it easier to benchmark file sync performance by ignoring the file
system.

Bug: https://issuetracker.google.com/150827486
Test: test_device.py
Change-Id: Icfa4b28eb5206f1914c0c163833d070a3748c3ea
2020-04-02 17:11:08 -07:00
Josh Gao ec44d35fde adb: implement LZ4 compression.
Add support for LZ4 compression, which compresses and decompresses far
more quickly than brotli, at the cost of worse compression ratio.

`adb sync -d system` speeds (in MB/s) on aosp_blueline-eng:

           none    brotli    lz4
USB 3.0     120       110    190
USB 2.0      38        75     63

Bug: https://issuetracker.google.com/150827486
Test: python3 -m unittest test_device.FileOperationsTest{Uncompressed,Brotli,LZ4}
Change-Id: Ibef6ac15a76b4e5dcd02d7fb9433cbb1c02b8382
2020-04-02 17:11:07 -07:00
Josh Gao 49ba558aa8 adb: add interfaces for Encoder/Decoder.
More groundwork to support more compression algorithms.

Bug: https://issuetracker.google.com/150827486
Test: python3 -m unittest test_device.FileOperationsTest{Uncompressed,Brotli}
Change-Id: I638493083b83e3f6c6854b631471e9d6b50bd79f
2020-04-02 17:11:07 -07:00
Josh Gao 6e697f2f8c adb: move things around to prepare to add LZ4.
Bug: https://issuetracker.google.com/150827486
Test: m adb com.android.adbd.apex
Change-Id: I038d1df4f179ddb23102016d781c9bd90da5a546
2020-04-01 13:09:29 -07:00
Yurii Zubrytskyi f8dc296dfd Merge "adb: fix apex installation." 2020-03-31 16:56:07 +00:00
Josh Gao 3c1803cab5 adb: fix apex installation.
Incremental doesn't support apex installation, but we were exiting
instead of returning -1 to fall back to regular installation.

Bug: http://b/151900478
Test: manual
Change-Id: Id27009250090a65fbb45bb65fc39c1799bf1f861
2020-03-31 02:08:14 +00:00
Josh Gao 0871824de6 Move adbd's legacy USB implementation to fastboot.
This code path is effectively dead in adbd, and fastboot's dependency on
libadbd makes it hard to refactor adbd's dependencies.

Bug: http://b/150317254
Test: built and flashed aosp_walleye-eng
Change-Id: I5118136d32fdcbbd011559ed0a4a71e1dc7bf064
2020-03-30 16:43:06 -07:00
Songchun Fan 26a8ed983c Merge "[adb] use zip iteration with functor" 2020-03-28 03:03:07 +00:00
Songchun Fan 0f11895157 [adb] use zip iteration with functor
Test: manual
BUG: 151676293
Change-Id: I738811bb5ff3ecc86ae49aa64df29427fb4ec69a
2020-03-27 18:38:50 -07:00
Treehugger Robot 551130fc2b Merge "Revert "Revert "Delay initial accept() until server initialized""" 2020-03-28 01:27:38 +00:00
Daniel Colascione c2612cd775 Revert "Revert "Delay initial accept() until server initialized""
This reverts commit b71b5af783.

Test: actually build device and host adb this time
Change-Id: I9f9375aa83c9d023a1e26d2f95676b734e12f926
2020-03-27 20:51:19 +00:00
Yurii Zubrytskyi 4532ea881a [adb] Use incremental installation by default
This CL turns on the incremental installation for all
"adb install ..." commands where no explicit mode has been set.
To disable this, set the ADB_INSTALL_DEFAULT_INCREMENTAL
environment variable to 0/n/no/false. Unset to enable back

+ improve the install command argument parsing a bit: allow
  --wait for all installation modes, --incr is enough for
  an incremental install (and --no-incr to disable it)

Bug: 150183149
Test: adb install with different apks and command line switches
Change-Id: I1a237f34b70d920146746ab16104e28ef555a5fd
2020-03-27 12:30:29 -07:00
Greg Kaiser 030f1effdf adb: Avoid extra string construction
Test: TreeHugger
Change-Id: Ie03062a17b98866f1bc419a7e509461cf4e6bb80
2020-03-27 06:10:11 -07:00
Yurii Zubrytskyi a1f41fc6f1 Merge "[adb] generic "cast_handle_to_int" function" 2020-03-27 07:03:22 +00:00
Yurii Zubrytskyi e3e64b8f54 [adb] generic "cast_handle_to_int" function
Now one can use it without platform #ifdefs

Bug: 150183149
Test: compiles
Change-Id: I6280dcdd2faaff33014f11237d9d04231ac16b82
2020-03-26 22:25:46 -07:00
Josh Gao 82f715b7b5 adb: fix bugreport status line.
The status line char* was getting converted to bool and used as the
compression argument, and the actual status line was defaulting to
nullptr. This didn't actually really change anything, since we only use
the name during transfer, and not for the final message at the end, so
the output is slightly uglier for 50 milliseconds.

This also had the side effect of turning on compression for the
already-compressed bugreport zip, which costs a bit of throughput with
brotli.

Bug: https://issuetracker.google.com/150827486
Test: adb bugreport foo.zip
Change-Id: I74bed9c09442c655a956eb1e8318c0123632ef4c
2020-03-26 21:10:31 -07:00
Josh Gao 4dc73c0b58 Merge "adb: implement compression for file sync." 2020-03-26 20:34:17 +00:00
Josh Gao 939fc19aee adb: implement compression for file sync.
This improves performance when syncing by up to 2x (remote cuttlefish
goes from 11.9 MB/s to 21.3 MB/s, blueline over USB 2.0 from 36 MB/s
to 70 MB/s).

This results in a slight drop in push speeds over USB 3.0 (125 -> 115
MB/s on blueline), presumably because we're compressing and extracting
on only a single thread, but the gains over lower bandwidth transports
make this worth it to submit this now and parallelize later.

Bug: https://issuetracker.google.com/150827486
Test: ADB_COMPRESSION={0, 1} test_device.py (with new/old adbd)
Change-Id: Ic2a0c974f1b6efecda115f87d336e3caac810035
2020-03-26 13:31:32 -07:00
Joshua Duong 1b273ed0e3 Pass the os handle to the tls connection.
This fixes the tls connection failure on Windows.

Bug: 150719467

Test: 'adb pair', 'adb connect' on Windows host machine.
Test: atest adb_tls_connection_test
Change-Id: I54b8945543ad8b430510fa51dd7bea64a119454f
2020-03-26 02:16:01 +00:00
Yurii Zubrytskyi 47a58ecd34 [adb] file sync performance on Windows
Print not more often than once a 100ms - it is smooth enough
and speeds up transfer even more on Windows, where a single
line output may take up to 5ms.
An added benefit is getting rid of some extra heap allocation
and string formatting when in the end the identical message
filtering would've dropped the line anyway. This is also
significantly more expensive on Windows.

Bug: 151900478
Test: manual, push/pull a file and a directory

Change-Id: I9038729e8a01d5f93fd301beaeb8a086f5039b77
Merged-In: I9038729e8a01d5f93fd301beaeb8a086f5039b77
2020-03-24 10:28:24 -07:00
Alex Buynytskyy 8539004821 Adding block kind to protocol for future streaming of a tree.
Bug: b/152050621
Test: adb install --incremental megacity.apk
Change-Id: I0c977080475088bf046a521a85f78595aac994f0
Merged-In: I0c977080475088bf046a521a85f78595aac994f0
2020-03-24 10:28:19 -07:00
Yurii Zubrytskyi b0fa864cc5 [adb] Print fewer progress messages for push/pull
Windows console IO is terribly slow. Reducing the number of
printed progress messages speeds up the transfer rate
from 80 to 130 MB/s on Windows laptop

Bug: 151900478
Test: adb push/pull
Change-Id: I223284c8a662bd8f2b8ba280cdcc8c930d3e5205
Merged-In: I223284c8a662bd8f2b8ba280cdcc8c930d3e5205
2020-03-24 10:28:14 -07:00
Yurii Zubrytskyi 2488c21d8b [adb] Fix incremental installation on Windows
Use only the syscalls that work with the wrapped ADB fds, or
extract the native handles for the case when need to call one
not wrapped.

Bug: 151239696
Test: adb install --incremental <apk> on Windows
Change-Id: Ia6de620171ab696b8136dcb60a2b63af6f86419f
Merged-In: Ia6de620171ab696b8136dcb60a2b63af6f86419f
2020-03-24 10:28:03 -07:00
Songchun Fan c3eb3014f6 [adb incremental] send priority blocks first
Before this change, "Success" is returned after all data is streamed,
around 7.5 seconds for Megacity.

After this change, "Success" is returned in about 1.5 seconds, before
streaming finishes.

BUG: 151676293
Test: manual
Change-Id: Ifda7de48da8e82623c99ae0194f70cb162fd72fa
Merged-In: Ifda7de48da8e82623c99ae0194f70cb162fd72fa
2020-03-24 10:27:56 -07:00
Songchun Fan 23bd5529c1 [adb data server] wait for installation results before terminates
Currently the server often quits before installation finishes. As a
result, there is no difference in the commandline output between a
successful installation and a failed one.

Let adb client wait till installation fails or succeeds by parsing the
output from the inc-server process.

Test: $ adb install --incremental ~/Downloads/base.apk
Test: Performing Incremental Install
Test: Serving...
Test: All files should be loaded. Notifying the device.
Test: Failure [INSTALL_PARSE_FAILED_NOT_APK: Failed to parse /data/app/vmdl749343150.tmp/base.apk: Failed to load asset path /data/app/vmdl749343150.tmp/base.apk]
Test: Install command complete (ms: 91 total, 0 apk prep, 91 install)

BUG: b/150865433
Change-Id: Ie33505f9cc08fc6d60ad4a5d709526e7aa9a0ad1
Merged-In: Ie33505f9cc08fc6d60ad4a5d709526e7aa9a0ad1
2020-03-24 10:27:47 -07:00
Alex Buynytskyy eebf9f7670 Implement the new v4 signing scheme in adb
To be submitted along with changes in apksigner tool and the framework.
Merged to AOSP after that.

Test: adb install --incremental <apk>

Bug: b/151241461
Change-Id: I26e187f8e389e31e2759037057b96fc6c9cb1e94
Merged-In: I26e187f8e389e31e2759037057b96fc6c9cb1e94
2020-03-24 10:27:39 -07:00
Josh Gao 672cdfeeff adb: fix sync.
adbd's file sync service doesn't handle a full socket gracefully,
immediately terminating the service as soon as it fails to write a
response. This would generally be fine if the socket's buffer were as
large as it claims (212992 by default with a 64-bit kernel), but this
buffer size is a giant lie, as each write has 576 bytes of overhead
that's used up in the send buffer. When setting the send buffer size,
the kernel helpfully doubles the value to attempt to account for the
overhead, but when writing 8 byte responses, only 2% of the buffer
actually gets used for responses, so we run out of buffer after 364
files instead of the 26624 that would be expected.

Fix this by processing the responses as they become available, and
calculate a maximum limit to how many sends we dispatch before we stop
and wait for responses to come in.

Test: manually modified adbd to respond with giant error messages, and
      modified adb to not read responses until we choose to block
Change-Id: Ieb8c935662864211e2fd16c337ffed0992990086
2020-03-21 20:06:39 -07:00
Elliott Hughes 4fe694a9c1 Merge "adb: support wait-for- with multiple states." 2020-03-13 15:19:34 +00:00
Greg Kaiser edfec96679 adb: Avoid a couple std::string constructions
With some internal API changing from "const char*" to
"const std::string&", we can change a couple calling sites to
directly pass a std::string reference, instead of getting the
c_str() version of the string.  This avoids us creating a
temporary std::string for the call.

Test: TreeHugger
Change-Id: I2fe0760ce8bf7d352010a341005356c0a801d351
2020-03-11 07:07:55 -07:00