Commit Graph

19 Commits

Author SHA1 Message Date
Patrice Arruda 589826bc1e Add and update comments in ui/metrics/metrics.go.
The metrics.go had missing comments and also did minor code refactoring
to be more in golang best practices.

Bug: b/172917718
Test: m nothing.
Change-Id: Id63e4af3f27b582fc13ef3f988ac5c8707fb9adf
2020-12-15 23:36:01 +00:00
Patrice Arruda 5c58b6410c Remove soong_metrics_proto from the import of the protobuf file.
No needed to define soong_metrics_proto since it is defined in
the Android.bp file.

Bug: b/172917718
Test: m nothing
Change-Id: Ib5f4cd6d06c08660803cf070da544e9c2d1dce1f
2020-12-15 19:41:40 +00:00
Patrice Arruda 48d55ad2db Add setup_tools performance metrics to Soong protobuf file.
The setup_tools metrics has been missing since day one and is
needed to track down slowness of setting up the build system.

Bug: b/175706305
Test: m nothing and ran the printproto command to check that the
      setup_tools field has been set.
Change-Id: Idc8a38fa830e0a906170feeede63435715eaa05b
2020-12-15 19:39:09 +00:00
Patrice Arruda 8a44a37623 Add and update comments in ui/metrics/event.go
The event.go code was refactored to follow the best practices
of golang code style.

Bug: b/172918681
Test: go test ./ui/metrics
Change-Id: Ic57d7cd576a9b146530ae37f03f4f5679d6cffef
2020-12-15 03:01:24 +00:00
Patrice Arruda 33e7f6a0ca Remove unnecessary break and default statements in metrics.go file.
break and default statements are not needed inside of a switch
statement.

Bug: b/173028918
Test: USE_BAZEL m nothing
Change-Id: I14aa8cbae3a9b3c859f147c5134600f1e75c6a89
2020-11-13 13:15:09 -08:00
Patrice Arruda b7cf9ba175 Add tracer to runBazel function.
Added a tracer named "bazel" in runBazel function to capture topline
metrics when running bazel.

Bug: b/173028918
Test: * USE_BAZEL=1 m nothing and checked soong_metrics file using
        printproto command to verify that bazel topline metrics
	were captured.
Change-Id: I5cb0728a4da48545dc701cc79627c318d5d9a29d
2020-11-13 13:14:51 -08:00
Patrice Arruda e92c30dff1 Capture the build command that executed the build system.
We need to capture the user build command to understand what additional
arguments is passed in to Soong. This data is useful to know which
targets are being executed and have statistics on top build targets
are specified.

Bug: b/169453712
Test: ran "m blueprint_tools", "m nothing", "lunch 1" and checked
      in soong_metrics proto.
Change-Id: I7ab94dcf037211f1abffe6afd012b03ffbf99756
2020-10-29 11:01:32 -07:00
Patrice Arruda 457c5d3426 Rename time to event in Soong metrics system.
Several structs and files are named time which records the total
execution of a traceable event. Soon, resource information of an
executed process is collected during a build event so a more generic
name is applied.

Bug: b/169453825
Test: go test and m nothing. Checked the soong_metrics protobuf.
Change-Id: I65a782603530d51018d6cc1192c18cfeef7566a0
2020-10-21 08:54:56 -07:00
Patrice Arruda 3edfd48ef4 Collect total number of CPU cores and available RAM for each build.
Collect the total number of CPU cores and available RAM for each build
as part of the system resource info. This is to know what kind of
build machines that the developers are using.

Bug: b/169453825
Test: m nothing and ran printproto on soong_metrics to validate the
      data.
Change-Id: I8617bdb9490b37d7689abd27d349af1a651533db
2020-10-13 23:58:41 +00:00
Patrice Arruda 4fb8adcc4c Collect the hostname of the machine as a metric.
The hostname is collected during a build to determine where the user
is build from (from office workstation, cloudtop, etc...). This is a
useful metrics for RBE related builds.

Bug: b/169395325
Test: m nothing, ran printproto to verify that soong_metrics has the
      hostname.
Change-Id: I3e5d64f948ae43c5dda60de1d4a52b187e36dff8
2020-10-12 22:44:27 +00:00
Patrice Arruda 9685036e6e Start collecting build configuration metrics.
This is an initial CL that collects the environment variables
USE_GOMA and USE_RBE. This is a needed metric to determine how
many users are currently using GOMA so we can help them migrate
to RBE.

Bug: b/163598127
Test: m nothing and checked soong_metrics
Change-Id: I39536eabad01aafbb3bd7182515d72230b36f658
2020-08-12 14:25:37 +00:00
Patrice Arruda add7ea9cc8 Set the OS name in the metrics protobuf file
There are cases where the developer is not building a product so
the metadata information is not being set. For now, set the OS
name so we can track which OS the user is using it.

Fixes: b/163118212
Test: m nothing and checked the blueprint
Change-Id: I9b521eee2d266b50625aa8e916ddaacb4e2feaea
2020-08-07 17:55:23 +00:00
Patrice Arruda 73c790f59e Use the build start time from Soong main UI for metrics build timestamp.
Currently, the build timestamp saved to the build_date_timestamp
field in the MetricsBase metrics protobuf usually comes from a text
file named build_date.txt. It may not be accurate for metrics purposes.
Instead, use the build start time from Soong main UI for a better
timestamp.

Bug: b/140638454
Test: Ran and checked the build_date_timestamp by running the printproto
      command.
Change-Id: I4d893fbbf9830ba21911e56ae13bc5272be47ae6
2020-07-14 17:23:23 +00:00
Patrice Arruda 62f1bf2378 Add RBE metrics dump in Soong UI.
From aosp/1329396, the RBE metrics protobuf file is part of the
metrics uploading process. The RBE metrics protobuf file is
generated by running the bootstrap shutdown command. A new function
named DumpRBEMetrics was written in order to generate the RBE metrics
protobuf file before sending to the uploading process.

Bug: b/140638454
Test: * Unit test cases
      * Ran RBE build on my local host and verified the
        metrics protobuf file is created.
      * Ran non-RBE build after RBE build and verified that
        the previous metrics protobuf file was deleted.

Change-Id: I4b8068905cb67c4b8c2d94793917b98974fed707
2020-07-09 21:06:22 +00:00
Colin Cross b72c909899 Collect metrics from inside soong_build
Collect the number of modules and variants and some basic statistics
on memory usage inside soong_build by writing out a proto that is
read back in by soong_ui.

Test: examine soong.log
Change-Id: I6926876377a4f6229cf41fdbf166ae03c885ea55
2020-02-25 20:50:00 +00:00
Colin Cross 74cda72f5e Add total build time to metrics
Determining the total build time from the existing metrics is
tricky because some of the metrics are overlapping.  For example,
the "soong" metric includes  "blueprint bootstrap",
"environment check", "minibp", "bpglob", "minibootstrap", and
"bootstrap".  Create a new singleton "total" metric that
covers the interesting parts of the build.

Test: m nothing
Change-Id: Iedcf5c9cf0d27252b694d35e6ff66ca1fcf734ac
2020-01-16 15:25:50 -08:00
Colin Cross d0be2109cf Add CUJ tests
Add a cuj_tests binary that runs through a sequence of critical
user journey builds and collects metrics.

Bug: 145688720
Test: cuj_tests
Change-Id: I6bb1a3fc4a458fd8cea7b7c924f565d0cf3e6df3
2019-12-05 11:11:37 -08:00
Patrice Arruda 0cc5b21516 Soong: Fix the package name in metrics.proto file
The metrics.proto needs to be imported to Google3 in order to unmarshal
the build metrics data from bigstore. Cleaned up the enum names to use
the Camel naming convention and renamed the build_metrics to
soong_metrics to be more specific to soong and to allow quering
soong_metrics files from bigstore.

Bug: b/135280521
Test: lunch andf m -j. Checked if soong_metrics was generated correctly
      by using printproto command.

Change-Id: I998c8d05db592e94a653d6ca32250b80df3c9b21
2019-06-14 15:27:46 -07:00
Nan Zhang 17f2767724 Generate build timing metrics to proto format file
Test: Dumped the text formated based metrics file to out dir,
and checked the file.
Bug: b/63815990

Change-Id: Iff476f72a0be74eb53b6b26ef468d11c0f24a404
2019-01-04 15:54:01 -08:00