api(trace): introduce artifacts options
This introduces launch({ artifactsPath }) and newContext({ relativeArtifactsPath, recordTrace }) options.
- artifactsPath option controls the directory where all artifacts go. If not passed, artifacts are not collected.
- relativeArtifactsPath can be used to put context-specific artifacts into a subfolder. If not passed, shared artifactsPath is used.
- recordTrace controls trace recording.
We also expose trace types under playwright/types/trace.d.ts.
In the follow up:
- videos will be put into artifactsPath;
- downloads will be put into artifactsPath, or keep using existing downloadsPath when artifactsPath is not specified.
This patch:
- moves ffmpeg binaries from `//bin/` to `//third_party/ffmpeg`
- adds [COPYING.GPLv3](https://github.com/FFmpeg/FFmpeg/blob/master/COPYING.GPLv3)
ffmpeg license
- changes npm packaging to include `//third_party/ffmpeg` only in `playwright` and `playwrihgt-chromium` a
This patch:
- stop relying on stdout from `//packages/build_package.js` to get
output paths. This was a legacy code that's not needed anymore
- remove stray output from `//packages/build_package.js`
Today we have a bunch of types used by the d.ts file that are not exported. We don't want to export them because it would greatly increase our semver API surface area, so this patch inlines them. Now users will not see names of types they can't import.
This introduces basic tracing enabled in our tests.
What is captured:
- network resources;
- snapshots at the start of most actions;
- snapshot after the test failure.
How this integrates with test runner:
- context fixture calls private method context._initSnapshotter() and uses Tracer to trace all events;
- all tests share a single test-results/trace-storage directory to store blobs;
- each test has its own trace file.
- npm run show-trace opens a bare-minimum trace viewer that renders snapshots.
This patch moves fixtures.js to base.fixtures.ts that sits next to tests. All tests get an extra import to get the base fixtures (both types and implementations).
This patch:
- refactors script to output per-browser package dependencies. This is to aid with
per-browser docker setup and Playwright github action.
- sorts package maps for both Ubuntu 18.04 and Ubuntu 20.04 alphabetically (and removes a stray dependency)
References #2926
This makes it easier to reason about our packages.
The only difference is what each package downloads.
When the browser is not downloaded, it will fail to launch.
Each browser gets a 'download' attribute in the browser.json file.
We now commit protocol.ts files during the roll.
New utils/roll_browser.js helps with that.
This makes our installation very shallow:
- build installer;
- download browsers.
This patch:
- makes environment a simple class with optional methods `beforeEach`, `afterEach`, `beforeAll`, `afterAll`, `globalSetup` and `globalTeardown`
- removes capability to have multiple hooks of the same name inside suite
- removes default environment for test. (`dit` now adds a `TraceTestEnvironment` to the test)
- extracts all environments that we use in our tests in `//test/environments.js`
Downsides:
- we no longer know hook locations for the environments. This, however, should not be a big deal since stack traces (if any) will still point into it.
- this also regresses hook locations for suites for simplicity. We can get them back, but it shouldn't be pressing since we now have only one hook of each kind in every suite.
This patch:
- drops dependency on playwright-core in all our packages. Instead of
the dependency, packages are now built with `//packages/build_package.sh`
script.
- unifies `browsers.json` - now there's a single `//browsers.json` file
that is used to manage browser revisions.
This patch temporary switches canary publishing to `--dryn-run` from CI/CD so that we
can verify that it does sane things.
We'll unify all our package management scripts under `//packages/` in a
follow-up.
Fixes#2268
This patch:
- teaches test runner to understand custom argument spelling, e.g. `--file=evalu` and `-j10`
- fixes `--file` filter to actually focus file paths instead of focusing
all tests with given path
There are a few places in the API where we use objects as maps. This patch adds them to docs and the types.
For `env`, we accept booleans and numbers as well because they are often used for their string values.
As of today, we have tooling in place that makes sure that our
main README.md **roughly** represents the state of the last release:
- browser versions and browser badges are those that we released last
- links to our API point to the last-released API version
This tooling, however, relies on the fact that every release is a sequence
of two consecutive commits:
- one commit that sets `package.json` version to a released version
- the following that "bumps" version to `-post`.
This release process is very unfortunate, because:
- it made releasing from branch impossible
- it required "freezing" commits to the master branch
This patch removes all the tooling and transitions `README.md` to always
represent tip-of-tree state. We will fully rely on
`https://playwright.dev` to show versioned docs.
This patch:
- removes releasing from Travis CI
- sets up a new GH Action that releases @next version from tip-of-tree
Once this GH Action proves to be working, we'll setup a `publish_release.yml`
workflow that will be triggered only by **release** github events
and that will publish released version with `LATEST` tag.
NOTE: this workflow does not actually run publishing - we're doing
`--dry-run` for now to see how it works in
`//utils/publish_all_packages.sh`.
This ensures we get a proper error when something goes wrong. Should
also help with producing the right error code in the case of internal error.
Drive-by: fix location issue which manifests on the bots.
Drive-by: remove the use of Array.prototype.flat to make it work on bots.
I was playing around today with different ways of changing the way we export types for #1439. I looked at only exporting 'Parameter' types, only exporting 'Return' types, only exporting a manual list of 'important' types. They all had different pros and cons, and it was very difficult to settle on a good answer.
For now, let's not export any parameter/return types. We can whitelist some types upon user request. I'm thinking `LaunchOptions` and `AccessibilitySnapshot` could be quite useful. We can always add new types after 1.0, but we can't remove them.
The patch looks funny because this was my original intent for the types, but I didn't know I had to `export {}` to tell typescript that my .d.ts shouldn't export everything.
This encapsulates selectors logic in one place, in a preparation for more complex scenarios like main-world selectors or piercing frames.
Note: we had `Page.fill should wait for visible visibilty` test, but we do not actually wait for visible in page.fill(). It happened to pass due to lucky evaluation order.
References #1316.
Adds logging comments to the doclint tests, and adds a new one with a bulleted list in a comment. Lists can only be used in comments where extra properties would be unexpected.
This generates typescript definitions based on the api.md, instead of autogenerating them from the typescript source code.
Now types
- only include the public api
- work with older versions of typescript
- include descriptions
- are more consistent
- are more complete
#6
This patch:
- removes `browserType.downloadBrowserIfNeeded()` method. The method
turned out to be ill-behaving and cannot not be used as we'd like to (see #1085)
- adds a `browserType.setExecutablePath` method to set a browser
exectuable.
With this patch, we take the following approach towards managing browser downloads:
- `playwright-core` doesn't download any browsers. In `playwright-core`, `playwright.chromium.executablePath()` returns `null` (same for firefox and webkit).
- clients of `playwright-core` (e.g. `playwright` and others) download browsers one way or another.
They can then configure `playwright` with executable paths and re-export the `playwright` object to their clients.
- `playwright`, `playwright-firefox`, `playwright-chromium` and `playwright-webkit` download
browsers. Once browsers are downloaded, their executable paths are saved to a `.downloaded-browsers.json` file. This file is read in `playwright/index.js` to configure browser executable paths and re-export the API.
- special case is `install-from-github.js` that also cleans up old browsers.
Currently in our API `?` means null, but sometimes it means optional. Linting optional/nulls with this patch is required for #1166 to land nicely.
Previously, return types were not being linted in `api.md`. This is fixed, along with many broken return types.
This patch considers `?` to mean nullable, and has some heuristics to determine optionality. I believe this to be the minimal patch needed to unblock #1166. After it lands, we can consider changing the api docs to hopefully remove some heuristics and strangeness.
Currently it was leading to an error if I tried to build it with a root user on a Linux environment. So in the end the type generation was just skipped:
```
22:51:55.757 $ node install-from-github.js
22:51:55.813 Building playwright...
22:52:14.094 chromium downloaded to /11994741/playwright/.local-chromium/linux-747023 22:52:14.123 Failed to launch browser!
22:52:14.123 [0310/215214.121201:ERROR:zygote_host_impl_linux.cc(89)] Running as root without --no-sandbox is not supported. See https://crbug.com/638180.
22:52:14.123 TROUBLESHOOTING: https://github.com/Microsoft/playwright/blob/master/docs/troubleshooting.md
```
Maybe we should also exit the script with an error code if the protocol generation was not successful.
This change introduces a TestWorker that can be in a certain state,
meaning it has run some beforeAll hooks of a certain test suite stack.
TestWorker can be created at any time, which allows for a number of features:
- don't run hooks for suites with no runnable tests;
- smarter test distribution (and possibility for variuos strategies);
- recovering from hook failures and test failure by creating a new worker;
- possible isolation between workers by running them in separate environments.
This patch sets continuous deployment of playwright packages
from tip-of-tree. These packages are released under the "@next",
so it should be possible to install tip-of-tree version of Playwright
from NPM using
```
npm i playwright@next
```
A few technical details:
- This uses travis to drive deployments
- This deploys from master and only if all browsers pass all tests on linux
- This is an attempt to deploy just two packages: `playwright-core` and `playwright-firefox`. If this works out, I'll follow-up with other packages as well
This patch re-implements matching and reporting for test runner.
Among other improvements:
- test failures now show a short snippet from test
- test failures now explicitly say what received and what was expected
- `expect.toBe()` now does text diff when gets strings as input
- `expect.toEqual` now does object diff
The plan is to collect logs for the whole test run and upload it later on
using https://github.com/actions/upload-artifact
Produced log size:
- 163MB (8.5MB zipped) for Chromium: `DEBUG=* npm run unit 2>log`
- 135MB (4.8MB zipped) for WebKit: `DEBUG=*,-pw:wrapped* npm run wunit 2>log`
- 29MB (4.0MB zipped) for Firefox: `DEBUG=* npm run funit 2>log`
This introduces a handful of new markdown preprocessor commands to
insert browser versions:
- `GEN:chromium-version-if-release` - inserts current Chromium version
if we're doing release; noop otherwise.
- `GEN:firefox-version-if-release` - inserts current Firefox version
if we're doing release; noop otherwise.
And to generate badge links:
- `GEN:chromium-version-badge-if-release` - inserts current Chromium version
badge if we're doing release; noop otherwise.
- `GEN:firefox-version-badge-if-release` - inserts current Firefox version
badge if we're doing release; noop otherwise.
This doesn't touch webkit at all - we're yet to figure what to do with
webkit version.
NOTE: versions will be updated only once we release. This way our
README.md always represents last released version.
I enabled vscode autocomplete in our test files. Typechecking had too many errors to enable, but it caught some real bugs that I will fix in a follow up.
This patch contains:
* `test/types.d.ts` - d.ts file for our test runner.
* `test/tsconfig.json` - typescript project for our tests.
* JSDoc header in all specs to mark the describe as a TestSuite
* Drive-by fix of a launcher test that was using `if` instead of `it`
* Some drive-by fixes of unimpactful typos in tests.
This is mostly copied from Puppeteer's troubleshooting.md, with everything under a Chromium header and a few tweaks to make things make sense for Playwright.
We can add other browsers to this document once we figure out how to troubleshoot them.
Fixes#583
Dmitry Gozman