Command-Line Arguments — web-platform-tests documentation
The wpt command-line application offers a number of features for interacting with WPT. The functionality is organized into “sub-commands”, and each accepts a different set of command-line arguments.
This page documents all of the available sub-commands and associated arguments.
usage:wpt-h{build-docs,test-jobs,make-hosts-file,regen-certs,update-built,tc-download,tc-decision,tc-sink-task,docker-run,docker-push,lint,manifest,manifest-download,test-paths,spec,serve,serve-wave,run,create,update-expectations,files-changed,tests-affected,install,rev-list,install-android-emulator,start-android-emulator,web-features-manifest}…
Sub-commands¶ build-docs¶ Build documentation
wptbuild-docs-h–typeTYPE–docker–serveSERVE
Named Arguments¶ –typeOutput type (default: html)
Default: “html”
–dockerRun inside the docs docker image
Default: False
–serveRun a server on the specified port (default: 8000)
test-jobs¶ List test jobs that should run for a set of commits
wpttest-jobs-h–all–includesINCLUDES…–jsonrevish
Positional Arguments¶ revishCommits to consider. Defaults to the commits on the current branch
Named Arguments¶ –allList all jobs unconditionally.
Default: False
–includesJobs to check for. Return code is 0 if all jobs are found, otherwise 1
–jsonOutput jobs as JSON, instead of one per line
Default: False
make-hosts-file¶ Output a hosts file to stdout
wptmake-hosts-file-haddress
Positional Arguments¶ addressAddress that hosts should point at
Default: “127.0.0.1”
regen-certs¶ Regenerate the WPT certificates
wptregen-certs-h–checkend-secondsCHECKEND_SECONDS–force
Named Arguments¶ –checkend-secondsThe number of seconds the certificates must be valid for
Default: 5184000
–forceRegenerate certificates even if not reaching expiry
Default: False
update-built¶ Update built tests
wptupdate-built-h–list–include{canvas,conformance-checkers,css-images,css-ui,css-writing-modes,fetch,html5lib,infrastructure,mimesniff,speculative-parsing}…
Named Arguments¶ –listList suites that can be updated and the related script files
Default: False
–includePossible choices: canvas, conformance-checkers, css-images, css-ui, css-writing-modes, fetch, html5lib, infrastructure, mimesniff, speculative-parsing
Suites to update (default is to update everything)
tc-download¶ Download logs from taskcluster
wpttc-download-h–refREF–artifact-nameARTIFACT_NAME–repo-nameREPO_NAME–token-fileTOKEN_FILE–out-dirOUT_DIR
Named Arguments¶ –refBranch (in the GitHub repository) or commit to fetch logs for
Default: “master”
–artifact-nameLog type to fetch
Default: “wpt_report.json.gz”
–repo-nameGitHub repo name in the format owner/repo. This must be the repo from which the Taskcluster run was scheduled (for PRs this is the repo into which the PR would merge)
Default: “web-platform-tests/wpt”
–token-fileFile containing GitHub token
–out-dirPath to save the logfiles
Default: “.”
tc-decision¶ Run the decision task
wpttc-decision-h–event-pathEVENT_PATH–dry-run–tasks-pathTASKS_PATH
Named Arguments¶ –event-pathPath to file containing serialized GitHub event
–dry-runDon’t actually create the tasks, just output the tasks that would be created
Default: False
–tasks-pathPath to file in which to write payload for all scheduled tasks
tc-sink-task¶ Run the sink task
wpttc-sink-task-h–github-checks-text-fileGITHUB_CHECKS_TEXT_FILEtaskstasks…
Positional Arguments¶ tasksA set of Taskcluster task ids to verify the state of.
Named Arguments¶ –github-checks-text-filePath to GitHub checks output file for Taskcluster runs
docker-run¶ Run wpt docker image
wptdocker-run-h–rebuild–checkoutCHECKOUT–privileged–tagTAG
Named Arguments¶ –rebuildForce rebuild of image
Default: False
–checkoutRevision to checkout in the image. If this is not supplied we mount the wpt checkout on the host as /home/test/web-platform-tests/
–privilegedRun the image in priviledged mode (required for emulators)
Default: False
–tagDocker image tag to use (default wpt:local)
Default: “wpt:local”
docker-push¶ Build and push wpt docker image
wptdocker-push-h–tagTAG–force
Named Arguments¶ –tagTag to use (default is taken from .taskcluster.yml)
–forceIgnore warnings and push anyway
Default: False
lint¶ Run the lint
wptlint-h–json–markdown–repo-rootREPO_ROOT–ignore-globIGNORE_GLOB–all–github-checks-text-fileGITHUB_CHECKS_TEXT_FILE-jJOBS–paths-filePATHS_FILEpaths…
Positional Arguments¶ pathsList of paths to lint
Named Arguments¶ –jsonOutput machine-readable JSON format
Default: False
–markdownOutput markdown
Default: False
–repo-rootThe WPT directory. Use this option if the lint script exists outside the repository
–ignore-globAdditional file glob to ignore (repeat to add more). Globs are matched against paths relative to REPO_ROOT using fnmatch, except that path separators are normalized.
–allIf no paths are passed, try to lint the whole working directory, not just files that changed
Default: False
–github-checks-text-filePath to GitHub checks output file for Taskcluster runs
-j, –jobsLevel to parallelism to use (defaults to 0, which detects the number of CPUs)
Default: 0
–paths-fileFile containing a list of files to lint, one per line
manifest¶ Update the MANIFEST.json file
wptmanifest-h-v-pPATH–tests-rootTESTS_ROOT-r–url-baseURL_BASE–no-download–cache-rootCACHE_ROOT–no-parallel
Named Arguments¶ -v, –verboseTurn on verbose logging
Default: False
-p, –pathPath to manifest file.
–tests-rootPath to root of tests.
Default: /home/runner/work/wpt/wpt
-r, –rebuildForce a full rebuild of the manifest.
Default: False
–url-baseBase url to use as the mount point for tests in this manifest.
Default: “/”
–no-downloadNever attempt to download the manifest.
Default: True
–cache-rootPath in which to store any caches (default <tests_root>/.wptcache/)
Default: “/home/runner/work/wpt/wpt/.wptcache”
–no-parallelDo not parallelize building the manifest
Default: True
manifest-download¶ Download recent pregenerated MANIFEST.json file
wptmanifest-download-h-pPATH–tests-rootTESTS_ROOT–force
Named Arguments¶ -p, –pathPath to manifest file.
–tests-rootPath to root of tests.
Default: /home/runner/work/wpt/wpt
–forceAlways download, even if the existing manifest is recent
Default: False
test-paths¶ Print test paths given a set of test ids
wpttest-paths-h-pPATH–src-rootSRC_ROOT–tests-rootTESTS_ROOT–no-update-r–url-baseURL_BASE–cache-rootCACHE_ROOT–jsontest_idstest_ids…
Positional Arguments¶ test_idsTest ids for which to get paths
Named Arguments¶ -p, –pathPath to manifest file.
–src-rootPath to root of sourcetree.
–tests-rootPath to root of tests.
Default: /home/runner/work/wpt/wpt
–no-updateDon’t update manifest before continuing
Default: True
-r, –rebuildForce a full rebuild of the manifest.
Default: False
–url-baseBase url to use as the mount point for tests in this manifest.
Default: “/”
–cache-rootPath in which to store any caches (default <tests_root>/.wptcache/)
Default: “/home/runner/work/wpt/wpt/.wptcache”
–jsonOutput as JSON
Default: False
spec¶ Update the SPEC_MANIFEST.json file
wptspec-h-v-pPATH–tests-rootTESTS_ROOT–url-baseURL_BASE–cache-rootCACHE_ROOT–no-parallel
Named Arguments¶ -v, –verboseTurn on verbose logging
Default: False
-p, –pathPath to manifest file.
–tests-rootPath to root of tests.
Default: /home/runner/work/wpt/wpt
–url-baseBase url to use as the mount point for tests in this manifest.
Default: “/”
–cache-rootPath in which to store any caches (default <tests_root>/.wptcache/)
Default: “/home/runner/work/wpt/wpt/.wptcache”
–no-parallelDo not parallelize building the manifest
Default: True
serve¶ Run wptserve server
wptserve-h–latencyLATENCY–configCONFIG_PATH–doc_rootDOC_ROOT–ws_doc_rootWS_DOC_ROOT–ws_extraWS_EXTRA–inject-scriptINJECT_SCRIPT–alias_fileALIAS_FILE–no-h2–webtransport-h3–exit-after-start–verbose
Named Arguments¶ –latencyArtificial latency to add before sending http responses, in ms
–configPath to external config file
–doc_rootPath to document root. Overrides config.
–ws_doc_rootPath to WebSockets document root. Overrides config.
–ws_extraPath to extra directory containing ws handlers. Overrides config.
Default:
–inject-scriptPath to script file to inject, useful for testing polyfills.
–alias_fileFile with entries for aliases/multiple doc roots. In form of /ALIAS_NAME/, DOC_ROOTn
–no-h2Disable the HTTP/2.0 server
–webtransport-h3Enable WebTransport over HTTP/3 server
Default: False
–exit-after-startExit after starting servers
Default: False
–verboseEnable verbose logging
Default: False
serve-wave¶ Run wptserve server for WAVE
wptserve-wave-h–latencyLATENCY–configCONFIG_PATH–doc_rootDOC_ROOT–ws_doc_rootWS_DOC_ROOT–ws_extraWS_EXTRA–inject-scriptINJECT_SCRIPT–alias_fileALIAS_FILE–no-h2–webtransport-h3–exit-after-start–verbose–report
Named Arguments¶ –latencyArtificial latency to add before sending http responses, in ms
–configPath to external config file
–doc_rootPath to document root. Overrides config.
–ws_doc_rootPath to WebSockets document root. Overrides config.
–ws_extraPath to extra directory containing ws handlers. Overrides config.
Default:
–inject-scriptPath to script file to inject, useful for testing polyfills.
–alias_fileFile with entries for aliases/multiple doc roots. In form of /ALIAS_NAME/, DOC_ROOTn
–no-h2Disable the HTTP/2.0 server
–webtransport-h3Enable WebTransport over HTTP/3 server
Default: False
–exit-after-startExit after starting servers
Default: False
–verboseEnable verbose logging
Default: False
–reportFlag for enabling the WPTReporting server
Default: False
run¶ Run tests in a browser
wptrun–channel{stable,release,beta,dev,canary,nightly,preview,experimental}–affectedAFFECTED–yes–install-browser–install-webdriver–logcat-dirLOGCAT_DIR-h–manifest-update–no-manifest-update–manifest-download–no-manifest-download–timeout-multiplierTIMEOUT_MULTIPLIER–run-by-dirRUN_BY_DIR-f–processesPROCESSES–max-restartsMAX_RESTARTS–no-capture-stdio–no-fail-on-unexpected–no-fail-on-unexpected-pass–no-restart-on-new-group–list-test-groups–list-disabled–list-tests–verify–verify-log-full–verify-repeat-loopVERIFY_REPEAT_LOOP–verify-repeat-restartVERIFY_REPEAT_RESTART–verify-no-chaos-mode|–verify-chaos-mode–verify-max-timeVERIFY_MAX_TIME–repeat-max-timeREPEAT_MAX_TIME–verify-no-output-results|–verify-output-results–test-types{crashtest,testharness,reftest,print-reftest,wdspec}…–subsuite-fileSUBSUITE_FILE–subsuiteSUBSUITES–small-subsuite-sizeSMALL_SUBSUITE_SIZE–includeINCLUDE–include-fileINCLUDE_FILE–excludeEXCLUDE–include-manifestINCLUDE_MANIFEST–test-groupsTEST_GROUPS_FILE–skip-timeout–skip-crash–skip-implementation-status{not-implementing,backlog,implementing}–enable-webtransport-h3–no-enable-webtransport-h3–tagTAGS–exclude-tagEXCLUDE_TAGS–default-exclude–debuggerDEBUGGER–debugger-argsDEBUGGER_ARGS–rerunRERUN–repeatREPEAT–repeat-until-unexpected–retry-unexpectedRETRY_UNEXPECTED–pause-after-test–no-pause-after-test–debug-test–pause-on-unexpected–no-restart-on-unexpected–symbols-pathSYMBOLS_PATH–stackwalk-binarySTACKWALK_BINARY–pdb–leak-check–no-leak-check–adb-binaryADB_BINARY–package-namePACKAGE_NAME–keep-app-data-directory–device-serialDEVICE_SERIAL–binaryBINARY–binary-argBINARY_ARGS–webdriver-binaryBINARY–webdriver-argWEBDRIVER_ARGS–metadataMETADATA_ROOT–testsTESTS_ROOT–manifestMANIFEST_PATH–run-infoRUN_INFO–product{android_webview,chrome,chrome_android,chrome_ios,chromium,edge,firefox,firefox_android,headless_shell,safari,sauce,servo,servodriver,opera,webkit,webkitgtk_minibrowser,wktr,epiphany,ladybird}–browser-versionBROWSER_VERSION–browser-channelBROWSER_CHANNEL–configCONFIG–install-fonts–no-install-fonts–font-dirFONT_DIR–inject-scriptINJECT_SCRIPT–headless–no-headless–instrument-to-fileINSTRUMENT_TO_FILE–suppress-handler-traceback–no-suppress-handler-traceback–ws-extraWS_EXTRA–debug-build|–release-build–total-chunksTOTAL_CHUNKS–this-chunkTHIS_CHUNK–chunk-type{none,hash,id_hash,dir_hash}–ssl-type{openssl,pregenerated,none}–openssl-binaryOPENSSL_BINARY–certutil-binaryCERTUTIL_BINARY–ca-cert-pathCA_CERT_PATH–host-key-pathHOST_KEY_PATH–host-cert-pathHOST_CERT_PATH–prefs-rootPREFS_ROOT–preload-browser–no-preload-browser–disable-e10s–disable-fission–stackfix-dirSTACKFIX_DIR–specialpowers-pathSPECIALPOWERS_PATH–setprefPREF=VALUE–reftest-internal–reftest-external–reftest-screenshot{always,fail,unexpected}–chaosCHAOS_MODE_FLAGS–setenvENV–user-stylesheetUSER_STYLESHEETS–enable-mojojs–mojojs-pathMOJOJS_PATH–enable-swiftshader–enable-experimental–no-enable-experimental–enable-sanitizer–reuse-window–sauce-browserSAUCE_BROWSER–sauce-platformSAUCE_PLATFORM–sauce-versionSAUCE_VERSION–sauce-buildSAUCE_BUILD–sauce-tagsSAUCE_TAGS…–sauce-tunnel-idSAUCE_TUNNEL_ID–sauce-userSAUCE_USER–sauce-keySAUCE_KEY–sauce-connect-binarySAUCE_CONNECT_BINARY–sauce-init-timeoutSAUCE_INIT_TIMEOUT–sauce-connect-argSAUCE_CONNECT_ARGS–github-checks-text-fileGITHUB_CHECKS_TEXT_FILE–webkit-portWEBKIT_PORT–kill-safari–log-rawLOG_RAW–log-unittestLOG_UNITTEST–log-xunitLOG_XUNIT–log-htmlLOG_HTML–log-machLOG_MACH–log-tbplLOG_TBPL–log-groupedLOG_GROUPED–log-chromiumLOG_CHROMIUM–log-wptreportLOG_WPTREPORT–log-wptscreenshotLOG_WPTSCREENSHOT–log-raw-levelLOG_RAW_LEVEL–log-mach-verbose–log-mach-levelLOG_MACH_LEVEL–log-mach-bufferLOG_MACH_BUFFER–log-mach-screenshot–log-mach-no-screenshot–log-tbpl-compact–log-tbpl-levelLOG_TBPL_LEVEL–log-tbpl-bufferLOG_TBPL_BUFFER–log-wptscreenshot-apiLOG_WPTSCREENSHOT_APIproducttest_list…
Positional Arguments¶ productBrowser to run tests in
test_listList of URLs for tests to run, or paths including tests to run. (equivalent to –include)
Named Arguments¶ –channelPossible choices: stable, release, beta, dev, canary, nightly, preview, experimental
Name of browser release channel (default: nightly). “stable” and “release” are synonyms for the latest browser stable release; “beta” is the beta release; “dev” is only meaningful for Chrome (i.e. Chrome Dev); “nightly”, “experimental”, and “preview” are all synonyms for the latest available development or trunk release. (For WebDriver installs, we attempt to select an appropriate, compatible version for the latest browser release on the selected channel.) This flag overrides –browser-channel.
Default: “nightly”
–affectedRun affected tests since revish
–yes, -yDon’t prompt before installing components
Default: True
–install-browserInstall the browser from the release channel specified by –channel (or the nightly channel by default).
Default: False
–install-webdriverInstall WebDriver from the release channel specified by –channel (or the nightly channel by default).
Default: False
–logcat-dirDirectory to write Android logcat files to
–manifest-updateRegenerate the test manifest.
–no-manifest-updatePrevent regeneration of the test manifest.
Default: True
–manifest-downloadAttempt to download a preexisting manifest when updating.
–no-manifest-downloadPrevent download of the test manifest.
Default: True
–timeout-multiplierMultiplier relative to standard test timeout to use
–run-by-dirSplit run into groups by directories. With a parameter,limit the depth of splits e.g. –run-by-dir=1 to split by top-leveldirectory
Default: False
-f, –fully-parallelRun every test in a separate group for fully parallelism.
Default: False
–processesNumber of simultaneous processes to use
–max-restartsMaximum number of browser restart retries
Default: 5
–no-capture-stdioDon’t capture stdio and write to logging
Default: False
–no-fail-on-unexpectedExit with status code 0 when test expectations are violated
Default: True
–no-fail-on-unexpected-passExit with status code 0 when all unexpected results are PASS
Default: True
–no-restart-on-new-groupDon’t restart test runner when start a new test group
Default: True
–verifyRun a stability check on the selected tests
Default: False
–verify-no-chaos-modeDisable chaos mode when running on Firefox
Default: True
–verify-chaos-modeEnable chaos mode when running on Firefox
Default: True
–verify-no-output-resultsPrints individuals test results and messages
Default: True
–verify-output-resultsDisable printing individuals test results and messages
Default: True
–debug-buildBuild is a debug build (overrides any mozinfo file)
–release-buildBuild is a release (overrides any mozinfo file)
Mode¶ –list-test-groupsList the top level directories containing tests that will run.
Default: False
–list-disabledList the tests that are disabled on the current platform
Default: False
–list-testsList all tests that will run
Default: False
–verify-log-fullOutput per-iteration test results when running verify
Default: False
–verify-repeat-loopNumber of iterations for a run that reloads each test without restart.
Default: 10
–verify-repeat-restartNumber of iterations, for a run that restarts the runner between each iteration
Default: 5
–verify-max-timeThe maximum number of minutes for the job to run
–repeat-max-timeThe maximum number of minutes for the test suite to attempt repeat runs
Default: 100
Test Selection¶ –test-typesPossible choices: crashtest, testharness, reftest, print-reftest, wdspec
Test types to run
Default: {‘crashtest’, ‘testharness’, ‘reftest’, ‘print-reftest’, ‘wdspec’}
–subsuite-filePath to JSON file containing subsuite configuration
–subsuiteSubsuite names to run. Runs all subsuites when omitted.
–small-subsuite-sizeMaximum number of tests a subsuite can have to be treated as small subsuite.Tests from a small subsuite will be grouped in one group.
Default: 50
–includeURL prefix to include
–include-fileA file listing URL prefix for tests
–excludeURL prefix to exclude
–include-manifestPath to manifest listing tests to include
–test-groupsPath to json file containing a mapping {group_name: test_ids}
–skip-timeoutSkip tests that are expected to time out
Default: False
–skip-crashSkip tests that are expected to crash
Default: False
–skip-implementation-statusPossible choices: not-implementing, backlog, implementing
Skip tests that have the given implementation status
–enable-webtransport-h3Enable tests that require WebTransport over HTTP/3 server (default: false)
–no-enable-webtransport-h3Do not enable WebTransport tests on experimental channels
Default: True
–tagLabels applied to tests to include in the run. Labels starting dir: are equivalent to top-level directories.
–exclude-tagLabels applied to tests to exclude in the run. Takes precedence over –tag. Labels starting dir: are equivalent to top-level directories.
–default-excludeOnly run the tests explicitly given in arguments. No tests will run if the list is empty, and the program will exit with status code 0.
Default: False
Debugging¶ –debuggerrun under a debugger, e.g. gdb or valgrind
–debugger-argsarguments to the debugger
–rerunNumber of times to re run each test without restarts
Default: 1
–repeatNumber of times to run the tests, restarting between each run
Default: 1
–repeat-until-unexpectedRun tests in a loop until one returns an unexpected result
–retry-unexpectedMaximum number of times to retry unexpected tests. A test is retried until it gets one of the expected status, or until it exhausts the maximum number of retries.
Default: 0
–pause-after-testHalt the test runner after each test (this happens by default if only a single test is run)
–no-pause-after-testDon’t halt the test runner irrespective of the number of tests run
Default: True
–debug-testRun tests with additional debugging features enabled
Default: False
–pause-on-unexpectedHalt the test runner when an unexpected result is encountered
Default: False
–no-restart-on-unexpectedDon’t restart on an unexpected result
Default: True
–symbols-pathPath or url to symbols file used to analyse crash minidumps.
–stackwalk-binaryPath to stackwalker program used to analyse minidumps.
–pdbDrop into pdb on python exception
Default: False
–leak-checkEnable leak checking for supported browsers (Gecko: enabled by default for debug builds, silently ignored for opt, mobile)
–no-leak-checkDisable leak checking
Android specific arguments¶ –adb-binaryPath to adb binary to use
–package-nameAndroid package name to run tests against
–keep-app-data-directoryDon’t delete the app data directory
Default: False
–device-serialRunning Android instances to connect to, if not emulator-5554
Default:
Configuration¶ –binaryDesktop binary to run tests against
–binary-argExtra argument for the binary
Default:
–webdriver-binaryWebDriver server binary to use
–webdriver-argExtra argument for the WebDriver binary
Default:
–metadataPath to root directory containing test metadata
–testsPath to root directory containing test files
–manifestPath to test manifest (default is ${metadata_root}/MANIFEST.json)
–run-infoPath to directory containing extra json files to add to run info
–productPossible choices: android_webview, chrome, chrome_android, chrome_ios, chromium, edge, firefox, firefox_android, headless_shell, safari, sauce, servo, servodriver, opera, webkit, webkitgtk_minibrowser, wktr, epiphany, ladybird
Browser against which to run tests
–browser-versionInformative string detailing the browser release version. This is included in the run_info data.
–browser-channelInformative string detailing the browser release channel. This is included in the run_info data.
–configPath to config file
–install-fontsInstall additional system fonts on your system
–no-install-fontsDo not install additional system fonts on your system
Default: True
–font-dirPath to local font installation directory
–inject-scriptPath to script file to inject, useful for testing polyfills.
–headlessRun browser in headless mode
–no-headlessDon’t run browser in headless mode
Default: True
–instrument-to-filePath to write instrumentation logs to
–suppress-handler-tracebackDon’t write the stacktrace for exceptions in server handlers
–no-suppress-handler-tracebackWrite the stacktrace for exceptions in server handlers
Default: True
–ws-extraExtra paths containing websockets handlers
Test Chunking¶ –total-chunksTotal number of chunks to use
Default: 1
–this-chunkChunk number to run
Default: 1
–chunk-typePossible choices: none, hash, id_hash, dir_hash
Chunking type to use
SSL/TLS¶ –ssl-typePossible choices: openssl, pregenerated, none
Type of ssl support to enable (running without ssl may lead to spurious errors)
–openssl-binaryPath to openssl binary
Default: “openssl”
–certutil-binaryPath to certutil binary for use with Firefox + ssl
–ca-cert-pathPath to ca certificate when using pregenerated ssl certificates
–host-key-pathPath to host private key when using pregenerated ssl certificates
–host-cert-pathPath to host certificate when using pregenerated ssl certificates
Gecko-specific¶ –prefs-rootPath to the folder containing browser prefs
–preload-browserPreload a gecko instance for faster restarts
–no-preload-browserDon’t preload a gecko instance for faster restarts
–disable-e10sRun tests without electrolysis preferences
Default: True
–disable-fissionDisable fission in Gecko.
Default: False
–stackfix-dirPath to directory containing assertion stack fixing scripts
–specialpowers-pathPath to specialPowers extension xpi file
–setprefDefines an extra user preference (overrides those in prefs_root)
Default:
–reftest-internalEnable reftest runner implemented inside Marionette
–reftest-externalDisable reftest runner implemented inside Marionette
Default: True
–reftest-screenshotPossible choices: always, fail, unexpected
With –reftest-internal, when to take a screenshot
–chaosEnable chaos mode with the specified feature flag (see http://searchfox.org/mozilla-central/source/mfbt/ChaosMode.h for details). If no value is supplied, all features are activated
GeckoView-specific¶ –setenvSet target environment variable, like FOO=BAR
Default:
Servo-specific¶ –user-stylesheetInject a user CSS stylesheet into every test.
Default:
Chrome-specific¶ –enable-mojojsEnable MojoJS for testing. Note that this flag is usally enabled automatically by wpt run, if it succeeds in downloading the right version of mojojs.zip or if –mojojs-path is specified.
Default: False
–mojojs-pathPath to mojojs gen/ directory. If it is not specified, wpt run will download and extract mojojs.zip into _venv2/mojojs/gen.
–enable-swiftshaderEnable SwiftShader for CPU-based 3D graphics. This can be used in environments with no hardware GPU available.
Default: False
–enable-experimentalEnable –enable-experimental-web-platform-features flag
–no-enable-experimentalDo not enable –enable-experimental-web-platform-features flag on experimental channels
Default: True
–enable-sanitizerOnly alert on sanitizer-related errors and crashes.
Default: False
–reuse-windowReuse a window across testharness.js tests where possible, which can speed up testing. Also useful for ensuring that the renderer process has a stable PID for a debugger to attach to.
Default: False
Sauce Labs-specific¶ –sauce-browserSauce Labs browser name
–sauce-platformSauce Labs OS platform
–sauce-versionSauce Labs browser version
–sauce-buildSauce Labs build identifier
–sauce-tagsSauce Labs identifying tag
Default:
–sauce-tunnel-idSauce Connect tunnel identifier
–sauce-userSauce Labs user name
–sauce-keySauce Labs access key
–sauce-connect-binaryPath to Sauce Connect binary
–sauce-init-timeoutNumber of seconds to wait for Sauce Connect tunnel to be available before aborting
Default: 30
–sauce-connect-argCommand-line argument to forward to the Sauce Connect binary (repeatable)
Default:
Taskcluster-specific¶ –github-checks-text-filePath to GitHub checks output file
WebKit-specific¶ –webkit-portWebKit port
Safari-specific¶ –kill-safariKill Safari when stopping the browser
Default: False
Output Logging¶ Each option represents a possible logging format and takes a filename to write that format to, or ‘-’ to write to stdout. Some options are provided by the mozlog utility; see https://firefox-source-docs.mozilla.org/mozbase/mozlog.html for extended documentation.
–log-rawRaw structured log messages (provided by mozlog)
–log-unittestUnittest style output (provided by mozlog)
–log-xunitxUnit compatible XML (provided by mozlog)
–log-htmlHTML report (provided by mozlog)
–log-machHuman-readable output (provided by mozlog)
–log-tbplTBPL style log format (provided by mozlog)
–log-groupedGrouped summary of test results (provided by mozlog)
–log-chromiumChromium Layout Tests format
–log-wptreportwptreport format
–log-wptscreenshotwpt.fyi screenshots
–log-raw-levelA least log level to subscribe to for the given formatter (debug, info, error, etc.)
–log-mach-verboseEnables verbose mode for the given formatter.
–log-mach-levelA least log level to subscribe to for the given formatter (debug, info, error, etc.)
–log-mach-bufferIf specified, enables message buffering at the given buffer size limit.
–log-mach-screenshotEnable logging reftest-analyzer compatible screenshot data.
–log-mach-no-screenshotDisable logging reftest-analyzer compatible screenshot data.
–log-tbpl-compactEnables compact mode for the given formatter.
–log-tbpl-levelA least log level to subscribe to for the given formatter (debug, info, error, etc.)
–log-tbpl-bufferIf specified, enables message buffering at the given buffer size limit.
–log-wptscreenshot-apiCache API (default: https://wpt.fyi/api/screenshots/hashes)
create¶ Create a new wpt test
wptcreate-h–no-editor-eEDITOR–long-timeout–overwrite-r-mREF–mismatch–wait–tests-rootTESTS_ROOTpath
Positional Arguments¶ pathPath to the test file
Named Arguments¶ –no-editorDon’t try to open the test in an editor
Default: False
-e, –editorEditor to use
–long-timeoutTest should be given a long timeout (typically 60s rather than 10s, but varies depending on environment)
Default: False
–overwriteAllow overwriting an existing test file
Default: False
-r, –reftestCreate a reftest rather than a testharness (js) test
Default: False
-m, –referencePath to the reference file
–mismatchCreate a mismatch reftest
Default: False
–waitCreate a reftest that waits until takeScreenshot() is called
Default: False
–tests-rootPath to the root of the wpt directory
Default: “/home/runner/work/wpt/wpt/tools/wpt/../..”
update-expectations¶ Update expectations files from raw logs.
wptupdate-expectations-h–configCONFIG–metadataMETADATA_ROOT–testsTESTS_ROOT–manifestMANIFEST_PATH–full–disable-intermittentDISABLE_INTERMITTENT–update-intermittent–remove-intermittent–no-remove-obsolete–properties-filePROPERTIES_FILE–no-properties-file–extra-propertyEXTRA_PROPERTY–log-rawLOG_RAW–log-unittestLOG_UNITTEST–log-xunitLOG_XUNIT–log-htmlLOG_HTML–log-machLOG_MACH–log-tbplLOG_TBPL–log-groupedLOG_GROUPED–log-chromiumLOG_CHROMIUM–log-wptreportLOG_WPTREPORT–log-wptscreenshotLOG_WPTSCREENSHOT–log-raw-levelLOG_RAW_LEVEL–log-mach-verbose–log-mach-levelLOG_MACH_LEVEL–log-mach-bufferLOG_MACH_BUFFER–log-mach-screenshot–log-mach-no-screenshot–log-tbpl-compact–log-tbpl-levelLOG_TBPL_LEVEL–log-tbpl-bufferLOG_TBPL_BUFFER–log-wptscreenshot-apiLOG_WPTSCREENSHOT_APIrun_log…
Positional Arguments¶ run_logLog file from run of tests
Named Arguments¶ –configPath to config file
–metadataPath to the folder containing test metadata
–testsPath to web-platform-tests
–manifestPath to test manifest (default is ${metadata_root}/MANIFEST.json)
–fullFor all tests that are updated, remove any existing conditions and missing subtests
Default: False
–disable-intermittentReason for disabling tests. When updating test results, disable tests that have inconsistent results across many runs with the given reason.
–update-intermittentUpdate test metadata with expected intermittent statuses.
Default: False
–remove-intermittentRemove obsolete intermittent statuses from expected statuses.
Default: False
–no-remove-obsoleteDon’t remove metadata files that no longer correspond to a test file
Default: True
–properties-filePath to a JSON file containing run_info properties to use in update. This must be of the form{“properties”: <name>, “dependents”: {<property name>: <name>}}
–no-properties-fileDon’t use the default properties file at ${metadata_root}/update_properties.json, even if it exists.
Default: False
–extra-propertyExtra property from run_info.json to use in metadata update.
Default:
Output Logging¶ Each option represents a possible logging format and takes a filename to write that format to, or ‘-’ to write to stdout. Some options are provided by the mozlog utility; see https://firefox-source-docs.mozilla.org/mozbase/mozlog.html for extended documentation.
–log-rawRaw structured log messages (provided by mozlog)
–log-unittestUnittest style output (provided by mozlog)
–log-xunitxUnit compatible XML (provided by mozlog)
–log-htmlHTML report (provided by mozlog)
–log-machHuman-readable output (provided by mozlog)
–log-tbplTBPL style log format (provided by mozlog)
–log-groupedGrouped summary of test results (provided by mozlog)
–log-chromiumChromium Layout Tests format
–log-wptreportwptreport format
–log-wptscreenshotwpt.fyi screenshots
–log-raw-levelA least log level to subscribe to for the given formatter (debug, info, error, etc.)
–log-mach-verboseEnables verbose mode for the given formatter.
–log-mach-levelA least log level to subscribe to for the given formatter (debug, info, error, etc.)
–log-mach-bufferIf specified, enables message buffering at the given buffer size limit.
–log-mach-screenshotEnable logging reftest-analyzer compatible screenshot data.
–log-mach-no-screenshotDisable logging reftest-analyzer compatible screenshot data.
–log-tbpl-compactEnables compact mode for the given formatter.
–log-tbpl-levelA least log level to subscribe to for the given formatter (debug, info, error, etc.)
–log-tbpl-bufferIf specified, enables message buffering at the given buffer size limit.
–log-wptscreenshot-apiCache API (default: https://wpt.fyi/api/screenshots/hashes)
files-changed¶ Get a list of files that have changed
wptfiles-changed-h–ignore-ruleIGNORE_RULE–modified–new–show-type–nullrevish
Positional Arguments¶ revishCommits to consider. Defaults to the commits on the current branch
Named Arguments¶ –ignore-ruleOverride the rules for paths to exclude from lists of changes. Rules are paths relative to the test root, with * before a separator or the end matching anything other than a path separator and ** in that position matching anything. This flag can be used multiple times for multiple rules. Specifying this flag overrides the default: resources/testharness*, resources/testdriver*
–modifiedInclude files under version control that have been modified or staged
Default: False
–newInclude files in the worktree that are not in version control
Default: False
–show-typePrint the test type along with each affected test
Default: False
–nullSeparate items with a null byte
Default: False
tests-affected¶ Get a list of tests affected by changes
wpttests-affected-h–ignore-ruleIGNORE_RULE–modified–new–show-type–null–metadataMETADATA_ROOTrevish
Positional Arguments¶ revishCommits to consider. Defaults to the commits on the current branch
Named Arguments¶ –ignore-ruleOverride the rules for paths to exclude from lists of changes. Rules are paths relative to the test root, with * before a separator or the end matching anything other than a path separator and ** in that position matching anything. This flag can be used multiple times for multiple rules. Specifying this flag overrides the default: resources/testharness*, resources/testdriver*
–modifiedInclude files under version control that have been modified or staged
Default: False
–newInclude files in the worktree that are not in version control
Default: False
–show-typePrint the test type along with each affected test
Default: False
–nullSeparate items with a null byte
Default: False
–metadataDirectory that will contain MANIFEST.json
Default: “/home/runner/work/wpt/wpt”
install¶ Install browser components
wptinstall-h–channel{stable,release,beta,dev,canary,nightly,preview,experimental}–download-only–renameRENAME-dDESTINATION–revisionREVISION{firefox,chrome,chromium,servo,safari,wktr}{browser,webdriver}
Positional Arguments¶ browserPossible choices: firefox, chrome, chromium, servo, safari, wktr
name of web browser product
componentPossible choices: browser, webdriver
name of component
Named Arguments¶ –channelPossible choices: stable, release, beta, dev, canary, nightly, preview, experimental
Name of browser release channel (default: nightly). “stable” and “release” are synonyms for the latest browser stable release; “beta” is the beta release; “dev” is only meaningful for Chrome (i.e. Chrome Dev); “nightly”, “experimental”, and “preview” are all synonyms for the latest available development or trunk release. (For WebDriver installs, we attempt to select an appropriate, compatible version for the latest browser release on the selected channel.) This flag overrides –browser-channel.
Default: “nightly”
–download-onlyDownload the selected component but don’t install it
Default: False
–renameFilename, excluding extension for downloaded archive (only with –download-only)
-d, –destinationfilesystem directory to place the component
–revisionChromium revision to install from snapshots
rev-list¶ List tagged revisions at regular intervals
wptrev-list-h–epochEPOCH–max-countMAX_COUNT
Named Arguments¶ –epochregular interval of time selected to get the tagged revisions. Valid values are digits followed by h/d/w (e.x. 9h, 9d, 9w …) where the mimimun selectable interval is one hour (1h)
Default: 1d
–max-countmaximum number of revisions to be returned by the command
Default: 1
install-android-emulator¶ Setup the x86 android emulator
wptinstall-android-emulator-h–pathDEST–reinstall–prompt–no-prompt
Named Arguments¶ –pathRoot path to use for emulator tooling
–reinstallForce reinstall even if the emulator already exists
Default: False
–promptEnable confirmation prompts
Default: False
–no-promptSkip confirmation prompts
Default: True
start-android-emulator¶ Start the x86 android emulator
wptstart-android-emulator-h–pathDEST–reinstall–prompt–no-prompt–device-serialDEVICE_SERIAL
Named Arguments¶ –pathRoot path to use for emulator tooling
–reinstallForce reinstall even if the emulator already exists
Default: False
–promptEnable confirmation prompts
Default: False
–no-promptSkip confirmation prompts
Default: True
–device-serialDevice serial number for Android emulator, if not emulator-5554
web-features-manifest¶ Create the WEB_FEATURES_MANIFEST.json
wptweb-features-manifest-h-pPATH
Named Arguments¶ -p, –pathPath to manifest file.
build-docs¶ Build documentation
wptbuild-docs-h–typeTYPE–docker–serveSERVE
Named Arguments¶ –typeOutput type (default: html)
Default: “html”
–dockerRun inside the docs docker image
Default: False
–serveRun a server on the specified port (default: 8000)
Build documentation
wptbuild-docs-h–typeTYPE–docker–serveSERVE
Named Arguments¶ –typeOutput type (default: html)
Default: “html”
–dockerRun inside the docs docker image
Default: False
–serveRun a server on the specified port (default: 8000)
–typeOutput type (default: html)
Default: “html”
–dockerRun inside the docs docker image
Default: False
–serveRun a server on the specified port (default: 8000)
test-jobs¶ List test jobs that should run for a set of commits
wpttest-jobs-h–all–includesINCLUDES…–jsonrevish
Positional Arguments¶ revishCommits to consider. Defaults to the commits on the current branch
Named Arguments¶ –allList all jobs unconditionally.
Default: False
–includesJobs to check for. Return code is 0 if all jobs are found, otherwise 1
–jsonOutput jobs as JSON, instead of one per line
Default: False
List test jobs that should run for a set of commits
wpttest-jobs-h–all–includesINCLUDES…–jsonrevish
Positional Arguments¶ revishCommits to consider. Defaults to the commits on the current branch
revishCommits to consider. Defaults to the commits on the current branch
Named Arguments¶ –allList all jobs unconditionally.
Default: False
–includesJobs to check for. Return code is 0 if all jobs are found, otherwise 1
–jsonOutput jobs as JSON, instead of one per line
Default: False
–allList all jobs unconditionally.
Default: False
–includesJobs to check for. Return code is 0 if all jobs are found, otherwise 1
–jsonOutput jobs as JSON, instead of one per line
Default: False
make-hosts-file¶ Output a hosts file to stdout
wptmake-hosts-file-haddress
Positional Arguments¶ addressAddress that hosts should point at
Default: “127.0.0.1”
Output a hosts file to stdout
wptmake-hosts-file-haddress
Positional Arguments¶ addressAddress that hosts should point at
Default: “127.0.0.1”
addressAddress that hosts should point at
Default: “127.0.0.1”
regen-certs¶ Regenerate the WPT certificates
wptregen-certs-h–checkend-secondsCHECKEND_SECONDS–force
Named Arguments¶ –checkend-secondsThe number of seconds the certificates must be valid for
Default: 5184000
–forceRegenerate certificates even if not reaching expiry
Default: False
Regenerate the WPT certificates
wptregen-certs-h–checkend-secondsCHECKEND_SECONDS–force
Named Arguments¶ –checkend-secondsThe number of seconds the certificates must be valid for
Default: 5184000
–forceRegenerate certificates even if not reaching expiry
Default: False
–checkend-secondsThe number of seconds the certificates must be valid for
Default: 5184000
–forceRegenerate certificates even if not reaching expiry
Default: False
update-built¶ Update built tests
wptupdate-built-h–list–include{canvas,conformance-checkers,css-images,css-ui,css-writing-modes,fetch,html5lib,infrastructure,mimesniff,speculative-parsing}…
Named Arguments¶ –listList suites that can be updated and the related script files
Default: False
–includePossible choices: canvas, conformance-checkers, css-images, css-ui, css-writing-modes, fetch, html5lib, infrastructure, mimesniff, speculative-parsing
Suites to update (default is to update everything)
Update built tests
wptupdate-built-h–list–include{canvas,conformance-checkers,css-images,css-ui,css-writing-modes,fetch,html5lib,infrastructure,mimesniff,speculative-parsing}…
Named Arguments¶ –listList suites that can be updated and the related script files
Default: False
–includePossible choices: canvas, conformance-checkers, css-images, css-ui, css-writing-modes, fetch, html5lib, infrastructure, mimesniff, speculative-parsing
Suites to update (default is to update everything)
–listList suites that can be updated and the related script files
Default: False
–includePossible choices: canvas, conformance-checkers, css-images, css-ui, css-writing-modes, fetch, html5lib, infrastructure, mimesniff, speculative-parsing
Suites to update (default is to update everything)
tc-download¶ Download logs from taskcluster
wpttc-download-h–refREF–artifact-nameARTIFACT_NAME–repo-nameREPO_NAME–token-fileTOKEN_FILE–out-dirOUT_DIR
Named Arguments¶ –refBranch (in the GitHub repository) or commit to fetch logs for
Default: “master”
–artifact-nameLog type to fetch
Default: “wpt_report.json.gz”
–repo-nameGitHub repo name in the format owner/repo. This must be the repo from which the Taskcluster run was scheduled (for PRs this is the repo into which the PR would merge)
Default: “web-platform-tests/wpt”
–token-fileFile containing GitHub token
–out-dirPath to save the logfiles
Default: “.”
Download logs from taskcluster
wpttc-download-h–refREF–artifact-nameARTIFACT_NAME–repo-nameREPO_NAME–token-fileTOKEN_FILE–out-dirOUT_DIR
Named Arguments¶ –refBranch (in the GitHub repository) or commit to fetch logs for
Default: “master”
–artifact-nameLog type to fetch
Default: “wpt_report.json.gz”
–repo-nameGitHub repo name in the format owner/repo. This must be the repo from which the Taskcluster run was scheduled (for PRs this is the repo into which the PR would merge)
Default: “web-platform-tests/wpt”
–token-fileFile containing GitHub token
–out-dirPath to save the logfiles
Default: “.”
–refBranch (in the GitHub repository) or commit to fetch logs for
Default: “master”
–artifact-nameLog type to fetch
Default: “wpt_report.json.gz”
–repo-nameGitHub repo name in the format owner/repo. This must be the repo from which the Taskcluster run was scheduled (for PRs this is the repo into which the PR would merge)
Default: “web-platform-tests/wpt”
–token-fileFile containing GitHub token
–out-dirPath to save the logfiles
Default: “.”
tc-decision¶ Run the decision task
wpttc-decision-h–event-pathEVENT_PATH–dry-run–tasks-pathTASKS_PATH
Named Arguments¶ –event-pathPath to file containing serialized GitHub event
–dry-runDon’t actually create the tasks, just output the tasks that would be created
Default: False
–tasks-pathPath to file in which to write payload for all scheduled tasks
Run the decision task
wpttc-decision-h–event-pathEVENT_PATH–dry-run–tasks-pathTASKS_PATH
Named Arguments¶ –event-pathPath to file containing serialized GitHub event
–dry-runDon’t actually create the tasks, just output the tasks that would be created
Default: False
–tasks-pathPath to file in which to write payload for all scheduled tasks
–event-pathPath to file containing serialized GitHub event
–dry-runDon’t actually create the tasks, just output the tasks that would be created
Default: False
–tasks-pathPath to file in which to write payload for all scheduled tasks
tc-sink-task¶ Run the sink task
wpttc-sink-task-h–github-checks-text-fileGITHUB_CHECKS_TEXT_FILEtaskstasks…
Positional Arguments¶ tasksA set of Taskcluster task ids to verify the state of.
Named Arguments¶ –github-checks-text-filePath to GitHub checks output file for Taskcluster runs
Run the sink task
wpttc-sink-task-h–github-checks-text-fileGITHUB_CHECKS_TEXT_FILEtaskstasks…
Positional Arguments¶ tasksA set of Taskcluster task ids to verify the state of.
tasksA set of Taskcluster task ids to verify the state of.
Named Arguments¶ –github-checks-text-filePath to GitHub checks output file for Taskcluster runs
–github-checks-text-filePath to GitHub checks output file for Taskcluster runs
docker-run¶ Run wpt docker image
wptdocker-run-h–rebuild–checkoutCHECKOUT–privileged–tagTAG
Named Arguments¶ –rebuildForce rebuild of image
Default: False
–checkoutRevision to checkout in the image. If this is not supplied we mount the wpt checkout on the host as /home/test/web-platform-tests/
–privilegedRun the image in priviledged mode (required for emulators)
Default: False
–tagDocker image tag to use (default wpt:local)
Default: “wpt:local”
Run wpt docker image
wptdocker-run-h–rebuild–checkoutCHECKOUT–privileged–tagTAG
Named Arguments¶ –rebuildForce rebuild of image
Default: False
–checkoutRevision to checkout in the image. If this is not supplied we mount the wpt checkout on the host as /home/test/web-platform-tests/
–privilegedRun the image in priviledged mode (required for emulators)
Default: False
–tagDocker image tag to use (default wpt:local)
Default: “wpt:local”
–rebuildForce rebuild of image
Default: False
–checkoutRevision to checkout in the image. If this is not supplied we mount the wpt checkout on the host as /home/test/web-platform-tests/
–privilegedRun the image in priviledged mode (required for emulators)
Default: False
–tagDocker image tag to use (default wpt:local)
Default: “wpt:local”
docker-push¶ Build and push wpt docker image
wptdocker-push-h–tagTAG–force
Named Arguments¶ –tagTag to use (default is taken from .taskcluster.yml)
–forceIgnore warnings and push anyway
Default: False
Build and push wpt docker image
wptdocker-push-h–tagTAG–force
Named Arguments¶ –tagTag to use (default is taken from .taskcluster.yml)
–forceIgnore warnings and push anyway
Default: False
–tagTag to use (default is taken from .taskcluster.yml)
–forceIgnore warnings and push anyway
Default: False
lint¶ Run the lint
wptlint-h–json–markdown–repo-rootREPO_ROOT–ignore-globIGNORE_GLOB–all–github-checks-text-fileGITHUB_CHECKS_TEXT_FILE-jJOBS–paths-filePATHS_FILEpaths…
Positional Arguments¶ pathsList of paths to lint
Named Arguments¶ –jsonOutput machine-readable JSON format
Default: False
–markdownOutput markdown
Default: False
–repo-rootThe WPT directory. Use this option if the lint script exists outside the repository
–ignore-globAdditional file glob to ignore (repeat to add more). Globs are matched against paths relative to REPO_ROOT using fnmatch, except that path separators are normalized.
–allIf no paths are passed, try to lint the whole working directory, not just files that changed
Default: False
–github-checks-text-filePath to GitHub checks output file for Taskcluster runs
-j, –jobsLevel to parallelism to use (defaults to 0, which detects the number of CPUs)
Default: 0
–paths-fileFile containing a list of files to lint, one per line
Run the lint
wptlint-h–json–markdown–repo-rootREPO_ROOT–ignore-globIGNORE_GLOB–all–github-checks-text-fileGITHUB_CHECKS_TEXT_FILE-jJOBS–paths-filePATHS_FILEpaths…
Positional Arguments¶ pathsList of paths to lint
pathsList of paths to lint
Named Arguments¶ –jsonOutput machine-readable JSON format
Default: False
–markdownOutput markdown
Default: False
–repo-rootThe WPT directory. Use this option if the lint script exists outside the repository
–ignore-globAdditional file glob to ignore (repeat to add more). Globs are matched against paths relative to REPO_ROOT using fnmatch, except that path separators are normalized.
–allIf no paths are passed, try to lint the whole working directory, not just files that changed
Default: False
–github-checks-text-filePath to GitHub checks output file for Taskcluster runs
-j, –jobsLevel to parallelism to use (defaults to 0, which detects the number of CPUs)
Default: 0
–paths-fileFile containing a list of files to lint, one per line
–jsonOutput machine-readable JSON format
Default: False
–markdownOutput markdown
Default: False
–repo-rootThe WPT directory. Use this option if the lint script exists outside the repository
–ignore-globAdditional file glob to ignore (repeat to add more). Globs are matched against paths relative to REPO_ROOT using fnmatch, except that path separators are normalized.
–allIf no paths are passed, try to lint the whole working directory, not just files that changed
Default: False
–github-checks-text-filePath to GitHub checks output file for Taskcluster runs
-j, –jobsLevel to parallelism to use (defaults to 0, which detects the number of CPUs)
Default: 0
–paths-fileFile containing a list of files to lint, one per line
manifest¶ Update the MANIFEST.json file
wptmanifest-h-v-pPATH–tests-rootTESTS_ROOT-r–url-baseURL_BASE–no-download–cache-rootCACHE_ROOT–no-parallel
Named Arguments¶ -v, –verboseTurn on verbose logging
Default: False
-p, –pathPath to manifest file.
–tests-rootPath to root of tests.
Default: /home/runner/work/wpt/wpt
-r, –rebuildForce a full rebuild of the manifest.
Default: False
–url-baseBase url to use as the mount point for tests in this manifest.
Default: “/”
–no-downloadNever attempt to download the manifest.
Default: True
–cache-rootPath in which to store any caches (default <tests_root>/.wptcache/)
Default: “/home/runner/work/wpt/wpt/.wptcache”
–no-parallelDo not parallelize building the manifest
Default: True
Update the MANIFEST.json file
wptmanifest-h-v-pPATH–tests-rootTESTS_ROOT-r–url-baseURL_BASE–no-download–cache-rootCACHE_ROOT–no-parallel
Named Arguments¶ -v, –verboseTurn on verbose logging
Default: False
-p, –pathPath to manifest file.
–tests-rootPath to root of tests.
Default: /home/runner/work/wpt/wpt
-r, –rebuildForce a full rebuild of the manifest.
Default: False
–url-baseBase url to use as the mount point for tests in this manifest.
Default: “/”
–no-downloadNever attempt to download the manifest.
Default: True
–cache-rootPath in which to store any caches (default <tests_root>/.wptcache/)
Default: “/home/runner/work/wpt/wpt/.wptcache”
–no-parallelDo not parallelize building the manifest
Default: True
-v, –verboseTurn on verbose logging
Default: False
-p, –pathPath to manifest file.
–tests-rootPath to root of tests.
Default: /home/runner/work/wpt/wpt
-r, –rebuildForce a full rebuild of the manifest.
Default: False
–url-baseBase url to use as the mount point for tests in this manifest.
Default: “/”
–no-downloadNever attempt to download the manifest.
Default: True
–cache-rootPath in which to store any caches (default <tests_root>/.wptcache/)
Default: “/home/runner/work/wpt/wpt/.wptcache”
–no-parallelDo not parallelize building the manifest
Default: True
manifest-download¶ Download recent pregenerated MANIFEST.json file
wptmanifest-download-h-pPATH–tests-rootTESTS_ROOT–force
Named Arguments¶ -p, –pathPath to manifest file.
–tests-rootPath to root of tests.
Default: /home/runner/work/wpt/wpt
–forceAlways download, even if the existing manifest is recent
Default: False
Download recent pregenerated MANIFEST.json file
wptmanifest-download-h-pPATH–tests-rootTESTS_ROOT–force
Named Arguments¶ -p, –pathPath to manifest file.
–tests-rootPath to root of tests.
Default: /home/runner/work/wpt/wpt
–forceAlways download, even if the existing manifest is recent
Default: False
-p, –pathPath to manifest file.
–tests-rootPath to root of tests.
Default: /home/runner/work/wpt/wpt
–forceAlways download, even if the existing manifest is recent
Default: False
test-paths¶ Print test paths given a set of test ids
wpttest-paths-h-pPATH–src-rootSRC_ROOT–tests-rootTESTS_ROOT–no-update-r–url-baseURL_BASE–cache-rootCACHE_ROOT–jsontest_idstest_ids…
Positional Arguments¶ test_idsTest ids for which to get paths
Named Arguments¶ -p, –pathPath to manifest file.
–src-rootPath to root of sourcetree.
–tests-rootPath to root of tests.
Default: /home/runner/work/wpt/wpt
–no-updateDon’t update manifest before continuing
Default: True
-r, –rebuildForce a full rebuild of the manifest.
Default: False
–url-baseBase url to use as the mount point for tests in this manifest.
Default: “/”
–cache-rootPath in which to store any caches (default <tests_root>/.wptcache/)
Default: “/home/runner/work/wpt/wpt/.wptcache”
–jsonOutput as JSON
Default: False
Print test paths given a set of test ids
wpttest-paths-h-pPATH–src-rootSRC_ROOT–tests-rootTESTS_ROOT–no-update-r–url-baseURL_BASE–cache-rootCACHE_ROOT–jsontest_idstest_ids…
Positional Arguments¶ test_idsTest ids for which to get paths
test_idsTest ids for which to get paths
Named Arguments¶ -p, –pathPath to manifest file.
–src-rootPath to root of sourcetree.
–tests-rootPath to root of tests.
Default: /home/runner/work/wpt/wpt
–no-updateDon’t update manifest before continuing
Default: True
-r, –rebuildForce a full rebuild of the manifest.
Default: False
–url-baseBase url to use as the mount point for tests in this manifest.
Default: “/”
–cache-rootPath in which to store any caches (default <tests_root>/.wptcache/)
Default: “/home/runner/work/wpt/wpt/.wptcache”
–jsonOutput as JSON
Default: False
-p, –pathPath to manifest file.
–src-rootPath to root of sourcetree.
–tests-rootPath to root of tests.
Default: /home/runner/work/wpt/wpt
–no-updateDon’t update manifest before continuing
Default: True
-r, –rebuildForce a full rebuild of the manifest.
Default: False
–url-baseBase url to use as the mount point for tests in this manifest.
Default: “/”
–cache-rootPath in which to store any caches (default <tests_root>/.wptcache/)
Default: “/home/runner/work/wpt/wpt/.wptcache”
–jsonOutput as JSON
Default: False
spec¶ Update the SPEC_MANIFEST.json file
wptspec-h-v-pPATH–tests-rootTESTS_ROOT–url-baseURL_BASE–cache-rootCACHE_ROOT–no-parallel
Named Arguments¶ -v, –verboseTurn on verbose logging
Default: False
-p, –pathPath to manifest file.
–tests-rootPath to root of tests.
Default: /home/runner/work/wpt/wpt
–url-baseBase url to use as the mount point for tests in this manifest.
Default: “/”
–cache-rootPath in which to store any caches (default <tests_root>/.wptcache/)
Default: “/home/runner/work/wpt/wpt/.wptcache”
–no-parallelDo not parallelize building the manifest
Default: True
Update the SPEC_MANIFEST.json file
wptspec-h-v-pPATH–tests-rootTESTS_ROOT–url-baseURL_BASE–cache-rootCACHE_ROOT–no-parallel
Named Arguments¶ -v, –verboseTurn on verbose logging
Default: False
-p, –pathPath to manifest file.
–tests-rootPath to root of tests.
Default: /home/runner/work/wpt/wpt
–url-baseBase url to use as the mount point for tests in this manifest.
Default: “/”
–cache-rootPath in which to store any caches (default <tests_root>/.wptcache/)
Default: “/home/runner/work/wpt/wpt/.wptcache”
–no-parallelDo not parallelize building the manifest
Default: True
-v, –verboseTurn on verbose logging
Default: False
-p, –pathPath to manifest file.
–tests-rootPath to root of tests.
Default: /home/runner/work/wpt/wpt
–url-baseBase url to use as the mount point for tests in this manifest.
Default: “/”
–cache-rootPath in which to store any caches (default <tests_root>/.wptcache/)
Default: “/home/runner/work/wpt/wpt/.wptcache”
–no-parallelDo not parallelize building the manifest
Default: True
serve¶ Run wptserve server
wptserve-h–latencyLATENCY–configCONFIG_PATH–doc_rootDOC_ROOT–ws_doc_rootWS_DOC_ROOT–ws_extraWS_EXTRA–inject-scriptINJECT_SCRIPT–alias_fileALIAS_FILE–no-h2–webtransport-h3–exit-after-start–verbose
Named Arguments¶ –latencyArtificial latency to add before sending http responses, in ms
–configPath to external config file
–doc_rootPath to document root. Overrides config.
–ws_doc_rootPath to WebSockets document root. Overrides config.
–ws_extraPath to extra directory containing ws handlers. Overrides config.
Default:
–inject-scriptPath to script file to inject, useful for testing polyfills.
–alias_fileFile with entries for aliases/multiple doc roots. In form of /ALIAS_NAME/, DOC_ROOTn
–no-h2Disable the HTTP/2.0 server
–webtransport-h3Enable WebTransport over HTTP/3 server
Default: False
–exit-after-startExit after starting servers
Default: False
–verboseEnable verbose logging
Default: False
Run wptserve server
wptserve-h–latencyLATENCY–configCONFIG_PATH–doc_rootDOC_ROOT–ws_doc_rootWS_DOC_ROOT–ws_extraWS_EXTRA–inject-scriptINJECT_SCRIPT–alias_fileALIAS_FILE–no-h2–webtransport-h3–exit-after-start–verbose
Named Arguments¶ –latencyArtificial latency to add before sending http responses, in ms
–configPath to external config file
–doc_rootPath to document root. Overrides config.
–ws_doc_rootPath to WebSockets document root. Overrides config.
–ws_extraPath to extra directory containing ws handlers. Overrides config.
Default:
–inject-scriptPath to script file to inject, useful for testing polyfills.
–alias_fileFile with entries for aliases/multiple doc roots. In form of /ALIAS_NAME/, DOC_ROOTn
–no-h2Disable the HTTP/2.0 server
–webtransport-h3Enable WebTransport over HTTP/3 server
Default: False
–exit-after-startExit after starting servers
Default: False
–verboseEnable verbose logging
Default: False
–latencyArtificial latency to add before sending http responses, in ms
–configPath to external config file
–doc_rootPath to document root. Overrides config.
–ws_doc_rootPath to WebSockets document root. Overrides config.
–ws_extraPath to extra directory containing ws handlers. Overrides config.
Default:
–inject-scriptPath to script file to inject, useful for testing polyfills.
–alias_fileFile with entries for aliases/multiple doc roots. In form of /ALIAS_NAME/, DOC_ROOTn
–no-h2Disable the HTTP/2.0 server
–webtransport-h3Enable WebTransport over HTTP/3 server
Default: False
–exit-after-startExit after starting servers
Default: False
–verboseEnable verbose logging
Default: False
serve-wave¶ Run wptserve server for WAVE
wptserve-wave-h–latencyLATENCY–configCONFIG_PATH–doc_rootDOC_ROOT–ws_doc_rootWS_DOC_ROOT–ws_extraWS_EXTRA–inject-scriptINJECT_SCRIPT–alias_fileALIAS_FILE–no-h2–webtransport-h3–exit-after-start–verbose–report
Named Arguments¶ –latencyArtificial latency to add before sending http responses, in ms
–configPath to external config file
–doc_rootPath to document root. Overrides config.
–ws_doc_rootPath to WebSockets document root. Overrides config.
–ws_extraPath to extra directory containing ws handlers. Overrides config.
Default:
–inject-scriptPath to script file to inject, useful for testing polyfills.
–alias_fileFile with entries for aliases/multiple doc roots. In form of /ALIAS_NAME/, DOC_ROOTn
–no-h2Disable the HTTP/2.0 server
–webtransport-h3Enable WebTransport over HTTP/3 server
Default: False
–exit-after-startExit after starting servers
Default: False
–verboseEnable verbose logging
Default: False
–reportFlag for enabling the WPTReporting server
Default: False
Run wptserve server for WAVE
wptserve-wave-h–latencyLATENCY–configCONFIG_PATH–doc_rootDOC_ROOT–ws_doc_rootWS_DOC_ROOT–ws_extraWS_EXTRA–inject-scriptINJECT_SCRIPT–alias_fileALIAS_FILE–no-h2–webtransport-h3–exit-after-start–verbose–report
Named Arguments¶ –latencyArtificial latency to add before sending http responses, in ms
–configPath to external config file
–doc_rootPath to document root. Overrides config.
–ws_doc_rootPath to WebSockets document root. Overrides config.
–ws_extraPath to extra directory containing ws handlers. Overrides config.
Default:
–inject-scriptPath to script file to inject, useful for testing polyfills.
–alias_fileFile with entries for aliases/multiple doc roots. In form of /ALIAS_NAME/, DOC_ROOTn
–no-h2Disable the HTTP/2.0 server
–webtransport-h3Enable WebTransport over HTTP/3 server
Default: False
–exit-after-startExit after starting servers
Default: False
–verboseEnable verbose logging
Default: False
–reportFlag for enabling the WPTReporting server
Default: False
–latencyArtificial latency to add before sending http responses, in ms
–configPath to external config file
–doc_rootPath to document root. Overrides config.
–ws_doc_rootPath to WebSockets document root. Overrides config.
–ws_extraPath to extra directory containing ws handlers. Overrides config.
Default:
–inject-scriptPath to script file to inject, useful for testing polyfills.
–alias_fileFile with entries for aliases/multiple doc roots. In form of /ALIAS_NAME/, DOC_ROOTn
–no-h2Disable the HTTP/2.0 server
–webtransport-h3Enable WebTransport over HTTP/3 server
Default: False
–exit-after-startExit after starting servers
Default: False
–verboseEnable verbose logging
Default: False
–reportFlag for enabling the WPTReporting server
Default: False
run¶ Run tests in a browser
wptrun–channel{stable,release,beta,dev,canary,nightly,preview,experimental}–affectedAFFECTED–yes–install-browser–install-webdriver–logcat-dirLOGCAT_DIR-h–manifest-update–no-manifest-update–manifest-download–no-manifest-download–timeout-multiplierTIMEOUT_MULTIPLIER–run-by-dirRUN_BY_DIR-f–processesPROCESSES–max-restartsMAX_RESTARTS–no-capture-stdio–no-fail-on-unexpected–no-fail-on-unexpected-pass–no-restart-on-new-group–list-test-groups–list-disabled–list-tests–verify–verify-log-full–verify-repeat-loopVERIFY_REPEAT_LOOP–verify-repeat-restartVERIFY_REPEAT_RESTART–verify-no-chaos-mode|–verify-chaos-mode–verify-max-timeVERIFY_MAX_TIME–repeat-max-timeREPEAT_MAX_TIME–verify-no-output-results|–verify-output-results–test-types{crashtest,testharness,reftest,print-reftest,wdspec}…–subsuite-fileSUBSUITE_FILE–subsuiteSUBSUITES–small-subsuite-sizeSMALL_SUBSUITE_SIZE–includeINCLUDE–include-fileINCLUDE_FILE–excludeEXCLUDE–include-manifestINCLUDE_MANIFEST–test-groupsTEST_GROUPS_FILE–skip-timeout–skip-crash–skip-implementation-status{not-implementing,backlog,implementing}–enable-webtransport-h3–no-enable-webtransport-h3–tagTAGS–exclude-tagEXCLUDE_TAGS–default-exclude–debuggerDEBUGGER–debugger-argsDEBUGGER_ARGS–rerunRERUN–repeatREPEAT–repeat-until-unexpected–retry-unexpectedRETRY_UNEXPECTED–pause-after-test–no-pause-after-test–debug-test–pause-on-unexpected–no-restart-on-unexpected–symbols-pathSYMBOLS_PATH–stackwalk-binarySTACKWALK_BINARY–pdb–leak-check–no-leak-check–adb-binaryADB_BINARY–package-namePACKAGE_NAME–keep-app-data-directory–device-serialDEVICE_SERIAL–binaryBINARY–binary-argBINARY_ARGS–webdriver-binaryBINARY–webdriver-argWEBDRIVER_ARGS–metadataMETADATA_ROOT–testsTESTS_ROOT–manifestMANIFEST_PATH–run-infoRUN_INFO–product{android_webview,chrome,chrome_android,chrome_ios,chromium,edge,firefox,firefox_android,headless_shell,safari,sauce,servo,servodriver,opera,webkit,webkitgtk_minibrowser,wktr,epiphany,ladybird}–browser-versionBROWSER_VERSION–browser-channelBROWSER_CHANNEL–configCONFIG–install-fonts–no-install-fonts–font-dirFONT_DIR–inject-scriptINJECT_SCRIPT–headless–no-headless–instrument-to-fileINSTRUMENT_TO_FILE–suppress-handler-traceback–no-suppress-handler-traceback–ws-extraWS_EXTRA–debug-build|–release-build–total-chunksTOTAL_CHUNKS–this-chunkTHIS_CHUNK–chunk-type{none,hash,id_hash,dir_hash}–ssl-type{openssl,pregenerated,none}–openssl-binaryOPENSSL_BINARY–certutil-binaryCERTUTIL_BINARY–ca-cert-pathCA_CERT_PATH–host-key-pathHOST_KEY_PATH–host-cert-pathHOST_CERT_PATH–prefs-rootPREFS_ROOT–preload-browser–no-preload-browser–disable-e10s–disable-fission–stackfix-dirSTACKFIX_DIR–specialpowers-pathSPECIALPOWERS_PATH–setprefPREF=VALUE–reftest-internal–reftest-external–reftest-screenshot{always,fail,unexpected}–chaosCHAOS_MODE_FLAGS–setenvENV–user-stylesheetUSER_STYLESHEETS–enable-mojojs–mojojs-pathMOJOJS_PATH–enable-swiftshader–enable-experimental–no-enable-experimental–enable-sanitizer–reuse-window–sauce-browserSAUCE_BROWSER–sauce-platformSAUCE_PLATFORM–sauce-versionSAUCE_VERSION–sauce-buildSAUCE_BUILD–sauce-tagsSAUCE_TAGS…–sauce-tunnel-idSAUCE_TUNNEL_ID–sauce-userSAUCE_USER–sauce-keySAUCE_KEY–sauce-connect-binarySAUCE_CONNECT_BINARY–sauce-init-timeoutSAUCE_INIT_TIMEOUT–sauce-connect-argSAUCE_CONNECT_ARGS–github-checks-text-fileGITHUB_CHECKS_TEXT_FILE–webkit-portWEBKIT_PORT–kill-safari–log-rawLOG_RAW–log-unittestLOG_UNITTEST–log-xunitLOG_XUNIT–log-htmlLOG_HTML–log-machLOG_MACH–log-tbplLOG_TBPL–log-groupedLOG_GROUPED–log-chromiumLOG_CHROMIUM–log-wptreportLOG_WPTREPORT–log-wptscreenshotLOG_WPTSCREENSHOT–log-raw-levelLOG_RAW_LEVEL–log-mach-verbose–log-mach-levelLOG_MACH_LEVEL–log-mach-bufferLOG_MACH_BUFFER–log-mach-screenshot–log-mach-no-screenshot–log-tbpl-compact–log-tbpl-levelLOG_TBPL_LEVEL–log-tbpl-bufferLOG_TBPL_BUFFER–log-wptscreenshot-apiLOG_WPTSCREENSHOT_APIproducttest_list…
Positional Arguments¶ productBrowser to run tests in
test_listList of URLs for tests to run, or paths including tests to run. (equivalent to –include)
Named Arguments¶ –channelPossible choices: stable, release, beta, dev, canary, nightly, preview, experimental
Name of browser release channel (default: nightly). “stable” and “release” are synonyms for the latest browser stable release; “beta” is the beta release; “dev” is only meaningful for Chrome (i.e. Chrome Dev); “nightly”, “experimental”, and “preview” are all synonyms for the latest available development or trunk release. (For WebDriver installs, we attempt to select an appropriate, compatible version for the latest browser release on the selected channel.) This flag overrides –browser-channel.
Default: “nightly”
–affectedRun affected tests since revish
–yes, -yDon’t prompt before installing components
Default: True
–install-browserInstall the browser from the release channel specified by –channel (or the nightly channel by default).
Default: False
–install-webdriverInstall WebDriver from the release channel specified by –channel (or the nightly channel by default).
Default: False
–logcat-dirDirectory to write Android logcat files to
–manifest-updateRegenerate the test manifest.
–no-manifest-updatePrevent regeneration of the test manifest.
Default: True
–manifest-downloadAttempt to download a preexisting manifest when updating.
–no-manifest-downloadPrevent download of the test manifest.
Default: True
–timeout-multiplierMultiplier relative to standard test timeout to use
–run-by-dirSplit run into groups by directories. With a parameter,limit the depth of splits e.g. –run-by-dir=1 to split by top-leveldirectory
Default: False
-f, –fully-parallelRun every test in a separate group for fully parallelism.
Default: False
–processesNumber of simultaneous processes to use
–max-restartsMaximum number of browser restart retries
Default: 5
–no-capture-stdioDon’t capture stdio and write to logging
Default: False
–no-fail-on-unexpectedExit with status code 0 when test expectations are violated
Default: True
–no-fail-on-unexpected-passExit with status code 0 when all unexpected results are PASS
Default: True
–no-restart-on-new-groupDon’t restart test runner when start a new test group
Default: True
–verifyRun a stability check on the selected tests
Default: False
–verify-no-chaos-modeDisable chaos mode when running on Firefox
Default: True
–verify-chaos-modeEnable chaos mode when running on Firefox
Default: True
–verify-no-output-resultsPrints individuals test results and messages
Default: True
–verify-output-resultsDisable printing individuals test results and messages
Default: True
–debug-buildBuild is a debug build (overrides any mozinfo file)
–release-buildBuild is a release (overrides any mozinfo file)
Mode¶ –list-test-groupsList the top level directories containing tests that will run.
Default: False
–list-disabledList the tests that are disabled on the current platform
Default: False
–list-testsList all tests that will run
Default: False
–verify-log-fullOutput per-iteration test results when running verify
Default: False
–verify-repeat-loopNumber of iterations for a run that reloads each test without restart.
Default: 10
–verify-repeat-restartNumber of iterations, for a run that restarts the runner between each iteration
Default: 5
–verify-max-timeThe maximum number of minutes for the job to run
–repeat-max-timeThe maximum number of minutes for the test suite to attempt repeat runs
Default: 100
Test Selection¶ –test-typesPossible choices: crashtest, testharness, reftest, print-reftest, wdspec
Test types to run
Default: {‘crashtest’, ‘testharness’, ‘reftest’, ‘print-reftest’, ‘wdspec’}
–subsuite-filePath to JSON file containing subsuite configuration
–subsuiteSubsuite names to run. Runs all subsuites when omitted.
–small-subsuite-sizeMaximum number of tests a subsuite can have to be treated as small subsuite.Tests from a small subsuite will be grouped in one group.
Default: 50
–includeURL prefix to include
–include-fileA file listing URL prefix for tests
–excludeURL prefix to exclude
–include-manifestPath to manifest listing tests to include
–test-groupsPath to json file containing a mapping {group_name: test_ids}
–skip-timeoutSkip tests that are expected to time out
Default: False
–skip-crashSkip tests that are expected to crash
Default: False
–skip-implementation-statusPossible choices: not-implementing, backlog, implementing
Skip tests that have the given implementation status
–enable-webtransport-h3Enable tests that require WebTransport over HTTP/3 server (default: false)
–no-enable-webtransport-h3Do not enable WebTransport tests on experimental channels
Default: True
–tagLabels applied to tests to include in the run. Labels starting dir: are equivalent to top-level directories.
–exclude-tagLabels applied to tests to exclude in the run. Takes precedence over –tag. Labels starting dir: are equivalent to top-level directories.
–default-excludeOnly run the tests explicitly given in arguments. No tests will run if the list is empty, and the program will exit with status code 0.
Default: False
Debugging¶ –debuggerrun under a debugger, e.g. gdb or valgrind
–debugger-argsarguments to the debugger
–rerunNumber of times to re run each test without restarts
Default: 1
–repeatNumber of times to run the tests, restarting between each run
Default: 1
–repeat-until-unexpectedRun tests in a loop until one returns an unexpected result
–retry-unexpectedMaximum number of times to retry unexpected tests. A test is retried until it gets one of the expected status, or until it exhausts the maximum number of retries.
Default: 0
–pause-after-testHalt the test runner after each test (this happens by default if only a single test is run)
–no-pause-after-testDon’t halt the test runner irrespective of the number of tests run
Default: True
–debug-testRun tests with additional debugging features enabled
Default: False
–pause-on-unexpectedHalt the test runner when an unexpected result is encountered
Default: False
–no-restart-on-unexpectedDon’t restart on an unexpected result
Default: True
–symbols-pathPath or url to symbols file used to analyse crash minidumps.
–stackwalk-binaryPath to stackwalker program used to analyse minidumps.
–pdbDrop into pdb on python exception
Default: False
–leak-checkEnable leak checking for supported browsers (Gecko: enabled by default for debug builds, silently ignored for opt, mobile)
–no-leak-checkDisable leak checking
Android specific arguments¶ –adb-binaryPath to adb binary to use
–package-nameAndroid package name to run tests against
–keep-app-data-directoryDon’t delete the app data directory
Default: False
–device-serialRunning Android instances to connect to, if not emulator-5554
Default:
Configuration¶ –binaryDesktop binary to run tests against
–binary-argExtra argument for the binary
Default:
–webdriver-binaryWebDriver server binary to use
–webdriver-argExtra argument for the WebDriver binary
Default:
–metadataPath to root directory containing test metadata
–testsPath to root directory containing test files
–manifestPath to test manifest (default is ${metadata_root}/MANIFEST.json)
–run-infoPath to directory containing extra json files to add to run info
–productPossible choices: android_webview, chrome, chrome_android, chrome_ios, chromium, edge, firefox, firefox_android, headless_shell, safari, sauce, servo, servodriver, opera, webkit, webkitgtk_minibrowser, wktr, epiphany, ladybird
Browser against which to run tests
–browser-versionInformative string detailing the browser release version. This is included in the run_info data.
–browser-channelInformative string detailing the browser release channel. This is included in the run_info data.
–configPath to config file
–install-fontsInstall additional system fonts on your system
–no-install-fontsDo not install additional system fonts on your system
Default: True
–font-dirPath to local font installation directory
–inject-scriptPath to script file to inject, useful for testing polyfills.
–headlessRun browser in headless mode
–no-headlessDon’t run browser in headless mode
Default: True
–instrument-to-filePath to write instrumentation logs to
–suppress-handler-tracebackDon’t write the stacktrace for exceptions in server handlers
–no-suppress-handler-tracebackWrite the stacktrace for exceptions in server handlers
Default: True
–ws-extraExtra paths containing websockets handlers
Test Chunking¶ –total-chunksTotal number of chunks to use
Default: 1
–this-chunkChunk number to run
Default: 1
–chunk-typePossible choices: none, hash, id_hash, dir_hash
Chunking type to use
SSL/TLS¶ –ssl-typePossible choices: openssl, pregenerated, none
Type of ssl support to enable (running without ssl may lead to spurious errors)
–openssl-binaryPath to openssl binary
Default: “openssl”
–certutil-binaryPath to certutil binary for use with Firefox + ssl
–ca-cert-pathPath to ca certificate when using pregenerated ssl certificates
–host-key-pathPath to host private key when using pregenerated ssl certificates
–host-cert-pathPath to host certificate when using pregenerated ssl certificates
Gecko-specific¶ –prefs-rootPath to the folder containing browser prefs
–preload-browserPreload a gecko instance for faster restarts
–no-preload-browserDon’t preload a gecko instance for faster restarts
–disable-e10sRun tests without electrolysis preferences
Default: True
–disable-fissionDisable fission in Gecko.
Default: False
–stackfix-dirPath to directory containing assertion stack fixing scripts
–specialpowers-pathPath to specialPowers extension xpi file
–setprefDefines an extra user preference (overrides those in prefs_root)
Default:
–reftest-internalEnable reftest runner implemented inside Marionette
–reftest-externalDisable reftest runner implemented inside Marionette
Default: True
–reftest-screenshotPossible choices: always, fail, unexpected
With –reftest-internal, when to take a screenshot
–chaosEnable chaos mode with the specified feature flag (see http://searchfox.org/mozilla-central/source/mfbt/ChaosMode.h for details). If no value is supplied, all features are activated
GeckoView-specific¶ –setenvSet target environment variable, like FOO=BAR
Default:
Servo-specific¶ –user-stylesheetInject a user CSS stylesheet into every test.
Default:
Chrome-specific¶ –enable-mojojsEnable MojoJS for testing. Note that this flag is usally enabled automatically by wpt run, if it succeeds in downloading the right version of mojojs.zip or if –mojojs-path is specified.
Default: False
–mojojs-pathPath to mojojs gen/ directory. If it is not specified, wpt run will download and extract mojojs.zip into _venv2/mojojs/gen.
–enable-swiftshaderEnable SwiftShader for CPU-based 3D graphics. This can be used in environments with no hardware GPU available.
Default: False
–enable-experimentalEnable –enable-experimental-web-platform-features flag
–no-enable-experimentalDo not enable –enable-experimental-web-platform-features flag on experimental channels
Default: True
–enable-sanitizerOnly alert on sanitizer-related errors and crashes.
Default: False
–reuse-windowReuse a window across testharness.js tests where possible, which can speed up testing. Also useful for ensuring that the renderer process has a stable PID for a debugger to attach to.
Default: False
Sauce Labs-specific¶ –sauce-browserSauce Labs browser name
–sauce-platformSauce Labs OS platform
–sauce-versionSauce Labs browser version
–sauce-buildSauce Labs build identifier
–sauce-tagsSauce Labs identifying tag
Default:
–sauce-tunnel-idSauce Connect tunnel identifier
–sauce-userSauce Labs user name
–sauce-keySauce Labs access key
–sauce-connect-binaryPath to Sauce Connect binary
–sauce-init-timeoutNumber of seconds to wait for Sauce Connect tunnel to be available before aborting
Default: 30
–sauce-connect-argCommand-line argument to forward to the Sauce Connect binary (repeatable)
Default:
Taskcluster-specific¶ –github-checks-text-filePath to GitHub checks output file
WebKit-specific¶ –webkit-portWebKit port
Safari-specific¶ –kill-safariKill Safari when stopping the browser
Default: False
Output Logging¶ Each option represents a possible logging format and takes a filename to write that format to, or ‘-’ to write to stdout. Some options are provided by the mozlog utility; see https://firefox-source-docs.mozilla.org/mozbase/mozlog.html for extended documentation.
–log-rawRaw structured log messages (provided by mozlog)
–log-unittestUnittest style output (provided by mozlog)
–log-xunitxUnit compatible XML (provided by mozlog)
–log-htmlHTML report (provided by mozlog)
–log-machHuman-readable output (provided by mozlog)
–log-tbplTBPL style log format (provided by mozlog)
–log-groupedGrouped summary of test results (provided by mozlog)
–log-chromiumChromium Layout Tests format
–log-wptreportwptreport format
–log-wptscreenshotwpt.fyi screenshots
–log-raw-levelA least log level to subscribe to for the given formatter (debug, info, error, etc.)
–log-mach-verboseEnables verbose mode for the given formatter.
–log-mach-levelA least log level to subscribe to for the given formatter (debug, info, error, etc.)
–log-mach-bufferIf specified, enables message buffering at the given buffer size limit.
–log-mach-screenshotEnable logging reftest-analyzer compatible screenshot data.
–log-mach-no-screenshotDisable logging reftest-analyzer compatible screenshot data.
–log-tbpl-compactEnables compact mode for the given formatter.
–log-tbpl-levelA least log level to subscribe to for the given formatter (debug, info, error, etc.)
–log-tbpl-bufferIf specified, enables message buffering at the given buffer size limit.
–log-wptscreenshot-apiCache API (default: https://wpt.fyi/api/screenshots/hashes)
Run tests in a browser
wptrun–channel{stable,release,beta,dev,canary,nightly,preview,experimental}–affectedAFFECTED–yes–install-browser–install-webdriver–logcat-dirLOGCAT_DIR-h–manifest-update–no-manifest-update–manifest-download–no-manifest-download–timeout-multiplierTIMEOUT_MULTIPLIER–run-by-dirRUN_BY_DIR-f–processesPROCESSES–max-restartsMAX_RESTARTS–no-capture-stdio–no-fail-on-unexpected–no-fail-on-unexpected-pass–no-restart-on-new-group–list-test-groups–list-disabled–list-tests–verify–verify-log-full–verify-repeat-loopVERIFY_REPEAT_LOOP–verify-repeat-restartVERIFY_REPEAT_RESTART–verify-no-chaos-mode|–verify-chaos-mode–verify-max-timeVERIFY_MAX_TIME–repeat-max-timeREPEAT_MAX_TIME–verify-no-output-results|–verify-output-results–test-types{crashtest,testharness,reftest,print-reftest,wdspec}…–subsuite-fileSUBSUITE_FILE–subsuiteSUBSUITES–small-subsuite-sizeSMALL_SUBSUITE_SIZE–includeINCLUDE–include-fileINCLUDE_FILE–excludeEXCLUDE–include-manifestINCLUDE_MANIFEST–test-groupsTEST_GROUPS_FILE–skip-timeout–skip-crash–skip-implementation-status{not-implementing,backlog,implementing}–enable-webtransport-h3–no-enable-webtransport-h3–tagTAGS–exclude-tagEXCLUDE_TAGS–default-exclude–debuggerDEBUGGER–debugger-argsDEBUGGER_ARGS–rerunRERUN–repeatREPEAT–repeat-until-unexpected–retry-unexpectedRETRY_UNEXPECTED–pause-after-test–no-pause-after-test–debug-test–pause-on-unexpected–no-restart-on-unexpected–symbols-pathSYMBOLS_PATH–stackwalk-binarySTACKWALK_BINARY–pdb–leak-check–no-leak-check–adb-binaryADB_BINARY–package-namePACKAGE_NAME–keep-app-data-directory–device-serialDEVICE_SERIAL–binaryBINARY–binary-argBINARY_ARGS–webdriver-binaryBINARY–webdriver-argWEBDRIVER_ARGS–metadataMETADATA_ROOT–testsTESTS_ROOT–manifestMANIFEST_PATH–run-infoRUN_INFO–product{android_webview,chrome,chrome_android,chrome_ios,chromium,edge,firefox,firefox_android,headless_shell,safari,sauce,servo,servodriver,opera,webkit,webkitgtk_minibrowser,wktr,epiphany,ladybird}–browser-versionBROWSER_VERSION–browser-channelBROWSER_CHANNEL–configCONFIG–install-fonts–no-install-fonts–font-dirFONT_DIR–inject-scriptINJECT_SCRIPT–headless–no-headless–instrument-to-fileINSTRUMENT_TO_FILE–suppress-handler-traceback–no-suppress-handler-traceback–ws-extraWS_EXTRA–debug-build|–release-build–total-chunksTOTAL_CHUNKS–this-chunkTHIS_CHUNK–chunk-type{none,hash,id_hash,dir_hash}–ssl-type{openssl,pregenerated,none}–openssl-binaryOPENSSL_BINARY–certutil-binaryCERTUTIL_BINARY–ca-cert-pathCA_CERT_PATH–host-key-pathHOST_KEY_PATH–host-cert-pathHOST_CERT_PATH–prefs-rootPREFS_ROOT–preload-browser–no-preload-browser–disable-e10s–disable-fission–stackfix-dirSTACKFIX_DIR–specialpowers-pathSPECIALPOWERS_PATH–setprefPREF=VALUE–reftest-internal–reftest-external–reftest-screenshot{always,fail,unexpected}–chaosCHAOS_MODE_FLAGS–setenvENV–user-stylesheetUSER_STYLESHEETS–enable-mojojs–mojojs-pathMOJOJS_PATH–enable-swiftshader–enable-experimental–no-enable-experimental–enable-sanitizer–reuse-window–sauce-browserSAUCE_BROWSER–sauce-platformSAUCE_PLATFORM–sauce-versionSAUCE_VERSION–sauce-buildSAUCE_BUILD–sauce-tagsSAUCE_TAGS…–sauce-tunnel-idSAUCE_TUNNEL_ID–sauce-userSAUCE_USER–sauce-keySAUCE_KEY–sauce-connect-binarySAUCE_CONNECT_BINARY–sauce-init-timeoutSAUCE_INIT_TIMEOUT–sauce-connect-argSAUCE_CONNECT_ARGS–github-checks-text-fileGITHUB_CHECKS_TEXT_FILE–webkit-portWEBKIT_PORT–kill-safari–log-rawLOG_RAW–log-unittestLOG_UNITTEST–log-xunitLOG_XUNIT–log-htmlLOG_HTML–log-machLOG_MACH–log-tbplLOG_TBPL–log-groupedLOG_GROUPED–log-chromiumLOG_CHROMIUM–log-wptreportLOG_WPTREPORT–log-wptscreenshotLOG_WPTSCREENSHOT–log-raw-levelLOG_RAW_LEVEL–log-mach-verbose–log-mach-levelLOG_MACH_LEVEL–log-mach-bufferLOG_MACH_BUFFER–log-mach-screenshot–log-mach-no-screenshot–log-tbpl-compact–log-tbpl-levelLOG_TBPL_LEVEL–log-tbpl-bufferLOG_TBPL_BUFFER–log-wptscreenshot-apiLOG_WPTSCREENSHOT_APIproducttest_list…
Positional Arguments¶ productBrowser to run tests in
test_listList of URLs for tests to run, or paths including tests to run. (equivalent to –include)
productBrowser to run tests in
test_listList of URLs for tests to run, or paths including tests to run. (equivalent to –include)
Named Arguments¶ –channelPossible choices: stable, release, beta, dev, canary, nightly, preview, experimental
Name of browser release channel (default: nightly). “stable” and “release” are synonyms for the latest browser stable release; “beta” is the beta release; “dev” is only meaningful for Chrome (i.e. Chrome Dev); “nightly”, “experimental”, and “preview” are all synonyms for the latest available development or trunk release. (For WebDriver installs, we attempt to select an appropriate, compatible version for the latest browser release on the selected channel.) This flag overrides –browser-channel.
Default: “nightly”
–affectedRun affected tests since revish
–yes, -yDon’t prompt before installing components
Default: True
–install-browserInstall the browser from the release channel specified by –channel (or the nightly channel by default).
Default: False
–install-webdriverInstall WebDriver from the release channel specified by –channel (or the nightly channel by default).
Default: False
–logcat-dirDirectory to write Android logcat files to
–manifest-updateRegenerate the test manifest.
–no-manifest-updatePrevent regeneration of the test manifest.
Default: True
–manifest-downloadAttempt to download a preexisting manifest when updating.
–no-manifest-downloadPrevent download of the test manifest.
Default: True
–timeout-multiplierMultiplier relative to standard test timeout to use
–run-by-dirSplit run into groups by directories. With a parameter,limit the depth of splits e.g. –run-by-dir=1 to split by top-leveldirectory
Default: False
-f, –fully-parallelRun every test in a separate group for fully parallelism.
Default: False
–processesNumber of simultaneous processes to use
–max-restartsMaximum number of browser restart retries
Default: 5
–no-capture-stdioDon’t capture stdio and write to logging
Default: False
–no-fail-on-unexpectedExit with status code 0 when test expectations are violated
Default: True
–no-fail-on-unexpected-passExit with status code 0 when all unexpected results are PASS
Default: True
–no-restart-on-new-groupDon’t restart test runner when start a new test group
Default: True
–verifyRun a stability check on the selected tests
Default: False
–verify-no-chaos-modeDisable chaos mode when running on Firefox
Default: True
–verify-chaos-modeEnable chaos mode when running on Firefox
Default: True
–verify-no-output-resultsPrints individuals test results and messages
Default: True
–verify-output-resultsDisable printing individuals test results and messages
Default: True
–debug-buildBuild is a debug build (overrides any mozinfo file)
–release-buildBuild is a release (overrides any mozinfo file)
–channelPossible choices: stable, release, beta, dev, canary, nightly, preview, experimental
Name of browser release channel (default: nightly). “stable” and “release” are synonyms for the latest browser stable release; “beta” is the beta release; “dev” is only meaningful for Chrome (i.e. Chrome Dev); “nightly”, “experimental”, and “preview” are all synonyms for the latest available development or trunk release. (For WebDriver installs, we attempt to select an appropriate, compatible version for the latest browser release on the selected channel.) This flag overrides –browser-channel.
Default: “nightly”
–affectedRun affected tests since revish
–yes, -yDon’t prompt before installing components
Default: True
–install-browserInstall the browser from the release channel specified by –channel (or the nightly channel by default).
Default: False
–install-webdriverInstall WebDriver from the release channel specified by –channel (or the nightly channel by default).
Default: False
–logcat-dirDirectory to write Android logcat files to
–manifest-updateRegenerate the test manifest.
–no-manifest-updatePrevent regeneration of the test manifest.
Default: True
–manifest-downloadAttempt to download a preexisting manifest when updating.
–no-manifest-downloadPrevent download of the test manifest.
Default: True
–timeout-multiplierMultiplier relative to standard test timeout to use
–run-by-dirSplit run into groups by directories. With a parameter,limit the depth of splits e.g. –run-by-dir=1 to split by top-leveldirectory
Default: False
-f, –fully-parallelRun every test in a separate group for fully parallelism.
Default: False
–processesNumber of simultaneous processes to use
–max-restartsMaximum number of browser restart retries
Default: 5
–no-capture-stdioDon’t capture stdio and write to logging
Default: False
–no-fail-on-unexpectedExit with status code 0 when test expectations are violated
Default: True
–no-fail-on-unexpected-passExit with status code 0 when all unexpected results are PASS
Default: True
–no-restart-on-new-groupDon’t restart test runner when start a new test group
Default: True
–verifyRun a stability check on the selected tests
Default: False
–verify-no-chaos-modeDisable chaos mode when running on Firefox
Default: True
–verify-chaos-modeEnable chaos mode when running on Firefox
Default: True
–verify-no-output-resultsPrints individuals test results and messages
Default: True
–verify-output-resultsDisable printing individuals test results and messages
Default: True
–debug-buildBuild is a debug build (overrides any mozinfo file)
–release-buildBuild is a release (overrides any mozinfo file)
Mode¶ –list-test-groupsList the top level directories containing tests that will run.
Default: False
–list-disabledList the tests that are disabled on the current platform
Default: False
–list-testsList all tests that will run
Default: False
–verify-log-fullOutput per-iteration test results when running verify
Default: False
–verify-repeat-loopNumber of iterations for a run that reloads each test without restart.
Default: 10
–verify-repeat-restartNumber of iterations, for a run that restarts the runner between each iteration
Default: 5
–verify-max-timeThe maximum number of minutes for the job to run
–repeat-max-timeThe maximum number of minutes for the test suite to attempt repeat runs
Default: 100
–list-test-groupsList the top level directories containing tests that will run.
Default: False
–list-disabledList the tests that are disabled on the current platform
Default: False
–list-testsList all tests that will run
Default: False
–verify-log-fullOutput per-iteration test results when running verify
Default: False
–verify-repeat-loopNumber of iterations for a run that reloads each test without restart.
Default: 10
–verify-repeat-restartNumber of iterations, for a run that restarts the runner between each iteration
Default: 5
–verify-max-timeThe maximum number of minutes for the job to run
–repeat-max-timeThe maximum number of minutes for the test suite to attempt repeat runs
Default: 100
Test Selection¶ –test-typesPossible choices: crashtest, testharness, reftest, print-reftest, wdspec
Test types to run
Default: {‘crashtest’, ‘testharness’, ‘reftest’, ‘print-reftest’, ‘wdspec’}
–subsuite-filePath to JSON file containing subsuite configuration
–subsuiteSubsuite names to run. Runs all subsuites when omitted.
–small-subsuite-sizeMaximum number of tests a subsuite can have to be treated as small subsuite.Tests from a small subsuite will be grouped in one group.
Default: 50
–includeURL prefix to include
–include-fileA file listing URL prefix for tests
–excludeURL prefix to exclude
–include-manifestPath to manifest listing tests to include
–test-groupsPath to json file containing a mapping {group_name: test_ids}
–skip-timeoutSkip tests that are expected to time out
Default: False
–skip-crashSkip tests that are expected to crash
Default: False
–skip-implementation-statusPossible choices: not-implementing, backlog, implementing
Skip tests that have the given implementation status
–enable-webtransport-h3Enable tests that require WebTransport over HTTP/3 server (default: false)
–no-enable-webtransport-h3Do not enable WebTransport tests on experimental channels
Default: True
–tagLabels applied to tests to include in the run. Labels starting dir: are equivalent to top-level directories.
–exclude-tagLabels applied to tests to exclude in the run. Takes precedence over –tag. Labels starting dir: are equivalent to top-level directories.
–default-excludeOnly run the tests explicitly given in arguments. No tests will run if the list is empty, and the program will exit with status code 0.
Default: False
–test-typesPossible choices: crashtest, testharness, reftest, print-reftest, wdspec
Test types to run
Default: {‘crashtest’, ‘testharness’, ‘reftest’, ‘print-reftest’, ‘wdspec’}
–subsuite-filePath to JSON file containing subsuite configuration
–subsuiteSubsuite names to run. Runs all subsuites when omitted.
–small-subsuite-sizeMaximum number of tests a subsuite can have to be treated as small subsuite.Tests from a small subsuite will be grouped in one group.
Default: 50
–includeURL prefix to include
–include-fileA file listing URL prefix for tests
–excludeURL prefix to exclude
–include-manifestPath to manifest listing tests to include
–test-groupsPath to json file containing a mapping {group_name: test_ids}
–skip-timeoutSkip tests that are expected to time out
Default: False
–skip-crashSkip tests that are expected to crash
Default: False
–skip-implementation-statusPossible choices: not-implementing, backlog, implementing
Skip tests that have the given implementation status
–enable-webtransport-h3Enable tests that require WebTransport over HTTP/3 server (default: false)
–no-enable-webtransport-h3Do not enable WebTransport tests on experimental channels
Default: True
–tagLabels applied to tests to include in the run. Labels starting dir: are equivalent to top-level directories.
–exclude-tagLabels applied to tests to exclude in the run. Takes precedence over –tag. Labels starting dir: are equivalent to top-level directories.
–default-excludeOnly run the tests explicitly given in arguments. No tests will run if the list is empty, and the program will exit with status code 0.
Default: False
Debugging¶ –debuggerrun under a debugger, e.g. gdb or valgrind
–debugger-argsarguments to the debugger
–rerunNumber of times to re run each test without restarts
Default: 1
–repeatNumber of times to run the tests, restarting between each run
Default: 1
–repeat-until-unexpectedRun tests in a loop until one returns an unexpected result
–retry-unexpectedMaximum number of times to retry unexpected tests. A test is retried until it gets one of the expected status, or until it exhausts the maximum number of retries.
Default: 0
–pause-after-testHalt the test runner after each test (this happens by default if only a single test is run)
–no-pause-after-testDon’t halt the test runner irrespective of the number of tests run
Default: True
–debug-testRun tests with additional debugging features enabled
Default: False
–pause-on-unexpectedHalt the test runner when an unexpected result is encountered
Default: False
–no-restart-on-unexpectedDon’t restart on an unexpected result
Default: True
–symbols-pathPath or url to symbols file used to analyse crash minidumps.
–stackwalk-binaryPath to stackwalker program used to analyse minidumps.
–pdbDrop into pdb on python exception
Default: False
–leak-checkEnable leak checking for supported browsers (Gecko: enabled by default for debug builds, silently ignored for opt, mobile)
–no-leak-checkDisable leak checking
–debuggerrun under a debugger, e.g. gdb or valgrind
–debugger-argsarguments to the debugger
–rerunNumber of times to re run each test without restarts
Default: 1
–repeatNumber of times to run the tests, restarting between each run
Default: 1
–repeat-until-unexpectedRun tests in a loop until one returns an unexpected result
–retry-unexpectedMaximum number of times to retry unexpected tests. A test is retried until it gets one of the expected status, or until it exhausts the maximum number of retries.
Default: 0
–pause-after-testHalt the test runner after each test (this happens by default if only a single test is run)
–no-pause-after-testDon’t halt the test runner irrespective of the number of tests run
Default: True
–debug-testRun tests with additional debugging features enabled
Default: False
–pause-on-unexpectedHalt the test runner when an unexpected result is encountered
Default: False
–no-restart-on-unexpectedDon’t restart on an unexpected result
Default: True
–symbols-pathPath or url to symbols file used to analyse crash minidumps.
–stackwalk-binaryPath to stackwalker program used to analyse minidumps.
–pdbDrop into pdb on python exception
Default: False
–leak-checkEnable leak checking for supported browsers (Gecko: enabled by default for debug builds, silently ignored for opt, mobile)
–no-leak-checkDisable leak checking
Android specific arguments¶ –adb-binaryPath to adb binary to use
–package-nameAndroid package name to run tests against
–keep-app-data-directoryDon’t delete the app data directory
Default: False
–device-serialRunning Android instances to connect to, if not emulator-5554
Default:
–adb-binaryPath to adb binary to use
–package-nameAndroid package name to run tests against
–keep-app-data-directoryDon’t delete the app data directory
Default: False
–device-serialRunning Android instances to connect to, if not emulator-5554
Default:
Configuration¶ –binaryDesktop binary to run tests against
–binary-argExtra argument for the binary
Default:
–webdriver-binaryWebDriver server binary to use
–webdriver-argExtra argument for the WebDriver binary
Default:
–metadataPath to root directory containing test metadata
–testsPath to root directory containing test files
–manifestPath to test manifest (default is ${metadata_root}/MANIFEST.json)
–run-infoPath to directory containing extra json files to add to run info
–productPossible choices: android_webview, chrome, chrome_android, chrome_ios, chromium, edge, firefox, firefox_android, headless_shell, safari, sauce, servo, servodriver, opera, webkit, webkitgtk_minibrowser, wktr, epiphany, ladybird
Browser against which to run tests
–browser-versionInformative string detailing the browser release version. This is included in the run_info data.
–browser-channelInformative string detailing the browser release channel. This is included in the run_info data.
–configPath to config file
–install-fontsInstall additional system fonts on your system
–no-install-fontsDo not install additional system fonts on your system
Default: True
–font-dirPath to local font installation directory
–inject-scriptPath to script file to inject, useful for testing polyfills.
–headlessRun browser in headless mode
–no-headlessDon’t run browser in headless mode
Default: True
–instrument-to-filePath to write instrumentation logs to
–suppress-handler-tracebackDon’t write the stacktrace for exceptions in server handlers
–no-suppress-handler-tracebackWrite the stacktrace for exceptions in server handlers
Default: True
–ws-extraExtra paths containing websockets handlers
–binaryDesktop binary to run tests against
–binary-argExtra argument for the binary
Default:
–webdriver-binaryWebDriver server binary to use
–webdriver-argExtra argument for the WebDriver binary
Default:
–metadataPath to root directory containing test metadata
–testsPath to root directory containing test files
–manifestPath to test manifest (default is ${metadata_root}/MANIFEST.json)
–run-infoPath to directory containing extra json files to add to run info
–productPossible choices: android_webview, chrome, chrome_android, chrome_ios, chromium, edge, firefox, firefox_android, headless_shell, safari, sauce, servo, servodriver, opera, webkit, webkitgtk_minibrowser, wktr, epiphany, ladybird
Browser against which to run tests
–browser-versionInformative string detailing the browser release version. This is included in the run_info data.
–browser-channelInformative string detailing the browser release channel. This is included in the run_info data.
–configPath to config file
–install-fontsInstall additional system fonts on your system
–no-install-fontsDo not install additional system fonts on your system
Default: True
–font-dirPath to local font installation directory
–inject-scriptPath to script file to inject, useful for testing polyfills.
–headlessRun browser in headless mode
–no-headlessDon’t run browser in headless mode
Default: True
–instrument-to-filePath to write instrumentation logs to
–suppress-handler-tracebackDon’t write the stacktrace for exceptions in server handlers
–no-suppress-handler-tracebackWrite the stacktrace for exceptions in server handlers
Default: True
–ws-extraExtra paths containing websockets handlers
Test Chunking¶ –total-chunksTotal number of chunks to use
Default: 1
–this-chunkChunk number to run
Default: 1
–chunk-typePossible choices: none, hash, id_hash, dir_hash
Chunking type to use
–total-chunksTotal number of chunks to use
Default: 1
–this-chunkChunk number to run
Default: 1
–chunk-typePossible choices: none, hash, id_hash, dir_hash
Chunking type to use
SSL/TLS¶ –ssl-typePossible choices: openssl, pregenerated, none
Type of ssl support to enable (running without ssl may lead to spurious errors)
–openssl-binaryPath to openssl binary
Default: “openssl”
–certutil-binaryPath to certutil binary for use with Firefox + ssl
–ca-cert-pathPath to ca certificate when using pregenerated ssl certificates
–host-key-pathPath to host private key when using pregenerated ssl certificates
–host-cert-pathPath to host certificate when using pregenerated ssl certificates
–ssl-typePossible choices: openssl, pregenerated, none
Type of ssl support to enable (running without ssl may lead to spurious errors)
–openssl-binaryPath to openssl binary
Default: “openssl”
–certutil-binaryPath to certutil binary for use with Firefox + ssl
–ca-cert-pathPath to ca certificate when using pregenerated ssl certificates
–host-key-pathPath to host private key when using pregenerated ssl certificates
–host-cert-pathPath to host certificate when using pregenerated ssl certificates
Gecko-specific¶ –prefs-rootPath to the folder containing browser prefs
–preload-browserPreload a gecko instance for faster restarts
–no-preload-browserDon’t preload a gecko instance for faster restarts
–disable-e10sRun tests without electrolysis preferences
Default: True
–disable-fissionDisable fission in Gecko.
Default: False
–stackfix-dirPath to directory containing assertion stack fixing scripts
–specialpowers-pathPath to specialPowers extension xpi file
–setprefDefines an extra user preference (overrides those in prefs_root)
Default:
–reftest-internalEnable reftest runner implemented inside Marionette
–reftest-externalDisable reftest runner implemented inside Marionette
Default: True
–reftest-screenshotPossible choices: always, fail, unexpected
With –reftest-internal, when to take a screenshot
–chaosEnable chaos mode with the specified feature flag (see http://searchfox.org/mozilla-central/source/mfbt/ChaosMode.h for details). If no value is supplied, all features are activated
–prefs-rootPath to the folder containing browser prefs
–preload-browserPreload a gecko instance for faster restarts
–no-preload-browserDon’t preload a gecko instance for faster restarts
–disable-e10sRun tests without electrolysis preferences
Default: True
–disable-fissionDisable fission in Gecko.
Default: False
–stackfix-dirPath to directory containing assertion stack fixing scripts
–specialpowers-pathPath to specialPowers extension xpi file
–setprefDefines an extra user preference (overrides those in prefs_root)
Default:
–reftest-internalEnable reftest runner implemented inside Marionette
–reftest-externalDisable reftest runner implemented inside Marionette
Default: True
–reftest-screenshotPossible choices: always, fail, unexpected
With –reftest-internal, when to take a screenshot
–chaosEnable chaos mode with the specified feature flag (see http://searchfox.org/mozilla-central/source/mfbt/ChaosMode.h for details). If no value is supplied, all features are activated
GeckoView-specific¶ –setenvSet target environment variable, like FOO=BAR
Default:
–setenvSet target environment variable, like FOO=BAR
Default:
Servo-specific¶ –user-stylesheetInject a user CSS stylesheet into every test.
Default:
–user-stylesheetInject a user CSS stylesheet into every test.
Default:
Chrome-specific¶ –enable-mojojsEnable MojoJS for testing. Note that this flag is usally enabled automatically by wpt run, if it succeeds in downloading the right version of mojojs.zip or if –mojojs-path is specified.
Default: False
–mojojs-pathPath to mojojs gen/ directory. If it is not specified, wpt run will download and extract mojojs.zip into _venv2/mojojs/gen.
–enable-swiftshaderEnable SwiftShader for CPU-based 3D graphics. This can be used in environments with no hardware GPU available.
Default: False
–enable-experimentalEnable –enable-experimental-web-platform-features flag
–no-enable-experimentalDo not enable –enable-experimental-web-platform-features flag on experimental channels
Default: True
–enable-sanitizerOnly alert on sanitizer-related errors and crashes.
Default: False
–reuse-windowReuse a window across testharness.js tests where possible, which can speed up testing. Also useful for ensuring that the renderer process has a stable PID for a debugger to attach to.
Default: False
–enable-mojojsEnable MojoJS for testing. Note that this flag is usally enabled automatically by wpt run, if it succeeds in downloading the right version of mojojs.zip or if –mojojs-path is specified.
Default: False
–mojojs-pathPath to mojojs gen/ directory. If it is not specified, wpt run will download and extract mojojs.zip into _venv2/mojojs/gen.
–enable-swiftshaderEnable SwiftShader for CPU-based 3D graphics. This can be used in environments with no hardware GPU available.
Default: False
–enable-experimentalEnable –enable-experimental-web-platform-features flag
–no-enable-experimentalDo not enable –enable-experimental-web-platform-features flag on experimental channels
Default: True
–enable-sanitizerOnly alert on sanitizer-related errors and crashes.
Default: False
–reuse-windowReuse a window across testharness.js tests where possible, which can speed up testing. Also useful for ensuring that the renderer process has a stable PID for a debugger to attach to.
Default: False
Sauce Labs-specific¶ –sauce-browserSauce Labs browser name
–sauce-platformSauce Labs OS platform
–sauce-versionSauce Labs browser version
–sauce-buildSauce Labs build identifier
–sauce-tagsSauce Labs identifying tag
Default:
–sauce-tunnel-idSauce Connect tunnel identifier
–sauce-userSauce Labs user name
–sauce-keySauce Labs access key
–sauce-connect-binaryPath to Sauce Connect binary
–sauce-init-timeoutNumber of seconds to wait for Sauce Connect tunnel to be available before aborting
Default: 30
–sauce-connect-argCommand-line argument to forward to the Sauce Connect binary (repeatable)
Default:
–sauce-browserSauce Labs browser name
–sauce-platformSauce Labs OS platform
–sauce-versionSauce Labs browser version
–sauce-buildSauce Labs build identifier
–sauce-tagsSauce Labs identifying tag
Default:
–sauce-tunnel-idSauce Connect tunnel identifier
–sauce-userSauce Labs user name
–sauce-keySauce Labs access key
–sauce-connect-binaryPath to Sauce Connect binary
–sauce-init-timeoutNumber of seconds to wait for Sauce Connect tunnel to be available before aborting
Default: 30
–sauce-connect-argCommand-line argument to forward to the Sauce Connect binary (repeatable)
Default:
Taskcluster-specific¶ –github-checks-text-filePath to GitHub checks output file
–github-checks-text-filePath to GitHub checks output file
WebKit-specific¶ –webkit-portWebKit port
–webkit-portWebKit port
Safari-specific¶ –kill-safariKill Safari when stopping the browser
Default: False
–kill-safariKill Safari when stopping the browser
Default: False
Output Logging¶ Each option represents a possible logging format and takes a filename to write that format to, or ‘-’ to write to stdout. Some options are provided by the mozlog utility; see https://firefox-source-docs.mozilla.org/mozbase/mozlog.html for extended documentation.
–log-rawRaw structured log messages (provided by mozlog)
–log-unittestUnittest style output (provided by mozlog)
–log-xunitxUnit compatible XML (provided by mozlog)
–log-htmlHTML report (provided by mozlog)
–log-machHuman-readable output (provided by mozlog)
–log-tbplTBPL style log format (provided by mozlog)
–log-groupedGrouped summary of test results (provided by mozlog)
–log-chromiumChromium Layout Tests format
–log-wptreportwptreport format
–log-wptscreenshotwpt.fyi screenshots
–log-raw-levelA least log level to subscribe to for the given formatter (debug, info, error, etc.)
–log-mach-verboseEnables verbose mode for the given formatter.
–log-mach-levelA least log level to subscribe to for the given formatter (debug, info, error, etc.)
–log-mach-bufferIf specified, enables message buffering at the given buffer size limit.
–log-mach-screenshotEnable logging reftest-analyzer compatible screenshot data.
–log-mach-no-screenshotDisable logging reftest-analyzer compatible screenshot data.
–log-tbpl-compactEnables compact mode for the given formatter.
–log-tbpl-levelA least log level to subscribe to for the given formatter (debug, info, error, etc.)
–log-tbpl-bufferIf specified, enables message buffering at the given buffer size limit.
–log-wptscreenshot-apiCache API (default: https://wpt.fyi/api/screenshots/hashes)
Each option represents a possible logging format and takes a filename to write that format to, or ‘-’ to write to stdout. Some options are provided by the mozlog utility; see https://firefox-source-docs.mozilla.org/mozbase/mozlog.html for extended documentation.
–log-rawRaw structured log messages (provided by mozlog)
–log-unittestUnittest style output (provided by mozlog)
–log-xunitxUnit compatible XML (provided by mozlog)
–log-htmlHTML report (provided by mozlog)
–log-machHuman-readable output (provided by mozlog)
–log-tbplTBPL style log format (provided by mozlog)
–log-groupedGrouped summary of test results (provided by mozlog)
–log-chromiumChromium Layout Tests format
–log-wptreportwptreport format
–log-wptscreenshotwpt.fyi screenshots
–log-raw-levelA least log level to subscribe to for the given formatter (debug, info, error, etc.)
–log-mach-verboseEnables verbose mode for the given formatter.
–log-mach-levelA least log level to subscribe to for the given formatter (debug, info, error, etc.)
–log-mach-bufferIf specified, enables message buffering at the given buffer size limit.
–log-mach-screenshotEnable logging reftest-analyzer compatible screenshot data.
–log-mach-no-screenshotDisable logging reftest-analyzer compatible screenshot data.
–log-tbpl-compactEnables compact mode for the given formatter.
–log-tbpl-levelA least log level to subscribe to for the given formatter (debug, info, error, etc.)
–log-tbpl-bufferIf specified, enables message buffering at the given buffer size limit.
–log-wptscreenshot-apiCache API (default: https://wpt.fyi/api/screenshots/hashes)
create¶ Create a new wpt test
wptcreate-h–no-editor-eEDITOR–long-timeout–overwrite-r-mREF–mismatch–wait–tests-rootTESTS_ROOTpath
Positional Arguments¶ pathPath to the test file
Named Arguments¶ –no-editorDon’t try to open the test in an editor
Default: False
-e, –editorEditor to use
–long-timeoutTest should be given a long timeout (typically 60s rather than 10s, but varies depending on environment)
Default: False
–overwriteAllow overwriting an existing test file
Default: False
-r, –reftestCreate a reftest rather than a testharness (js) test
Default: False
-m, –referencePath to the reference file
–mismatchCreate a mismatch reftest
Default: False
–waitCreate a reftest that waits until takeScreenshot() is called
Default: False
–tests-rootPath to the root of the wpt directory
Default: “/home/runner/work/wpt/wpt/tools/wpt/../..”
Create a new wpt test
wptcreate-h–no-editor-eEDITOR–long-timeout–overwrite-r-mREF–mismatch–wait–tests-rootTESTS_ROOTpath
Positional Arguments¶ pathPath to the test file
pathPath to the test file
Named Arguments¶ –no-editorDon’t try to open the test in an editor
Default: False
-e, –editorEditor to use
–long-timeoutTest should be given a long timeout (typically 60s rather than 10s, but varies depending on environment)
Default: False
–overwriteAllow overwriting an existing test file
Default: False
-r, –reftestCreate a reftest rather than a testharness (js) test
Default: False
-m, –referencePath to the reference file
–mismatchCreate a mismatch reftest
Default: False
–waitCreate a reftest that waits until takeScreenshot() is called
Default: False
–tests-rootPath to the root of the wpt directory
Default: “/home/runner/work/wpt/wpt/tools/wpt/../..”
–no-editorDon’t try to open the test in an editor
Default: False
-e, –editorEditor to use
–long-timeoutTest should be given a long timeout (typically 60s rather than 10s, but varies depending on environment)
Default: False
–overwriteAllow overwriting an existing test file
Default: False
-r, –reftestCreate a reftest rather than a testharness (js) test
Default: False
-m, –referencePath to the reference file
–mismatchCreate a mismatch reftest
Default: False
–waitCreate a reftest that waits until takeScreenshot() is called
Default: False
–tests-rootPath to the root of the wpt directory
Default: “/home/runner/work/wpt/wpt/tools/wpt/../..”
update-expectations¶ Update expectations files from raw logs.
wptupdate-expectations-h–configCONFIG–metadataMETADATA_ROOT–testsTESTS_ROOT–manifestMANIFEST_PATH–full–disable-intermittentDISABLE_INTERMITTENT–update-intermittent–remove-intermittent–no-remove-obsolete–properties-filePROPERTIES_FILE–no-properties-file–extra-propertyEXTRA_PROPERTY–log-rawLOG_RAW–log-unittestLOG_UNITTEST–log-xunitLOG_XUNIT–log-htmlLOG_HTML–log-machLOG_MACH–log-tbplLOG_TBPL–log-groupedLOG_GROUPED–log-chromiumLOG_CHROMIUM–log-wptreportLOG_WPTREPORT–log-wptscreenshotLOG_WPTSCREENSHOT–log-raw-levelLOG_RAW_LEVEL–log-mach-verbose–log-mach-levelLOG_MACH_LEVEL–log-mach-bufferLOG_MACH_BUFFER–log-mach-screenshot–log-mach-no-screenshot–log-tbpl-compact–log-tbpl-levelLOG_TBPL_LEVEL–log-tbpl-bufferLOG_TBPL_BUFFER–log-wptscreenshot-apiLOG_WPTSCREENSHOT_APIrun_log…
Positional Arguments¶ run_logLog file from run of tests
Named Arguments¶ –configPath to config file
–metadataPath to the folder containing test metadata
–testsPath to web-platform-tests
–manifestPath to test manifest (default is ${metadata_root}/MANIFEST.json)
–fullFor all tests that are updated, remove any existing conditions and missing subtests
Default: False
–disable-intermittentReason for disabling tests. When updating test results, disable tests that have inconsistent results across many runs with the given reason.
–update-intermittentUpdate test metadata with expected intermittent statuses.
Default: False
–remove-intermittentRemove obsolete intermittent statuses from expected statuses.
Default: False
–no-remove-obsoleteDon’t remove metadata files that no longer correspond to a test file
Default: True
–properties-filePath to a JSON file containing run_info properties to use in update. This must be of the form{“properties”: <name>, “dependents”: {<property name>: <name>}}
–no-properties-fileDon’t use the default properties file at ${metadata_root}/update_properties.json, even if it exists.
Default: False
–extra-propertyExtra property from run_info.json to use in metadata update.
Default:
Output Logging¶ Each option represents a possible logging format and takes a filename to write that format to, or ‘-’ to write to stdout. Some options are provided by the mozlog utility; see https://firefox-source-docs.mozilla.org/mozbase/mozlog.html for extended documentation.
–log-rawRaw structured log messages (provided by mozlog)
–log-unittestUnittest style output (provided by mozlog)
–log-xunitxUnit compatible XML (provided by mozlog)
–log-htmlHTML report (provided by mozlog)
–log-machHuman-readable output (provided by mozlog)
–log-tbplTBPL style log format (provided by mozlog)
–log-groupedGrouped summary of test results (provided by mozlog)
–log-chromiumChromium Layout Tests format
–log-wptreportwptreport format
–log-wptscreenshotwpt.fyi screenshots
–log-raw-levelA least log level to subscribe to for the given formatter (debug, info, error, etc.)
–log-mach-verboseEnables verbose mode for the given formatter.
–log-mach-levelA least log level to subscribe to for the given formatter (debug, info, error, etc.)
–log-mach-bufferIf specified, enables message buffering at the given buffer size limit.
–log-mach-screenshotEnable logging reftest-analyzer compatible screenshot data.
–log-mach-no-screenshotDisable logging reftest-analyzer compatible screenshot data.
–log-tbpl-compactEnables compact mode for the given formatter.
–log-tbpl-levelA least log level to subscribe to for the given formatter (debug, info, error, etc.)
–log-tbpl-bufferIf specified, enables message buffering at the given buffer size limit.
–log-wptscreenshot-apiCache API (default: https://wpt.fyi/api/screenshots/hashes)
Update expectations files from raw logs.
wptupdate-expectations-h–configCONFIG–metadataMETADATA_ROOT–testsTESTS_ROOT–manifestMANIFEST_PATH–full–disable-intermittentDISABLE_INTERMITTENT–update-intermittent–remove-intermittent–no-remove-obsolete–properties-filePROPERTIES_FILE–no-properties-file–extra-propertyEXTRA_PROPERTY–log-rawLOG_RAW–log-unittestLOG_UNITTEST–log-xunitLOG_XUNIT–log-htmlLOG_HTML–log-machLOG_MACH–log-tbplLOG_TBPL–log-groupedLOG_GROUPED–log-chromiumLOG_CHROMIUM–log-wptreportLOG_WPTREPORT–log-wptscreenshotLOG_WPTSCREENSHOT–log-raw-levelLOG_RAW_LEVEL–log-mach-verbose–log-mach-levelLOG_MACH_LEVEL–log-mach-bufferLOG_MACH_BUFFER–log-mach-screenshot–log-mach-no-screenshot–log-tbpl-compact–log-tbpl-levelLOG_TBPL_LEVEL–log-tbpl-bufferLOG_TBPL_BUFFER–log-wptscreenshot-apiLOG_WPTSCREENSHOT_APIrun_log…
Positional Arguments¶ run_logLog file from run of tests
run_logLog file from run of tests
Named Arguments¶ –configPath to config file
–metadataPath to the folder containing test metadata
–testsPath to web-platform-tests
–manifestPath to test manifest (default is ${metadata_root}/MANIFEST.json)
–fullFor all tests that are updated, remove any existing conditions and missing subtests
Default: False
–disable-intermittentReason for disabling tests. When updating test results, disable tests that have inconsistent results across many runs with the given reason.
–update-intermittentUpdate test metadata with expected intermittent statuses.
Default: False
–remove-intermittentRemove obsolete intermittent statuses from expected statuses.
Default: False
–no-remove-obsoleteDon’t remove metadata files that no longer correspond to a test file
Default: True
–properties-filePath to a JSON file containing run_info properties to use in update. This must be of the form{“properties”: <name>, “dependents”: {<property name>: <name>}}
–no-properties-fileDon’t use the default properties file at ${metadata_root}/update_properties.json, even if it exists.
Default: False
–extra-propertyExtra property from run_info.json to use in metadata update.
Default:
–configPath to config file
–metadataPath to the folder containing test metadata
–testsPath to web-platform-tests
–manifestPath to test manifest (default is ${metadata_root}/MANIFEST.json)
–fullFor all tests that are updated, remove any existing conditions and missing subtests
Default: False
–disable-intermittentReason for disabling tests. When updating test results, disable tests that have inconsistent results across many runs with the given reason.
–update-intermittentUpdate test metadata with expected intermittent statuses.
Default: False
–remove-intermittentRemove obsolete intermittent statuses from expected statuses.
Default: False
–no-remove-obsoleteDon’t remove metadata files that no longer correspond to a test file
Default: True
–properties-filePath to a JSON file containing run_info properties to use in update. This must be of the form{“properties”: <name>, “dependents”: {<property name>: <name>}}
–no-properties-fileDon’t use the default properties file at ${metadata_root}/update_properties.json, even if it exists.
Default: False
–extra-propertyExtra property from run_info.json to use in metadata update.
Default:
Output Logging¶ Each option represents a possible logging format and takes a filename to write that format to, or ‘-’ to write to stdout. Some options are provided by the mozlog utility; see https://firefox-source-docs.mozilla.org/mozbase/mozlog.html for extended documentation.
–log-rawRaw structured log messages (provided by mozlog)
–log-unittestUnittest style output (provided by mozlog)
–log-xunitxUnit compatible XML (provided by mozlog)
–log-htmlHTML report (provided by mozlog)
–log-machHuman-readable output (provided by mozlog)
–log-tbplTBPL style log format (provided by mozlog)
–log-groupedGrouped summary of test results (provided by mozlog)
–log-chromiumChromium Layout Tests format
–log-wptreportwptreport format
–log-wptscreenshotwpt.fyi screenshots
–log-raw-levelA least log level to subscribe to for the given formatter (debug, info, error, etc.)
–log-mach-verboseEnables verbose mode for the given formatter.
–log-mach-levelA least log level to subscribe to for the given formatter (debug, info, error, etc.)
–log-mach-bufferIf specified, enables message buffering at the given buffer size limit.
–log-mach-screenshotEnable logging reftest-analyzer compatible screenshot data.
–log-mach-no-screenshotDisable logging reftest-analyzer compatible screenshot data.
–log-tbpl-compactEnables compact mode for the given formatter.
–log-tbpl-levelA least log level to subscribe to for the given formatter (debug, info, error, etc.)
–log-tbpl-bufferIf specified, enables message buffering at the given buffer size limit.
–log-wptscreenshot-apiCache API (default: https://wpt.fyi/api/screenshots/hashes)
Each option represents a possible logging format and takes a filename to write that format to, or ‘-’ to write to stdout. Some options are provided by the mozlog utility; see https://firefox-source-docs.mozilla.org/mozbase/mozlog.html for extended documentation.
–log-rawRaw structured log messages (provided by mozlog)
–log-unittestUnittest style output (provided by mozlog)
–log-xunitxUnit compatible XML (provided by mozlog)
–log-htmlHTML report (provided by mozlog)
–log-machHuman-readable output (provided by mozlog)
–log-tbplTBPL style log format (provided by mozlog)
–log-groupedGrouped summary of test results (provided by mozlog)
–log-chromiumChromium Layout Tests format
–log-wptreportwptreport format
–log-wptscreenshotwpt.fyi screenshots
–log-raw-levelA least log level to subscribe to for the given formatter (debug, info, error, etc.)
–log-mach-verboseEnables verbose mode for the given formatter.
–log-mach-levelA least log level to subscribe to for the given formatter (debug, info, error, etc.)
–log-mach-bufferIf specified, enables message buffering at the given buffer size limit.
–log-mach-screenshotEnable logging reftest-analyzer compatible screenshot data.
–log-mach-no-screenshotDisable logging reftest-analyzer compatible screenshot data.
–log-tbpl-compactEnables compact mode for the given formatter.
–log-tbpl-levelA least log level to subscribe to for the given formatter (debug, info, error, etc.)
–log-tbpl-bufferIf specified, enables message buffering at the given buffer size limit.
–log-wptscreenshot-apiCache API (default: https://wpt.fyi/api/screenshots/hashes)
files-changed¶ Get a list of files that have changed
wptfiles-changed-h–ignore-ruleIGNORE_RULE–modified–new–show-type–nullrevish
Positional Arguments¶ revishCommits to consider. Defaults to the commits on the current branch
Named Arguments¶ –ignore-ruleOverride the rules for paths to exclude from lists of changes. Rules are paths relative to the test root, with * before a separator or the end matching anything other than a path separator and ** in that position matching anything. This flag can be used multiple times for multiple rules. Specifying this flag overrides the default: resources/testharness*, resources/testdriver*
–modifiedInclude files under version control that have been modified or staged
Default: False
–newInclude files in the worktree that are not in version control
Default: False
–show-typePrint the test type along with each affected test
Default: False
–nullSeparate items with a null byte
Default: False
Get a list of files that have changed
wptfiles-changed-h–ignore-ruleIGNORE_RULE–modified–new–show-type–nullrevish
Positional Arguments¶ revishCommits to consider. Defaults to the commits on the current branch
revishCommits to consider. Defaults to the commits on the current branch
Named Arguments¶ –ignore-ruleOverride the rules for paths to exclude from lists of changes. Rules are paths relative to the test root, with * before a separator or the end matching anything other than a path separator and ** in that position matching anything. This flag can be used multiple times for multiple rules. Specifying this flag overrides the default: resources/testharness*, resources/testdriver*
–modifiedInclude files under version control that have been modified or staged
Default: False
–newInclude files in the worktree that are not in version control
Default: False
–show-typePrint the test type along with each affected test
Default: False
–nullSeparate items with a null byte
Default: False
–ignore-ruleOverride the rules for paths to exclude from lists of changes. Rules are paths relative to the test root, with * before a separator or the end matching anything other than a path separator and ** in that position matching anything. This flag can be used multiple times for multiple rules. Specifying this flag overrides the default: resources/testharness*, resources/testdriver*
–modifiedInclude files under version control that have been modified or staged
Default: False
–newInclude files in the worktree that are not in version control
Default: False
–show-typePrint the test type along with each affected test
Default: False
–nullSeparate items with a null byte
Default: False
tests-affected¶ Get a list of tests affected by changes
wpttests-affected-h–ignore-ruleIGNORE_RULE–modified–new–show-type–null–metadataMETADATA_ROOTrevish
Positional Arguments¶ revishCommits to consider. Defaults to the commits on the current branch
Named Arguments¶ –ignore-ruleOverride the rules for paths to exclude from lists of changes. Rules are paths relative to the test root, with * before a separator or the end matching anything other than a path separator and ** in that position matching anything. This flag can be used multiple times for multiple rules. Specifying this flag overrides the default: resources/testharness*, resources/testdriver*
–modifiedInclude files under version control that have been modified or staged
Default: False
–newInclude files in the worktree that are not in version control
Default: False
–show-typePrint the test type along with each affected test
Default: False
–nullSeparate items with a null byte
Default: False
–metadataDirectory that will contain MANIFEST.json
Default: “/home/runner/work/wpt/wpt”
Get a list of tests affected by changes
wpttests-affected-h–ignore-ruleIGNORE_RULE–modified–new–show-type–null–metadataMETADATA_ROOTrevish
Positional Arguments¶ revishCommits to consider. Defaults to the commits on the current branch
revishCommits to consider. Defaults to the commits on the current branch
Named Arguments¶ –ignore-ruleOverride the rules for paths to exclude from lists of changes. Rules are paths relative to the test root, with * before a separator or the end matching anything other than a path separator and ** in that position matching anything. This flag can be used multiple times for multiple rules. Specifying this flag overrides the default: resources/testharness*, resources/testdriver*
–modifiedInclude files under version control that have been modified or staged
Default: False
–newInclude files in the worktree that are not in version control
Default: False
–show-typePrint the test type along with each affected test
Default: False
–nullSeparate items with a null byte
Default: False
–metadataDirectory that will contain MANIFEST.json
Default: “/home/runner/work/wpt/wpt”
–ignore-ruleOverride the rules for paths to exclude from lists of changes. Rules are paths relative to the test root, with * before a separator or the end matching anything other than a path separator and ** in that position matching anything. This flag can be used multiple times for multiple rules. Specifying this flag overrides the default: resources/testharness*, resources/testdriver*
–modifiedInclude files under version control that have been modified or staged
Default: False
–newInclude files in the worktree that are not in version control
Default: False
–show-typePrint the test type along with each affected test
Default: False
–nullSeparate items with a null byte
Default: False
–metadataDirectory that will contain MANIFEST.json
Default: “/home/runner/work/wpt/wpt”
install¶ Install browser components
wptinstall-h–channel{stable,release,beta,dev,canary,nightly,preview,experimental}–download-only–renameRENAME-dDESTINATION–revisionREVISION{firefox,chrome,chromium,servo,safari,wktr}{browser,webdriver}
Positional Arguments¶ browserPossible choices: firefox, chrome, chromium, servo, safari, wktr
name of web browser product
componentPossible choices: browser, webdriver
name of component
Named Arguments¶ –channelPossible choices: stable, release, beta, dev, canary, nightly, preview, experimental
Name of browser release channel (default: nightly). “stable” and “release” are synonyms for the latest browser stable release; “beta” is the beta release; “dev” is only meaningful for Chrome (i.e. Chrome Dev); “nightly”, “experimental”, and “preview” are all synonyms for the latest available development or trunk release. (For WebDriver installs, we attempt to select an appropriate, compatible version for the latest browser release on the selected channel.) This flag overrides –browser-channel.
Default: “nightly”
–download-onlyDownload the selected component but don’t install it
Default: False
–renameFilename, excluding extension for downloaded archive (only with –download-only)
-d, –destinationfilesystem directory to place the component
–revisionChromium revision to install from snapshots
Install browser components
wptinstall-h–channel{stable,release,beta,dev,canary,nightly,preview,experimental}–download-only–renameRENAME-dDESTINATION–revisionREVISION{firefox,chrome,chromium,servo,safari,wktr}{browser,webdriver}
Positional Arguments¶ browserPossible choices: firefox, chrome, chromium, servo, safari, wktr
name of web browser product
componentPossible choices: browser, webdriver
name of component
browserPossible choices: firefox, chrome, chromium, servo, safari, wktr
name of web browser product
componentPossible choices: browser, webdriver
name of component
Named Arguments¶ –channelPossible choices: stable, release, beta, dev, canary, nightly, preview, experimental
Name of browser release channel (default: nightly). “stable” and “release” are synonyms for the latest browser stable release; “beta” is the beta release; “dev” is only meaningful for Chrome (i.e. Chrome Dev); “nightly”, “experimental”, and “preview” are all synonyms for the latest available development or trunk release. (For WebDriver installs, we attempt to select an appropriate, compatible version for the latest browser release on the selected channel.) This flag overrides –browser-channel.
Default: “nightly”
–download-onlyDownload the selected component but don’t install it
Default: False
–renameFilename, excluding extension for downloaded archive (only with –download-only)
-d, –destinationfilesystem directory to place the component
–revisionChromium revision to install from snapshots
–channelPossible choices: stable, release, beta, dev, canary, nightly, preview, experimental
Name of browser release channel (default: nightly). “stable” and “release” are synonyms for the latest browser stable release; “beta” is the beta release; “dev” is only meaningful for Chrome (i.e. Chrome Dev); “nightly”, “experimental”, and “preview” are all synonyms for the latest available development or trunk release. (For WebDriver installs, we attempt to select an appropriate, compatible version for the latest browser release on the selected channel.) This flag overrides –browser-channel.
Default: “nightly”
–download-onlyDownload the selected component but don’t install it
Default: False
–renameFilename, excluding extension for downloaded archive (only with –download-only)
-d, –destinationfilesystem directory to place the component
–revisionChromium revision to install from snapshots
rev-list¶ List tagged revisions at regular intervals
wptrev-list-h–epochEPOCH–max-countMAX_COUNT
Named Arguments¶ –epochregular interval of time selected to get the tagged revisions. Valid values are digits followed by h/d/w (e.x. 9h, 9d, 9w …) where the mimimun selectable interval is one hour (1h)
Default: 1d
–max-countmaximum number of revisions to be returned by the command
Default: 1
List tagged revisions at regular intervals
wptrev-list-h–epochEPOCH–max-countMAX_COUNT
Named Arguments¶ –epochregular interval of time selected to get the tagged revisions. Valid values are digits followed by h/d/w (e.x. 9h, 9d, 9w …) where the mimimun selectable interval is one hour (1h)
Default: 1d
–max-countmaximum number of revisions to be returned by the command
Default: 1
–epochregular interval of time selected to get the tagged revisions. Valid values are digits followed by h/d/w (e.x. 9h, 9d, 9w …) where the mimimun selectable interval is one hour (1h)
Default: 1d
–max-countmaximum number of revisions to be returned by the command
Default: 1
install-android-emulator¶ Setup the x86 android emulator
wptinstall-android-emulator-h–pathDEST–reinstall–prompt–no-prompt
Named Arguments¶ –pathRoot path to use for emulator tooling
–reinstallForce reinstall even if the emulator already exists
Default: False
–promptEnable confirmation prompts
Default: False
–no-promptSkip confirmation prompts
Default: True
Setup the x86 android emulator
wptinstall-android-emulator-h–pathDEST–reinstall–prompt–no-prompt
Named Arguments¶ –pathRoot path to use for emulator tooling
–reinstallForce reinstall even if the emulator already exists
Default: False
–promptEnable confirmation prompts
Default: False
–no-promptSkip confirmation prompts
Default: True
–pathRoot path to use for emulator tooling
–reinstallForce reinstall even if the emulator already exists
Default: False
–promptEnable confirmation prompts
Default: False
–no-promptSkip confirmation prompts
Default: True
start-android-emulator¶ Start the x86 android emulator
wptstart-android-emulator-h–pathDEST–reinstall–prompt–no-prompt–device-serialDEVICE_SERIAL
Named Arguments¶ –pathRoot path to use for emulator tooling
–reinstallForce reinstall even if the emulator already exists
Default: False
–promptEnable confirmation prompts
Default: False
–no-promptSkip confirmation prompts
Default: True
–device-serialDevice serial number for Android emulator, if not emulator-5554
Start the x86 android emulator
wptstart-android-emulator-h–pathDEST–reinstall–prompt–no-prompt–device-serialDEVICE_SERIAL
Named Arguments¶ –pathRoot path to use for emulator tooling
–reinstallForce reinstall even if the emulator already exists
Default: False
–promptEnable confirmation prompts
Default: False
–no-promptSkip confirmation prompts
Default: True
–device-serialDevice serial number for Android emulator, if not emulator-5554
–pathRoot path to use for emulator tooling
–reinstallForce reinstall even if the emulator already exists
Default: False
–promptEnable confirmation prompts
Default: False
–no-promptSkip confirmation prompts
Default: True
–device-serialDevice serial number for Android emulator, if not emulator-5554
web-features-manifest¶ Create the WEB_FEATURES_MANIFEST.json
wptweb-features-manifest-h-pPATH
Named Arguments¶ -p, –pathPath to manifest file.
Create the WEB_FEATURES_MANIFEST.json
wptweb-features-manifest-h-pPATH
Named Arguments¶ -p, –pathPath to manifest file.
-p, –pathPath to manifest file.
Navigation Test Suite Design
“Introduction to WPT” video transcript
Running Tests
Running Tests from the Web
Running Tests from the Local System
Install WPT
Running Tests on CI
Writing Your Own Runner
WPT Test Certificates
Writing Tests
Reviewing Tests
Project Administration
Related Topics Documentation overview
Running Tests
Running Tests from the Local System
Previous: Running Tests from the Local System
Next: Chrome and Chromium
Documentation overview
Running Tests
Running Tests from the Local System
Previous: Running Tests from the Local System
Next: Chrome and Chromium
Quick search
