Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow specifying the viewport dimensions when running benchmark-web-vitals #164

Merged
merged 4 commits into from
Nov 26, 2024
Merged

Allow specifying the viewport dimensions when running benchmark-web-vitals #164

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
91dbc36
Allow specifying the viewport dimensions when running benchmark-web-v…
westonruter Nov 23, 2024
89d68ab
Fix prettier issues
westonruter Nov 24, 2024
1f3b1c5
Add mobile/desktop aliases for window viewport and use Ligthouse values
westonruter Nov 26, 2024
e78501d
Update readme
westonruter Nov 26, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions cli/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -162,6 +162,7 @@ Loads the provided URLs in a headless browser several times to measure median We
* `--show-percentiles` (`-p`): Whether to show more granular percentiles instead of only the median.
* `--throttle-cpu` (`-t`): Enable CPU throttling to emulate slow CPUs.
* `--network-conditions` (`-c`): Enable emulation of network conditions (may be either "Slow 3G" or "Fast 3G").
* `--window-viewport` (`-w`): Specify the viewport window size, like "mobile" (an alias for "412x823") or "desktop" (an alias for "1350x940"). Defaults to "960x700".

#### Examples

Expand Down Expand Up @@ -195,9 +196,9 @@ To run benchmark tests for URLs from a file:
benchmark-web-vitals -f path/to/urls.txt -n 5
```

To make a request that throttles the CPU 4x while also emulating Fast 3G network conditions:
To make a request that throttles the CPU 4x while also emulating Fast 3G network conditions on a mobile viewport:
```bash
benchmark-web-vitals --url https://example.com/ -t 4 -c "Fast 3G"
benchmark-web-vitals --url https://example.com/ -t 4 -c "Fast 3G" -w "360x800"
```

### `analyze-loading-optimization`
Expand Down
65 changes: 51 additions & 14 deletions cli/commands/benchmark-web-vitals.mjs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ import round from 'lodash-es/round.js';
/** @typedef {import("puppeteer").NetworkConditions} NetworkConditions */
/** @typedef {keyof typeof import("puppeteer").networkConditions} NetworkConditionName */
/* eslint-enable jsdoc/valid-types */
/** @typedef {{width: number, height: number}} ViewportDimensions */

/**
* Internal dependencies
Expand Down Expand Up @@ -96,19 +97,25 @@ export const options = [
description:
'Enable emulation of network conditions (may be either "Slow 3G" or "Fast 3G")',
},
{
argname: '-w, --window-viewport <dimensions>',
description:
'Open page with the supplied viewport dimensions such as "mobile" (an alias for "412x823") or "desktop" (an alias for "1350x940"), defaults to "960x700"',
},
];

/**
* @typedef {Object} Params
* @property {?string} url - See above.
* @property {number} amount - See above.
* @property {?string} file - See above.
* @property {?string[]} metrics - See above.
* @property {string} output - See above.
* @property {boolean} showPercentiles - See above.
* @property {boolean} showVariance - See above.
* @property {?number} cpuThrottleFactor - See above.
* @property {?NetworkConditions} networkConditions - See above.
* @property {?string} url - See above.
* @property {number} amount - See above.
* @property {?string} file - See above.
* @property {?string[]} metrics - See above.
* @property {string} output - See above.
* @property {boolean} showPercentiles - See above.
* @property {boolean} showVariance - See above.
* @property {?number} cpuThrottleFactor - See above.
* @property {?NetworkConditions} networkConditions - See above.
* @property {?ViewportDimensions} windowViewport - See above.
*/

/**
Expand All @@ -132,9 +139,11 @@ export const options = [
* @param {boolean} opt.showVariance
* @param {?string} opt.throttleCpu
* @param {?NetworkConditionName} opt.networkConditions
* @param {?string} opt.windowViewport
* @return {Params} Parameters.
*/
function getParamsFromOptions( opt ) {
/** @type {Params} */
const params = {
url: opt.url,
amount:
Expand All @@ -151,6 +160,7 @@ function getParamsFromOptions( opt ) {
showVariance: Boolean( opt.showVariance ),
cpuThrottleFactor: null,
networkConditions: null,
windowViewport: { width: 960, height: 700 }, // Viewport similar to @wordpress/e2e-test-utils 'large' configuration.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It makes sense to have this parameter, but I wonder whether we should additionally have more than just this one default, to simplify usage.

I'm thinking about our typical mobile vs desktop differentiation. Maybe we could allow passing special string values mobile and desktop for the --window-viewport argument, and in that case rely on a specific default for that viewport type that we set?

I wonder what PageSpeed Insights uses, maybe we could align with that?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea. That did cross my mind, but I didn't implement it. I picked the 360x800 and 1920x1080 since they are the most common mobile and desktop dimensions currently. In 1f3b1c5 I've implemented the mobile and desktop aliases, and I've set their default values to correspond to what Lighthouse uses.

};

if ( isNaN( params.amount ) ) {
Expand Down Expand Up @@ -190,6 +200,34 @@ function getParamsFromOptions( opt ) {
PredefinedNetworkConditions[ opt.networkConditions ];
}

if ( opt.windowViewport ) {
if ( 'mobile' === opt.windowViewport ) {
// This corresponds to the mobile viewport tested in Lighthouse: <https://github.com/GoogleChrome/lighthouse/blob/b64b3534542c9dcaabb33d40b84ed7c93eefbd7d/core/config/constants.js#L14-L22>.
// TODO: Consider deviceScaleFactor.
params.windowViewport = {
width: 412,
height: 823,
};
} else if ( 'desktop' === opt.windowViewport ) {
// This corresponds to the mobile viewport tested in Lighthouse: <https://github.com/GoogleChrome/lighthouse/blob/b64b3534542c9dcaabb33d40b84ed7c93eefbd7d/core/config/constants.js#L28-L34>.
params.windowViewport = {
width: 1350,
height: 940,
};
} else {
const matches = opt.windowViewport.match( /^(\d+)x(\d+)$/ );
if ( ! matches ) {
throw new Error(
`Invalid window viewport dimensions: ${ opt.windowViewport }. Must be 'mobile', 'desktop', or WIDTHxHEIGHT (e.g. '1024x768')`
);
}
params.windowViewport = {
width: parseInt( matches[ 1 ] ),
height: parseInt( matches[ 2 ] ),
};
}
}

return params;
}

Expand Down Expand Up @@ -310,7 +348,7 @@ export async function handler( opt ) {

for await ( const url of getURLs( opt ) ) {
if ( logURLProgress ) {
// If also logging individial iterations, put those on a new line.
// If also logging individual iterations, put those on a new line.
if ( logIterationsProgress ) {
log( `Benchmarking URL ${ url }...` );
} else {
Expand All @@ -329,7 +367,7 @@ export async function handler( opt ) {
);
results.push( [ url, completeRequests, metrics ] );
if ( logURLProgress ) {
// If also logging individial iterations, provide more context on benchmarking which URL was completed.
// If also logging individual iterations, provide more context on benchmarking which URL was completed.
if ( logIterationsProgress ) {
log(
formats.success(
Expand Down Expand Up @@ -411,12 +449,11 @@ async function benchmarkURL(
await page.emulateNetworkConditions( params.networkConditions );
}

// Set viewport similar to @wordpress/e2e-test-utils 'large' configuration.
await page.setViewport( { width: 960, height: 700 } ); // @todo This should be configurable via command options so that mobile viewport can be loaded.
await page.setViewport( params.windowViewport );
await page
.mainFrame()
.waitForFunction(
'window.innerWidth === 960 && window.innerHeight === 700'
`window.innerWidth === ${ params.windowViewport.width } && window.innerHeight === ${ params.windowViewport.height }`
);

// Load the page.
Expand Down