Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generate a list of assets to cache for offline support #1573

Merged
merged 53 commits into from
Jul 15, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
53 commits
Select commit Hold shift + click to select a range
6d9e165
ZIP WP assets
bgrgicak Jun 20, 2024
75bc2bb
Add afterWordPressInstall hook to boot
bgrgicak Jun 21, 2024
6c2b0ad
Download assets after WordPress is installed
bgrgicak Jun 21, 2024
57daebb
Merge branch 'trunk' into add/wp-static-asset-zip
bgrgicak Jun 21, 2024
42be4fc
Use getWordPressVersionFromPhp
bgrgicak Jun 21, 2024
8beee0c
Remove bootWordPress hook
bgrgicak Jun 24, 2024
6cf5fef
Use wordPressSiteUrl
bgrgicak Jun 24, 2024
1f2347e
Stream static asset response instead of awaiting
bgrgicak Jun 24, 2024
f1e2784
Use request handler to get document root
bgrgicak Jun 24, 2024
3c98912
Add warning if assets weren't downloaded
bgrgicak Jun 25, 2024
e703eff
Merge branch 'trunk' into add/wp-static-asset-zip
bgrgicak Jun 25, 2024
2e3910c
Async fetch
bgrgicak Jun 25, 2024
d2ca9c2
Update error messages
bgrgicak Jun 25, 2024
4a1f8d0
Use wpVersionToStaticAssetsDirectory
bgrgicak Jun 25, 2024
efb4b57
Use getLoadedWordPressVersion
bgrgicak Jun 25, 2024
b2ba22b
downloadWordPressAssets on load
bgrgicak Jun 26, 2024
1aad0d0
Remove logs
bgrgicak Jun 26, 2024
db8c25f
Merge branch 'trunk' into add/wp-static-asset-zip
bgrgicak Jun 26, 2024
de206a1
Merge branch 'trunk' into add/wp-static-asset-zip
bgrgicak Jun 28, 2024
a6f20bd
Empty remote asset paths after done instead of deleting it, update co…
bgrgicak Jun 28, 2024
ebfa224
Merge branch 'trunk' into add/wp-static-asset-zip
bgrgicak Jul 1, 2024
37c0b91
Update packages/playground/remote/src/lib/worker-thread.ts
bgrgicak Jul 2, 2024
ee044ae
Linter fixes
bgrgicak Jul 2, 2024
f8dab44
Build static assets
bgrgicak Jul 2, 2024
6f48337
Use unzipFile
bgrgicak Jul 4, 2024
d34f066
Fix dependencies
bgrgicak Jul 4, 2024
1feb41a
Update static asset zips
bgrgicak Jul 4, 2024
f3fe08d
Merge branch 'trunk' into add/wp-static-asset-zip
bgrgicak Jul 4, 2024
966f7bf
Generate a list assets to cache for offline support
bgrgicak Jul 4, 2024
97d95f3
Set web worker startup options with messages instead of query strings…
bgrgicak Jul 9, 2024
f0378d0
Remove unused zip delete
bgrgicak Jul 10, 2024
7c205fa
Update remote asset PR comment
bgrgicak Jul 10, 2024
e03c9ad
Merge branch 'trunk' into add/wp-static-asset-zip
bgrgicak Jul 10, 2024
9f2d128
Merge branch 'add/wp-static-asset-zip' into add/vite-cache-paths-plugin
bgrgicak Jul 10, 2024
9f1ad7c
Remove unused manifest
bgrgicak Jul 10, 2024
7c886d4
Remove worker url params
bgrgicak Jul 10, 2024
e23cbb8
Merge branch 'trunk' into add/vite-cache-paths-plugin
bgrgicak Jul 10, 2024
a746657
Update filter to exclude files instead include
bgrgicak Jul 10, 2024
2567903
Apply suggestions from code review
bgrgicak Jul 10, 2024
29f7868
Revert "Set web worker startup options with messages instead of query…
bgrgicak Jul 12, 2024
bdb6d41
Reset spawnPHPWorkerThread
bgrgicak Jul 12, 2024
194f7e9
Update patternsToNotCache list and document why files are excluded
bgrgicak Jul 12, 2024
da4b2d1
Apply suggestions from code review
bgrgicak Jul 15, 2024
0746125
Rename plugin and make file name configurable
bgrgicak Jul 15, 2024
2110676
Add distDirectoriesToList
bgrgicak Jul 15, 2024
83499b5
Rename to absoluteDirPath
bgrgicak Jul 15, 2024
9657274
Merge branch 'trunk' into add/vite-cache-paths-plugin
bgrgicak Jul 15, 2024
b5f3b9a
Update packages/vite-extensions/vite-list-assets-required-for-offline…
bgrgicak Jul 15, 2024
f2d8f7b
Address feedback
bgrgicak Jul 15, 2024
b762d99
Update packages/vite-extensions/vite-list-assets-required-for-offline…
bgrgicak Jul 15, 2024
23117c8
Merge branch 'trunk' into add/vite-cache-paths-plugin
bgrgicak Jul 15, 2024
6fbd0b7
Update packages/vite-extensions/vite-list-assets-required-for-offline…
bgrgicak Jul 15, 2024
6c79ca0
Move listFiles to top
bgrgicak Jul 15, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions packages/playground/website/vite.config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ import { fileURLToPath } from 'node:url';
import { copyFileSync, existsSync } from 'node:fs';
import { join } from 'node:path';
import { buildVersionPlugin } from '../../vite-extensions/vite-build-version';
import { listAssetsRequiredForOfflineMode } from '../../vite-extensions/vite-list-assets-required-for-offline-mode';

const proxy = {
'^/plugin-proxy': {
Expand Down Expand Up @@ -134,6 +135,13 @@ export default defineConfig(({ command, mode }) => {
}
},
} as Plugin,
/**
* Generate a list of files needed for the website to function offline.
*/
listAssetsRequiredForOfflineMode({
outputFile: 'assets-required-for-offline-mode.json',
distDirectoriesToList: ['./', '../remote', '../client'],
}) as Plugin,
],

// Configuration for building your library.
Expand Down
121 changes: 121 additions & 0 deletions packages/vite-extensions/vite-list-assets-required-for-offline-mode.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
import { readdirSync, statSync, writeFileSync } from 'node:fs';
import { join } from 'node:path';

/**
* We want to cache all files from the website root directory, except for the files listed below.
*/
const patternsToNotCache = [
/**
* Static files that are not needed for the website to function offline.
*/
'/package.json',
'/README.md',
'/.DS_Store',
'/index.cjs',
'/index.d.ts',
/\/lib\/.*/, // Remote lib files
/\/test-fixtures\/.*/, // Test fixtures
/**
* WordPress assets removed from the minified builds, for example:
*
* /wp-6.2/wp-content/themes/twentytwentyone/style.css
*
* We don't need to cache them, because they're only used in a short time
* window after the Playground is initially loaded and before they're all backfilled
* from a dedicated ZIP file shipping all the static assets for a given minified
* build.
*
* See <Assets backfilling PR link>
*/
/^\/wp-[\w+((\.\d+)?)]+\/.*/,
/**
* Demos are not needed for the website to function offline.
*/
/^\/demos\/.*/,
/**
* Files needed only by the playground.wordpress.net server.
*/
'/.htaccess',
/**
* We can't download the PHP scripts – only get their execution result. This is fine,
* we don't need them for the offline mode anyway. This includes things like plugins-proxy.php.
*/
/\/.*\.php$/,
/**
* WordPress, PHP, and SQLite files that are loaded during boot.
* Eagerly loading all the PHP and WordPress releases offered by Playground would use an
* extra ~200MB of bandwidth every time you load Playground on a new device.
*
* However, most of the time time you only want to load a specific Playground configuration.
*
* Therefore, in here we're excluding the PHP and WP releases from being loaded
* eagerly and instead we're defaulting to caching the specific release that's
* loaded anyway when booting Playgroung.
*/
/^\/assets\/php_.*\.wasm$/, // PHP WASM files
/^\/assets\/php_.*\.js$/, // PHP JS files
/^\/assets\/wp-.*\.zip$/, // Minified WordPress builds and static assets bundles
/^\/assets\/sqlite-database-integration-[\w]+\.zip/, // SQLite plugin
];

function listFiles(dirPath: string, fileList: string[] = []) {
const files = readdirSync(dirPath);

files.forEach((file) => {
const filePath = join(dirPath, file);
const fileStat = statSync(filePath);

if (fileStat.isDirectory()) {
listFiles(filePath, fileList);
} else {
fileList.push(filePath);
}
});

return fileList;
}

/**
* This Vite plugin saves a list of those files as a JSON file served on
* `playground.wordpress.net`. Playground then consults the list to
* download and cache all those files at the end of the boot process.
* .
*/
export const listAssetsRequiredForOfflineMode = ({
outputFile,
distDirectoriesToList,
}: {
outputFile: string;
distDirectoriesToList: string[];
}) => {
return {
name: 'list-assets-required-for-offline-mode',
apply: 'build',
writeBundle({ dir: outputDir }: { dir: string }) {
const files = distDirectoriesToList.flatMap((dir) => {
const absoluteDirPath = join(outputDir, dir);
console.log(`Listing files in ${absoluteDirPath}`);
return listFiles(absoluteDirPath)
.map((file) => {
file = file.replace(absoluteDirPath, '');
if (file.startsWith('/')) {
return file;
}
return `/${file}`;
})
.filter((item) => {
return !patternsToNotCache.some((pattern) => {
if (pattern instanceof RegExp) {
return pattern.test(item);
}
return pattern === item;
});
});
});
writeFileSync(
join(outputDir, outputFile),
JSON.stringify(files, null, 2)
);
},
};
};
Loading