Skip to content

Commit

Permalink
Perf work (#5740)
Browse files Browse the repository at this point in the history
**What's the problem this PR addresses?**

This diff implements a couple of small performance improvements.

**How did you fix it?**

- Removed `nodeUtils.builtinModules` (use `module.builtinModules`)
- Optimized some `ppath` functions on non-win32 platforms
- Removed `toFilename` (just cast to `Filename` if needed)
- The `yarnPath` same-file check now uses inodes on non-win32 platforms
- Hoisted a couple of regexps to avoid re-instantiations

| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|:---|---:|---:|---:|---:|
| `before` | 236.5 ± 2.7 | 233.2 | 241.0 | 1.06 ± 0.02 |
| `after` | 222.7 ± 3.2 | 219.6 | 230.7 | 1.00 |

**Checklist**
<!--- Don't worry if you miss something, chores are automatically
tested. -->
<!--- This checklist exists to help you remember doing the chores when
you submit a PR. -->
<!--- Put an `x` in all the boxes that apply. -->
- [x] I have read the [Contributing
Guide](https://yarnpkg.com/advanced/contributing).

<!-- See
https://yarnpkg.com/advanced/contributing#preparing-your-pr-to-be-released
for more details. -->
<!-- Check with `yarn version check` and fix with `yarn version check
-i` -->
- [x] I have set the packages that need to be released for my changes to
be effective.

<!-- The "Testing chores" workflow validates that your PR follows our
guidelines. -->
<!-- If it doesn't pass, click on it to see details as to what your PR
might be missing. -->
- [x] I will check that all automated PR checks pass before the PR gets
reviewed.

---------

Co-authored-by: Kristoffer K. <[email protected]>
  • Loading branch information
arcanis and merceyz authored Sep 21, 2023
1 parent 8748f5f commit 9287251
Show file tree
Hide file tree
Showing 22 changed files with 266 additions and 207 deletions.
42 changes: 20 additions & 22 deletions .pnp.cjs

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

38 changes: 18 additions & 20 deletions .pnp.loader.mjs

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

39 changes: 39 additions & 0 deletions .yarn/versions/0aa36442.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
releases:
"@yarnpkg/cli": major
"@yarnpkg/core": major
"@yarnpkg/fslib": major
"@yarnpkg/nm": major
"@yarnpkg/plugin-nm": major
"@yarnpkg/plugin-npm": major
"@yarnpkg/pnp": major
"@yarnpkg/pnpify": major

declined:
- "@yarnpkg/plugin-compat"
- "@yarnpkg/plugin-constraints"
- "@yarnpkg/plugin-dlx"
- "@yarnpkg/plugin-essentials"
- "@yarnpkg/plugin-exec"
- "@yarnpkg/plugin-file"
- "@yarnpkg/plugin-git"
- "@yarnpkg/plugin-github"
- "@yarnpkg/plugin-http"
- "@yarnpkg/plugin-init"
- "@yarnpkg/plugin-interactive-tools"
- "@yarnpkg/plugin-link"
- "@yarnpkg/plugin-npm-cli"
- "@yarnpkg/plugin-pack"
- "@yarnpkg/plugin-patch"
- "@yarnpkg/plugin-pnp"
- "@yarnpkg/plugin-pnpm"
- "@yarnpkg/plugin-stage"
- "@yarnpkg/plugin-typescript"
- "@yarnpkg/plugin-version"
- "@yarnpkg/plugin-workspace-tools"
- vscode-zipfs
- "@yarnpkg/builder"
- "@yarnpkg/doctor"
- "@yarnpkg/extensions"
- "@yarnpkg/libzip"
- "@yarnpkg/sdks"
- "@yarnpkg/shell"
48 changes: 24 additions & 24 deletions packages/acceptance-tests/pkg-tests-core/sources/utils/tests.ts
Original file line number Diff line number Diff line change
@@ -1,26 +1,26 @@
import {miscUtils, semverUtils} from '@yarnpkg/core';
import {PortablePath, npath, toFilename, xfs, ppath, Filename} from '@yarnpkg/fslib';
import {npmAuditTypes} from '@yarnpkg/plugin-npm-cli';
import assert from 'assert';
import crypto from 'crypto';
import finalhandler from 'finalhandler';
import https from 'https';
import {IncomingMessage, ServerResponse} from 'http';
import http from 'http';
import invariant from 'invariant';
import {AddressInfo} from 'net';
import os from 'os';
import pem from 'pem';
import semver from 'semver';
import serveStatic from 'serve-static';
import stream from 'stream';
import * as t from 'typanion';
import {promisify} from 'util';
import {v5 as uuidv5} from 'uuid';
import {Gzip} from 'zlib';

import {ExecResult} from './exec';
import * as fsUtils from './fs';
import {miscUtils, semverUtils} from '@yarnpkg/core';
import {PortablePath, npath, xfs, ppath, Filename} from '@yarnpkg/fslib';
import {npmAuditTypes} from '@yarnpkg/plugin-npm-cli';
import assert from 'assert';
import crypto from 'crypto';
import finalhandler from 'finalhandler';
import https from 'https';
import {IncomingMessage, ServerResponse} from 'http';
import http from 'http';
import invariant from 'invariant';
import {AddressInfo} from 'net';
import os from 'os';
import pem from 'pem';
import semver from 'semver';
import serveStatic from 'serve-static';
import stream from 'stream';
import * as t from 'typanion';
import {promisify} from 'util';
import {v5 as uuidv5} from 'uuid';
import {Gzip} from 'zlib';

import {ExecResult} from './exec';
import * as fsUtils from './fs';

const deepResolve = require(`super-resolve`);
const staticServer = serveStatic(npath.fromPortablePath(require(`pkg-tests-fixtures`)));
Expand Down Expand Up @@ -257,7 +257,7 @@ export const getPackageArchivePath = async (name: string, version: string): Prom
throw new Error(`Unknown version "${version}" for package "${name}"`);

const tmpDir = await xfs.mktempPromise();
const archivePath = `${tmpDir}/${toFilename(`${name}-${version}.tar.gz`)}` as PortablePath;
const archivePath = ppath.join(tmpDir, `${name}-${version}.tar.gz`);

await fsUtils.packToFile(archivePath, npath.toPortablePath(packageVersionEntry.path), {
virtualPath: npath.toPortablePath(`/package`),
Expand Down
14 changes: 7 additions & 7 deletions packages/plugin-nm/sources/NodeModulesLinker.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ import {Locator, Package, FinalizeInstallStatus, hashUtils} from
import {Linker, LinkOptions, MinimalLinkOptions, LinkType, WindowsLinkType} from '@yarnpkg/core';
import {LocatorHash, Descriptor, DependencyMeta, Configuration} from '@yarnpkg/core';
import {MessageName, Project, FetchResult, Installer} from '@yarnpkg/core';
import {PortablePath, npath, ppath, toFilename, Filename} from '@yarnpkg/fslib';
import {PortablePath, npath, ppath, Filename} from '@yarnpkg/fslib';
import {VirtualFS, xfs, FakeFS, NativePath} from '@yarnpkg/fslib';
import {ZipOpenFS} from '@yarnpkg/libzip';
import {buildNodeModulesTree} from '@yarnpkg/nm';
Expand Down Expand Up @@ -493,7 +493,7 @@ async function findInstallState(project: Project, {unrollAliases = false}: {unro
const location = ppath.join(rootPath, npath.toPortablePath(relativeLocation));
const symlinks = miscUtils.getMapWithDefault(binSymlinks, location);
for (const [name, target] of Object.entries(locationSymlinks as any)) {
symlinks.set(toFilename(name), npath.toPortablePath([location, NODE_MODULES, target].join(ppath.sep)));
symlinks.set(name as Filename, npath.toPortablePath([location, NODE_MODULES, target].join(ppath.sep)));
}
}
}
Expand Down Expand Up @@ -540,7 +540,7 @@ const removeDir = async (dir: PortablePath, options: {contentsOnly: boolean, inn
}
const entries = await xfs.readdirPromise(dir, {withFileTypes: true});
for (const entry of entries) {
const targetPath = ppath.join(dir, toFilename(entry.name));
const targetPath = ppath.join(dir, entry.name);
if (entry.isDirectory()) {
if (entry.name !== NODE_MODULES || (options && options.innerLoop)) {
await removeDir(targetPath, {innerLoop: true, contentsOnly: false});
Expand Down Expand Up @@ -688,7 +688,7 @@ const symlinkPromise = async (srcPath: PortablePath, dstPath: PortablePath, wind
};

async function atomicFileWrite(tmpDir: PortablePath, dstPath: PortablePath, content: Buffer) {
const tmpPath = ppath.join(tmpDir, toFilename(`${crypto.randomBytes(16).toString(`hex`)}.tmp`));
const tmpPath = ppath.join(tmpDir, `${crypto.randomBytes(16).toString(`hex`)}.tmp`);
try {
await xfs.writeFilePromise(tmpPath, content);
try {
Expand All @@ -713,7 +713,7 @@ async function copyFilePromise({srcPath, dstPath, entry, globalHardlinksStore, b
const contentDigest = await hashUtils.checksumFile(contentFilePath, {baseFs: xfs, algorithm: `sha1`});
if (contentDigest !== entry.digest) {
// If file content was modified by the user, or corrupted, we first move it out of the way
const tmpPath = ppath.join(globalHardlinksStore, toFilename(`${crypto.randomBytes(16).toString(`hex`)}.tmp`));
const tmpPath = ppath.join(globalHardlinksStore, `${crypto.randomBytes(16).toString(`hex`)}.tmp`);
await xfs.renamePromise(contentFilePath, tmpPath);

// Then we overwrite the temporary file, thus restorting content of original file in all the linked projects
Expand Down Expand Up @@ -997,7 +997,7 @@ async function createBinSymlinkMap(installState: NodeModulesLocatorMap, location
const binScripts = locatorScriptMap.get(node.locator)!;
for (const [filename, scriptPath] of binScripts) {
const symlinkTarget = ppath.join(location, npath.toPortablePath(scriptPath));
symlinks.set(toFilename(filename), symlinkTarget);
symlinks.set(filename, symlinkTarget);
}
for (const [childLocation, childNode] of node.children) {
const absChildLocation = ppath.join(location, childLocation);
Expand Down Expand Up @@ -1349,7 +1349,7 @@ async function persistBinSymlinks(previousBinSymlinks: BinSymlinkMap, binSymlink
// Remove outdated symlinks
await xfs.removePromise(ppath.join(binDir, name));
if (process.platform === `win32`) {
await xfs.removePromise(ppath.join(binDir, toFilename(`${name}.cmd`)));
await xfs.removePromise(ppath.join(binDir, `${name}.cmd`));
}
}
}
Expand Down
Loading

0 comments on commit 9287251

Please sign in to comment.