Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: allow folder uploads #31165

Merged
merged 10 commits into from
Jun 12, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/src/api/class-elementhandle.md
Original file line number Diff line number Diff line change
Expand Up @@ -953,6 +953,7 @@ When all steps combined have not finished during the specified [`option: timeout

Sets the value of the file input to these file paths or files. If some of the `filePaths` are relative paths, then they
are resolved relative to the current working directory. For empty array, clears the selected files.
For inputs with a `[webkitdirectory]` attribute, only a single directory path is supported.

This method expects [ElementHandle] to point to an
[input element](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/input). However, if the element is inside the `<label>` element that has an associated [control](https://developer.mozilla.org/en-US/docs/Web/API/HTMLLabelElement/control), targets the control instead.
Expand Down
1 change: 1 addition & 0 deletions docs/src/api/class-locator.md
Original file line number Diff line number Diff line change
Expand Up @@ -2164,6 +2164,7 @@ When all steps combined have not finished during the specified [`option: timeout
* since: v1.14

Upload file or multiple files into `<input type=file>`.
For inputs with a `[webkitdirectory]` attribute, only a single directory path is supported.

**Usage**

Expand Down
1 change: 1 addition & 0 deletions docs/src/api/class-page.md
Original file line number Diff line number Diff line change
Expand Up @@ -3927,6 +3927,7 @@ An object containing additional HTTP headers to be sent with every request. All

Sets the value of the file input to these file paths or files. If some of the `filePaths` are relative paths, then they
are resolved relative to the current working directory. For empty array, clears the selected files.
For inputs with a `[webkitdirectory]` attribute, only a single directory path is supported.

This method expects [`param: selector`] to point to an
[input element](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/input). However, if the element is inside the `<label>` element that has an associated [control](https://developer.mozilla.org/en-US/docs/Web/API/HTMLLabelElement/control), targets the control instead.
Expand Down
55 changes: 45 additions & 10 deletions packages/playwright-core/src/client/elementHandle.ts
Original file line number Diff line number Diff line change
Expand Up @@ -250,30 +250,65 @@ export function convertSelectOptionValues(values: string | api.ElementHandle | S
return { options: values as SelectOption[] };
}

type SetInputFilesFiles = Pick<channels.ElementHandleSetInputFilesParams, 'payloads' | 'localPaths' | 'streams'>;
type SetInputFilesFiles = Pick<channels.ElementHandleSetInputFilesParams, 'payloads' | 'localPaths' | 'localDirectory' | 'streams' | 'directoryStream'>;

function filePayloadExceedsSizeLimit(payloads: FilePayload[]) {
return payloads.reduce((size, item) => size + (item.buffer ? item.buffer.byteLength : 0), 0) >= fileUploadSizeLimit;
}

async function resolvePathsAndDirectoryForInputFiles(items: string[]): Promise<[string[] | undefined, string | undefined]> {
let localPaths: string[] | undefined;
let localDirectory: string | undefined;
for (const item of items) {
const stat = await fs.promises.stat(item as string);
if (stat.isDirectory()) {
if (localDirectory)
throw new Error('Multiple directories are not supported');
localDirectory = path.resolve(item as string);
} else {
localPaths ??= [];
localPaths.push(path.resolve(item as string));
}
}
if (localPaths?.length && localDirectory)
throw new Error('File paths must be all files or a single directory');
return [localPaths, localDirectory];
}

export async function convertInputFiles(files: string | FilePayload | string[] | FilePayload[], context: BrowserContext): Promise<SetInputFilesFiles> {
const items: (string | FilePayload)[] = Array.isArray(files) ? files.slice() : [files];

if (items.some(item => typeof item === 'string')) {
if (!items.every(item => typeof item === 'string'))
throw new Error('File paths cannot be mixed with buffers');

const [localPaths, localDirectory] = await resolvePathsAndDirectoryForInputFiles(items as string[]);

if (context._connection.isRemote()) {
const streams: channels.WritableStreamChannel[] = await Promise.all((items as string[]).map(async item => {
const lastModifiedMs = (await fs.promises.stat(item)).mtimeMs;
const { writableStream: stream } = await context._wrapApiCall(() => context._channel.createTempFile({ name: path.basename(item), lastModifiedMs }), true);
const writable = WritableStream.from(stream);
await pipelineAsync(fs.createReadStream(item), writable.stream());
return stream;
}));
return { streams };
const files = localDirectory ? (await fs.promises.readdir(localDirectory, { withFileTypes: true, recursive: true })).filter(f => f.isFile()).map(f => path.join(f.path, f.name)) : localPaths!;
const { writableStreams, rootDir } = await context._wrapApiCall(async () => context._channel.createTempFiles({
rootDirName: localDirectory ? path.basename(localDirectory as string) : undefined,
items: await Promise.all(files.map(async file => {
const lastModifiedMs = (await fs.promises.stat(file)).mtimeMs;
return {
name: localDirectory ? path.relative(localDirectory as string, file) : path.basename(file),
lastModifiedMs
};
})),
}), true);
for (let i = 0; i < files.length; i++) {
const writable = WritableStream.from(writableStreams[i]);
await pipelineAsync(fs.createReadStream(files[i]), writable.stream());
}
return {
directoryStream: rootDir,
streams: localDirectory ? undefined : writableStreams,
};
}
return { localPaths: items.map(f => path.resolve(f as string)) as string[] };
return {
localPaths,
localDirectory,
};
}

const payloads = items as FilePayload[];
Expand Down
18 changes: 13 additions & 5 deletions packages/playwright-core/src/protocol/validator.ts
Original file line number Diff line number Diff line change
Expand Up @@ -951,12 +951,16 @@ scheme.BrowserContextHarExportParams = tObject({
scheme.BrowserContextHarExportResult = tObject({
artifact: tChannel(['Artifact']),
});
scheme.BrowserContextCreateTempFileParams = tObject({
name: tString,
lastModifiedMs: tOptional(tNumber),
scheme.BrowserContextCreateTempFilesParams = tObject({
rootDirName: tOptional(tString),
items: tArray(tObject({
name: tString,
lastModifiedMs: tOptional(tNumber),
})),
});
scheme.BrowserContextCreateTempFileResult = tObject({
writableStream: tChannel(['WritableStream']),
scheme.BrowserContextCreateTempFilesResult = tObject({
rootDir: tOptional(tChannel(['WritableStream'])),
writableStreams: tArray(tChannel(['WritableStream'])),
});
scheme.BrowserContextUpdateSubscriptionParams = tObject({
event: tEnum(['console', 'dialog', 'request', 'response', 'requestFinished', 'requestFailed']),
Expand Down Expand Up @@ -1626,6 +1630,8 @@ scheme.FrameSetInputFilesParams = tObject({
mimeType: tOptional(tString),
buffer: tBinary,
}))),
localDirectory: tOptional(tString),
directoryStream: tOptional(tChannel(['WritableStream'])),
localPaths: tOptional(tArray(tString)),
streams: tOptional(tArray(tChannel(['WritableStream']))),
timeout: tOptional(tNumber),
Expand Down Expand Up @@ -1993,6 +1999,8 @@ scheme.ElementHandleSetInputFilesParams = tObject({
mimeType: tOptional(tString),
buffer: tBinary,
}))),
localDirectory: tOptional(tString),
directoryStream: tOptional(tChannel(['WritableStream'])),
localPaths: tOptional(tArray(tString)),
streams: tOptional(tArray(tChannel(['WritableStream']))),
timeout: tOptional(tNumber),
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -178,13 +178,20 @@ export class BrowserContextDispatcher extends Dispatcher<BrowserContext, channel
return false;
}

async createTempFile(params: channels.BrowserContextCreateTempFileParams): Promise<channels.BrowserContextCreateTempFileResult> {
async createTempFiles(params: channels.BrowserContextCreateTempFilesParams): Promise<channels.BrowserContextCreateTempFilesResult> {
const dir = this._context._browser.options.artifactsDir;
const tmpDir = path.join(dir, 'upload-' + createGuid());
await fs.promises.mkdir(tmpDir);
const tempDirWithRootName = params.rootDirName ? path.join(tmpDir, path.basename(params.rootDirName)) : tmpDir;
await fs.promises.mkdir(tempDirWithRootName, { recursive: true });
this._context._tempDirs.push(tmpDir);
const file = fs.createWriteStream(path.join(tmpDir, params.name));
return { writableStream: new WritableStreamDispatcher(this, file, params.lastModifiedMs) };
return {
rootDir: params.rootDirName ? new WritableStreamDispatcher(this, tempDirWithRootName) : undefined,
writableStreams: await Promise.all(params.items.map(async item => {
await fs.promises.mkdir(path.dirname(path.join(tempDirWithRootName, item.name)), { recursive: true });
dgozman marked this conversation as resolved.
Show resolved Hide resolved
const file = fs.createWriteStream(path.join(tempDirWithRootName, item.name));
return new WritableStreamDispatcher(this, file, item.lastModifiedMs);
}))
};
}

async setDefaultNavigationTimeoutNoReply(params: channels.BrowserContextSetDefaultNavigationTimeoutNoReplyParams) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,17 +20,19 @@ import * as fs from 'fs';
import { createGuid } from '../../utils';
import type { BrowserContextDispatcher } from './browserContextDispatcher';

export class WritableStreamDispatcher extends Dispatcher<{ guid: string, stream: fs.WriteStream }, channels.WritableStreamChannel, BrowserContextDispatcher> implements channels.WritableStreamChannel {
export class WritableStreamDispatcher extends Dispatcher<{ guid: string, streamOrDirectory: fs.WriteStream | string }, channels.WritableStreamChannel, BrowserContextDispatcher> implements channels.WritableStreamChannel {
_type_WritableStream = true;
private _lastModifiedMs: number | undefined;

constructor(scope: BrowserContextDispatcher, stream: fs.WriteStream, lastModifiedMs?: number) {
super(scope, { guid: 'writableStream@' + createGuid(), stream }, 'WritableStream', {});
constructor(scope: BrowserContextDispatcher, streamOrDirectory: fs.WriteStream | string, lastModifiedMs?: number) {
super(scope, { guid: 'writableStream@' + createGuid(), streamOrDirectory }, 'WritableStream', {});
this._lastModifiedMs = lastModifiedMs;
}

async write(params: channels.WritableStreamWriteParams): Promise<channels.WritableStreamWriteResult> {
const stream = this._object.stream;
if (typeof this._object.streamOrDirectory === 'string')
throw new Error('Cannot write to a directory');
const stream = this._object.streamOrDirectory;
await new Promise<void>((fulfill, reject) => {
stream.write(params.binary, error => {
if (error)
Expand All @@ -42,13 +44,17 @@ export class WritableStreamDispatcher extends Dispatcher<{ guid: string, stream:
}

async close() {
const stream = this._object.stream;
if (typeof this._object.streamOrDirectory === 'string')
throw new Error('Cannot close a directory');
const stream = this._object.streamOrDirectory;
await new Promise<void>(fulfill => stream.end(fulfill));
if (this._lastModifiedMs)
await fs.promises.utimes(this.path(), new Date(this._lastModifiedMs), new Date(this._lastModifiedMs));
}

path(): string {
return this._object.stream.path as string;
if (typeof this._object.streamOrDirectory === 'string')
return this._object.streamOrDirectory;
return this._object.streamOrDirectory.path as string;
}
}
26 changes: 18 additions & 8 deletions packages/playwright-core/src/server/dom.ts
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ import { prepareFilesForUpload } from './fileUploadUtils';
export type InputFilesItems = {
filePayloads?: types.FilePayload[],
localPaths?: string[]
localDirectory?: string
};

type ActionName = 'click' | 'hover' | 'dblclick' | 'tap' | 'move and up' | 'move and down';
Expand Down Expand Up @@ -625,29 +626,38 @@ export class ElementHandle<T extends Node = Node> extends js.JSHandle<T> {
}

async _setInputFiles(progress: Progress, items: InputFilesItems, options: types.NavigatingActionWaitOptions): Promise<'error:notconnected' | 'done'> {
const { filePayloads, localPaths } = items;
const { filePayloads, localPaths, localDirectory } = items;
const multiple = filePayloads && filePayloads.length > 1 || localPaths && localPaths.length > 1;
const result = await this.evaluateHandleInUtility(([injected, node, multiple]): Element | undefined => {
const result = await this.evaluateHandleInUtility(([injected, node, { multiple, directoryUpload }]): Element | undefined => {
const element = injected.retarget(node, 'follow-label');
if (!element)
return;
if (element.tagName !== 'INPUT')
throw injected.createStacklessError('Node is not an HTMLInputElement');
if (multiple && !(element as HTMLInputElement).multiple)
const inputElement = element as HTMLInputElement;
if (multiple && !inputElement.multiple && !inputElement.webkitdirectory)
throw injected.createStacklessError('Non-multiple file input can only accept single file');
return element;
}, multiple);
if (directoryUpload && !inputElement.webkitdirectory)
throw injected.createStacklessError('File input does not support directories, pass individual files instead');
return inputElement;
}, { multiple, directoryUpload: !!localDirectory });
if (result === 'error:notconnected' || !result.asElement())
return 'error:notconnected';
const retargeted = result.asElement() as ElementHandle<HTMLInputElement>;
await progress.beforeInputAction(this);
await this._page._frameManager.waitForSignalsCreatedBy(progress, options.noWaitAfter, async () => {
progress.throwIfAborted(); // Avoid action that has side-effects.
if (localPaths) {
await Promise.all(localPaths.map(localPath => (
if (localPaths || localDirectory) {
const localPathsOrDirectory = localDirectory ? [localDirectory] : localPaths!;
await Promise.all((localPathsOrDirectory).map(localPath => (
fs.promises.access(localPath, fs.constants.F_OK)
)));
await this._page._delegate.setInputFilePaths(retargeted, localPaths);
// Browsers traverse the given directory asynchronously and we want to ensure all files are uploaded.
const waitForInputEvent = localDirectory ? this.evaluate(node => new Promise<any>(fulfill => {
mxschmitt marked this conversation as resolved.
Show resolved Hide resolved
node.addEventListener('input', fulfill, { once: true });
})).catch(() => {}) : Promise.resolve();
await this._page._delegate.setInputFilePaths(retargeted, localPathsOrDirectory);
await waitForInputEvent;
} else {
await this._page._delegate.setInputFiles(retargeted, filePayloads!);
}
Expand Down
11 changes: 7 additions & 4 deletions packages/playwright-core/src/server/fileUploadUtils.ts
Original file line number Diff line number Diff line change
Expand Up @@ -30,14 +30,17 @@ async function filesExceedUploadLimit(files: string[]) {
}

export async function prepareFilesForUpload(frame: Frame, params: channels.ElementHandleSetInputFilesParams): Promise<InputFilesItems> {
const { payloads, streams } = params;
let { localPaths } = params;
const { payloads, streams, directoryStream } = params;
let { localPaths, localDirectory } = params;

if ([payloads, localPaths, streams].filter(Boolean).length !== 1)
if ([payloads, localPaths, localDirectory, streams, directoryStream].filter(Boolean).length !== 1)
throw new Error('Exactly one of payloads, localPaths and streams must be provided');

if (streams)
localPaths = streams.map(c => (c as WritableStreamDispatcher).path());
if (directoryStream)
localDirectory = (directoryStream as WritableStreamDispatcher).path();

if (localPaths) {
for (const p of localPaths)
assert(path.isAbsolute(p) && path.resolve(p) === p, 'Paths provided to localPaths must be absolute and fully resolved.');
Expand Down Expand Up @@ -73,5 +76,5 @@ export async function prepareFilesForUpload(frame: Frame, params: channels.Eleme
lastModifiedMs: payload.lastModifiedMs
}));

return { localPaths, filePayloads };
return { localPaths, localDirectory, filePayloads };
}
12 changes: 6 additions & 6 deletions packages/playwright-core/src/server/webkit/wkPage.ts
Original file line number Diff line number Diff line change
Expand Up @@ -225,12 +225,12 @@ export class WKPage implements PageDelegate {
}
if (this._page.fileChooserIntercepted())
promises.push(session.send('Page.setInterceptFileChooserDialog', { enabled: true }));
promises.push(session.send('Page.overrideSetting', { setting: 'DeviceOrientationEventEnabled' as any, value: contextOptions.isMobile }));
promises.push(session.send('Page.overrideSetting', { setting: 'FullScreenEnabled' as any, value: !contextOptions.isMobile }));
promises.push(session.send('Page.overrideSetting', { setting: 'NotificationsEnabled' as any, value: !contextOptions.isMobile }));
promises.push(session.send('Page.overrideSetting', { setting: 'PointerLockEnabled' as any, value: !contextOptions.isMobile }));
promises.push(session.send('Page.overrideSetting', { setting: 'InputTypeMonthEnabled' as any, value: contextOptions.isMobile }));
promises.push(session.send('Page.overrideSetting', { setting: 'InputTypeWeekEnabled' as any, value: contextOptions.isMobile }));
promises.push(session.send('Page.overrideSetting', { setting: 'DeviceOrientationEventEnabled', value: contextOptions.isMobile }));
promises.push(session.send('Page.overrideSetting', { setting: 'FullScreenEnabled', value: !contextOptions.isMobile }));
promises.push(session.send('Page.overrideSetting', { setting: 'NotificationsEnabled', value: !contextOptions.isMobile }));
promises.push(session.send('Page.overrideSetting', { setting: 'PointerLockEnabled', value: !contextOptions.isMobile }));
promises.push(session.send('Page.overrideSetting', { setting: 'InputTypeMonthEnabled', value: contextOptions.isMobile }));
promises.push(session.send('Page.overrideSetting', { setting: 'InputTypeWeekEnabled', value: contextOptions.isMobile }));
await Promise.all(promises);
}

Expand Down
9 changes: 6 additions & 3 deletions packages/playwright-core/types/types.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4055,7 +4055,8 @@ export interface Page {
* instead. Read more about [locators](https://playwright.dev/docs/locators).
*
* Sets the value of the file input to these file paths or files. If some of the `filePaths` are relative paths, then
* they are resolved relative to the current working directory. For empty array, clears the selected files.
* they are resolved relative to the current working directory. For empty array, clears the selected files. For inputs
* with a `[webkitdirectory]` attribute, only a single directory path is supported.
*
* This method expects `selector` to point to an
* [input element](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/input). However, if the element is inside
Expand Down Expand Up @@ -10579,7 +10580,8 @@ export interface ElementHandle<T=Node> extends JSHandle<T> {
* instead. Read more about [locators](https://playwright.dev/docs/locators).
*
* Sets the value of the file input to these file paths or files. If some of the `filePaths` are relative paths, then
* they are resolved relative to the current working directory. For empty array, clears the selected files.
* they are resolved relative to the current working directory. For empty array, clears the selected files. For inputs
* with a `[webkitdirectory]` attribute, only a single directory path is supported.
*
* This method expects {@link ElementHandle} to point to an
* [input element](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/input). However, if the element is inside
Expand Down Expand Up @@ -12786,7 +12788,8 @@ export interface Locator {
}): Promise<void>;

/**
* Upload file or multiple files into `<input type=file>`.
* Upload file or multiple files into `<input type=file>`. For inputs with a `[webkitdirectory]` attribute, only a
* single directory path is supported.
*
* **Usage**
*
Expand Down
Loading
Loading