Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: built in env files support #3759

Closed
wants to merge 76 commits into from
Closed
Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
76 commits
Select commit Hold shift + click to select a range
abe80f0
feat: handle env files
Apr 27, 2023
f5bcec5
chore: undo package json changes
Apr 27, 2023
011627e
refactor: undo changes to webpack cli
Apr 28, 2023
bb735ec
feat: add dotenv webpack plugin package
Apr 28, 2023
5a1b4db
refactor: remove cache
Apr 30, 2023
95ca34e
fix: default mode should be production
Apr 30, 2023
08d16ff
refactor: provide env files as default value
Apr 30, 2023
e3a23ce
Merge branch 'master' into feature/env-files
Apr 30, 2023
e2cf250
feat: add webpack cli runner code
May 2, 2023
ca5a782
Merge branch 'master' into feature/env-files
May 2, 2023
fd6d2ba
fix: add env plugin in build config
May 3, 2023
d544597
fix: add webpack as dev dep for dotenv webpack plugin
May 3, 2023
b83ccaf
fix: incorrect path being used in dotenv
May 3, 2023
a4388d7
test: add a simple test
May 3, 2023
ab00786
fix: check if webpack is installed before adding dotenv plugin
May 4, 2023
4e039ba
test: add test for value override
May 4, 2023
3790945
test: add test for the case when variable does not start with webpack_
May 4, 2023
c3d97d7
refactor: simplify test code
May 4, 2023
1aef466
test: add missing assertion
May 4, 2023
bd6c4a6
chore: debug failing smoketests
May 4, 2023
74ad4c8
feat: add cli option
May 6, 2023
afcbf8c
Merge branch 'webpack:master' into feature/env-files
burhanuday May 6, 2023
f203e8b
Merge branch 'master' into feature/env-files
May 6, 2023
9286cb5
refactor: redo applying the plugin
May 6, 2023
8689a2e
test: update snapshots
May 6, 2023
ca6fc8d
test: update tests
May 6, 2023
75ebe62
Merge branch 'feature/env-files' of github.com:burhanuday/webpack-cli…
May 6, 2023
4388c69
test: update snapshots
May 6, 2023
7ab13cf
fix: undo package json changes
May 6, 2023
bf8cb58
feat: add .example env file
May 8, 2023
ff7246f
chore: move dotenv webpack plugin to peerdeps
May 8, 2023
9d7e1db
feat: rename cli flag
May 8, 2023
8a45c8b
test: update snapshots
May 8, 2023
7401474
Merge branch 'master' into feature/env-files
May 8, 2023
f4b8835
test: add more test cases
May 8, 2023
9ee072d
test: add test case for mode local files
May 8, 2023
f7f79b8
fix: update types
May 8, 2023
88d3660
test: add test case for multiple configs
May 8, 2023
0158dca
fix: do validation of passed config
May 9, 2023
f8a6142
test: add more test cases
May 9, 2023
a165f38
fix: use correct logger name
May 10, 2023
06399c5
feat: add types and jsdoc
May 10, 2023
a0534f5
Merge branch 'master' into feature/env-files
May 10, 2023
7f8c4c7
Merge branch 'master' into feature/env-files
May 13, 2023
e00469c
chore: bump
May 15, 2023
d89d73c
feat: add schema utils package
May 16, 2023
0c477b0
feat: use schema utils package to validate input
May 16, 2023
8911ba7
test: update tests with new validation errors
May 16, 2023
00fb0b8
refactor: avoid using object assign
May 16, 2023
cdeb09f
perf: use webpack fs for better perf
May 16, 2023
07dbcd5
feat: add env files to build dependencies
May 17, 2023
99a7163
refactor: extract file exists into a function
May 17, 2023
03d285c
fix: use push instead of unshift for perf
May 17, 2023
ac666c4
Merge branch 'master' into feature/env-files
May 17, 2023
7de5892
feat: make env var prefix configurable
May 17, 2023
07ac04e
test: add tests for env var prefix
May 17, 2023
a3e81fa
test: update tests
May 17, 2023
8e8b78c
Merge branch 'master' into feature/env-files
May 20, 2023
c191cb2
feat: update priority order to make it same as next
May 20, 2023
67d8aed
test: add test case to check priority order of variables
May 20, 2023
a29fc1b
feat: reduce fs call
May 20, 2023
ca7a264
test: add test case to validate files
May 20, 2023
8a98a05
refactor: filter variables together
May 20, 2023
03211fe
feat: check if plugin is installed before adding
May 20, 2023
1cb9309
chore: update snapshots
May 20, 2023
26d136f
docs: add documentation for plugin
May 20, 2023
cdd9bde
docs: update cli docs
May 20, 2023
8acb20c
docs: update schema
May 23, 2023
1388d5e
feat: throw error when an empty value is passed
May 23, 2023
81edc4c
feat: update types
May 23, 2023
55b64a1
test: add tests for allowEmptyValues
May 23, 2023
c4ec39b
docs: add docs for allowemptyvalues option
May 23, 2023
f0e32aa
feat: fail the build on errors
May 27, 2023
6681233
feat: use webpack error to throw
May 27, 2023
8d92b67
Merge branch 'master' into feature/env-files
May 27, 2023
ccf341e
chore: remove author field from package file
Jun 5, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions packages/dotenv-webpack-plugin/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
{
"name": "dotenv-webpack-plugin",
"version": "1.0.0",
"description": "A webpack plugin to support env files",
"main": "index.js",
"repository": "https://github.com/webpack/webpack-cli",
"author": "Burhanuddin Udaipurwala (burhanuday)",
evenstensberg marked this conversation as resolved.
Show resolved Hide resolved
"license": "MIT",
"private": true,
"dependencies": {
"dotenv": "^16.0.3",
"dotenv-expand": "^10.0.0"
},
"peerDependencies": {
"webpack": "5.x.x"
}
}
68 changes: 68 additions & 0 deletions packages/dotenv-webpack-plugin/src/index.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
const fs = require("fs");
const path = require("path");
const dotenv = require("dotenv");
const dotenvExpand = require("dotenv-expand");
const { DefinePlugin } = require("webpack");

class DotenvWebpackPlugin {
constructor(config = {}) {
this.config = Object.assign(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really Object.assign? If we will not modify this.config we don't need to do extra clone, it spends CPU and memory.

Also let's rename to this.options

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since this plugin needs to work standalone as well, we take input from user which we merge with inbuilt config using Object.assign
renamed variable to options

{},
{
envFiles: [],
prefixes: ["process.env.", "import.meta.env."],
},
config,
);
this.cache = new Map();
burhanuday marked this conversation as resolved.
Show resolved Hide resolved
}

apply(compiler) {
const currentDirectory = path.resolve(process.cwd(), "environment");

const mode = compiler.options.mode || "development";
burhanuday marked this conversation as resolved.
Show resolved Hide resolved
// .local file variables will get precedence
const environmentFiles =
this.config.envFiles.length > 0
? this.config.envFiles
: [
`${currentDirectory}/.env`, // loaded in all cases
`${currentDirectory}/.env.local`, // loaded in all cases, ignored by git
`${currentDirectory}/.env.${mode}`, // only loaded in specified mode
`${currentDirectory}/.env.${mode}.local`, // only loaded in specified mode, ignored by git
];
burhanuday marked this conversation as resolved.
Show resolved Hide resolved
burhanuday marked this conversation as resolved.
Show resolved Hide resolved

let envVariables = {};
// parse environment vars from .env files
environmentFiles.forEach((environmentFile) => {
if (fs.existsSync(environmentFile)) {
burhanuday marked this conversation as resolved.
Show resolved Hide resolved
try {
const environmentFileContents = fs.readFileSync(environmentFile);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Each file should be added to buildDependecies, otherwise cache will be broken

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done. all paths added to buildDependencies

const parsedEnvVariables = dotenv.parse(environmentFileContents);
for (const [key, value] of Object.entries(parsedEnvVariables)) {
// only add variables starting with WEBPACK_
if (key.startsWith("WEBPACK_")) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why only WEBPACK_, we need to load all ENV variables

Copy link
Member Author

@burhanuday burhanuday May 9, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if any variable with sensitive information gets added to any env file, it will be shipped with the client bundle if the env var is used. having WEBPACK_ at the start of the variable makes sure that consumers are aware that this variable will be added to the build.

I've seen this pattern implemented in other bundlers and Create React App, and hence, ported it here. Can remove if not required

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It means it should be configurable, some application can be written with env without WEBPACK_ prefix and we can't force them to use WEBPACK_ prefix

for (let index = 0; index < this.config.prefixes.length; index++) {
const prefix = this.config.prefixes[index];
envVariables[`${prefix}${key}`] = value;
}
}
}
} catch (err) {
this.logger.error(`Could not read ${environmentFile}`);
}
}
});
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

from what i understand, that plugin implements a "blueprint" feature. We have not implemented it. Although i dont find it entirely necessary, I am open to suggestions on why this is required

Copy link
Member

@alexander-akait alexander-akait May 15, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because it allows to catch problems on startup


// expand environment vars
envVariables = dotenvExpand({
parsed: envVariables,
// don't write to process.env
ignoreProcessEnv: true,
}).parsed;

new DefinePlugin(envVariables).apply(compiler);
}
}

module.exports = DotenvWebpackPlugin;
2 changes: 0 additions & 2 deletions packages/webpack-cli/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -41,8 +41,6 @@
"colorette": "^2.0.14",
"commander": "^10.0.1",
"cross-spawn": "^7.0.3",
"dotenv": "^16.0.3",
"dotenv-expand": "^10.0.0",
"envinfo": "^7.7.3",
"fastest-levenshtein": "^1.0.12",
"import-local": "^3.0.2",
Expand Down
61 changes: 0 additions & 61 deletions packages/webpack-cli/src/webpack-cli.ts
Original file line number Diff line number Diff line change
Expand Up @@ -56,8 +56,6 @@ const path = require("path");
const { pathToFileURL } = require("url");
const util = require("util");
const { program, Option } = require("commander");
const dotenv = require("dotenv");
const dotenvExpand = require("dotenv-expand");

const WEBPACK_PACKAGE = process.env.WEBPACK_PACKAGE || "webpack";
const WEBPACK_DEV_SERVER_PACKAGE = process.env.WEBPACK_DEV_SERVER_PACKAGE || "webpack-dev-server";
Expand Down Expand Up @@ -1954,65 +1952,6 @@ class WebpackCLI implements IWebpackCLI {
}, {});
config.path.set(config.options, mergedConfigPaths as unknown as string);
}

const runFunctionOnEachConfiguration = (
options: Configuration | Configuration[],
fn: CallableFunction,
) => {
if (Array.isArray(options)) {
for (let item of options) {
item = fn(item);
}
} else {
options = fn(options);
}

return options;
};

// plugin env variables using dot env package
const _envDirectory = path.resolve(process.cwd(), "environment");

config.options = runFunctionOnEachConfiguration(config.options, (options: Configuration) => {
const mode = options.mode || "development";
// .local file variables will get precedence
const environmentFiles = [
`${_envDirectory}/.env`, // loaded in all cases
`${_envDirectory}/.env.local`, // loaded in all cases, ignored by git
`${_envDirectory}/.env.${mode}`, // only loaded in specified mode
`${_envDirectory}/.env.${mode}.local`, // only loaded in specified mode, ignored by git
];

let envVariables: Record<string, string> = {};
// parse environment vars from .env files
environmentFiles.forEach((environmentFile) => {
if (fs.existsSync(environmentFile)) {
try {
const environmentFileContents: string = fs.readFileSync(environmentFile);
const parsedEnvVariables: Record<string, string> =
dotenv.parse(environmentFileContents);
for (const [key, value] of Object.entries(parsedEnvVariables)) {
// only add variables starting with WEBPACK_
if (key.startsWith("WEBPACK_")) envVariables[`process.env.${key}`] = value;
}
} catch (err) {
this.logger.error(`Could not read ${environmentFile}`);
}
}
});

// expand environment vars
envVariables = dotenvExpand({
parsed: envVariables,
// don't write to process.env
ignoreProcessEnv: true,
}).parsed;

options.plugins = (options.plugins || []).concat([new webpack.DefinePlugin(envVariables)]);

return options;
});

return config;
}

Expand Down