[core] Flatten imports to speed up webpack build & node resolution

[anthonyalayo]

What's the problem? 🤔

Background

While playing around with Next.js, I installed a package that was using @mui. I immediately noticed huge delays running next dev and the large module count (10K+) listed in the terminal. Searching for an answer, I landed on a long list of requests for help:

• Super slow page load times in development environment vercel/next.js#17977
• SWC is slow when importing directly from @mui/material vercel/next.js#37614
• SWC doesn't tree shake library vercel/next.js#42343
• NextJs compiling extremely slow vercel/next.js#29559
• Slow perfomance & Compile zounds of modules vercel/next.js#43285
• https://www.reddit.com/r/nextjs/comments/swciuj/next_dev_is_slow/

But none of those had an actual answer with an actual solution, just guesses and workarounds. One of the popular answers was: https://mui.com/material-ui/guides/minimizing-bundle-size/

After attempting to fix the library I pulled in by following the first option on that guide:
https://mui.com/material-ui/guides/minimizing-bundle-size/#option-one-use-path-imports

I noticed that the module count was still in the thousands. Why?

After reading everything on webpack, modules, tree shaking, etc, I made a demo application using CRA and a single import to @mui to showcase the problem.

Problem

Reproduction Steps

1. Run npx create-react-app cra-test to get the latest with Webpack 5
2. Ejected from CRA so that I could modify the webpack config for more metrics
3. Using webpack dev server, I modified this to see what modules traversed in development
4. Using webpack prod builds, I modified this to see modules traversed in production

I used the following configurations for stats.toJson() to balance the verbosity of the output.
I tested with and without default imports for a single icon and component.

1. With default import {assets: false, chunks: false, modulesSpace: 6, nestedModules: false, reasonsSpace: 10}
2. With path import: {assets: false, chunks: false})

Demo App 1 - Default Import Icon

This is the scenario that we are told to avoid when using @mui, and for good reason.
I created this as a baseline to see what webpack had to traverse.

import { AbcRounded } from '@mui/icons-material'

function App() {
  return (
      <div>
        <AbcRounded />
      </div>
  );
}
export default App;

Demo App 1 - Webpack Metrics Output

As expected, using the default import for AbcRounded ended up pulling in all icons.

Webpack Module Summary

orphan modules 6.28 MiB [orphan] 10860 modules
runtime modules 28.5 KiB 14 modules

Demo App 2 - Path Import Icon

Following the docs, you would expect an import like this to be lightweight. It isn't.

import AbcRounded from '@mui/icons-material/AbcRounded'

function App() {
  return (
      <div>
        <AbcRounded />
      </div>
  );
}
export default App;

Demo App 2 - Webpack Metrics Output

This is the problem. Importing a single icon resulted in:

1. Importing ./utils/createSvgIcon
2. Importing @mui/material/utils
3. Importing a lot more...(doesn't fit in a single screenshot)

Webpack Module Summary

orphan modules 534 KiB [orphan] 274 modules
runtime modules 28.5 KiB 14 modules

Demo App 3 - Path Import Component

This problem isn't unique to @mui/icons-material either. Here's performing a path import of a button.

import Button from '@mui/material/Button'

function App() {
  return (
      <div>
          <Button>Hello World</Button>
      </div>
  );
}
export default App;

Demo App 3 - Webpack Metrics Output

Again, the problem. Importing a single button resulted in:

1. Importing ./buttonClasses
2. Importing @mui/utils
3. Importing a lot more...(doesn't fit in a single screenshot)

Webpack Module Summary

orphan modules 573 KiB [orphan] 291 modules
runtime modules 28.5 KiB 14 modules

We should not be importing hundreds of modules from a single icon or button.

But... Tree Shaking? Side Effects?

This was confusing for me too, and I had to go into the details to find the answer.

1. Webpack does not tree shake at the code level, it only tree shakes at the module level.
2. Terser performs dead code elimination when minifying, but webpack still has to traverse all those imports!
3. This happens whether you are building for development or production.
4. We "cue up" our code for deletion by using ESM, but it isn't done until the minification happens.

Yes the bundle will still be minimized successfully when following ESM practices, but thousands of modules being traversed bloats memory and slows down development servers.

What are the requirements? ❓

Importing from @mui should always result in a minimal dependency graph for that particular import.

What are our options? 💡

Option 1 - Proposal

Apply transformations to the @mui build process to ensure a minimal dependency graph for all files.

Option 2 - Alternative

Remove all barrel files from @mui. This option isn't great as the developer experience that they provide is desired by both library maintainers and library users alike.

Proposed solution 🟢

1. Apply import transformations within @mui packages.

Showcased above, even when importing a component or icon directly, thousands of downstream modules get pulled in. This happens because within @mui itself, barrel files are being utilized for their developer experience. In that case, why not follow the same recommendation that @mui gives, and add these transforms to the build process?

2. Make "modularize imports" work for all @mui packages

In [docs] Modularize Imports for Nextjs, the comment #35457 (comment) requested that the docs don't include @mui/material for import transformations via babel or swc, since there are outliers that cause the transformation to fail.

Instead of backing off here, the work should be put in to fix it. The same barrel files that @mui is using internally for better developer experience is what users of the library need as well. By fixing point 1, this will come for free.

Resources and benchmarks 🔗

Below are the webpack metrics I collected for the applications in the background statement:
mui-default-import-icon-truncated-metrics.txt
mui-path-import-button-metrics.txt
mui-path-import-icon-metrics.txt

[anthonyalayo]

@oliviertassinari this issue isn't specific to icons, please see "Demo App 3":

This problem isn't unique to @mui/icons-material either.

[flaviendelangle]

We could enforce a few rules to improve the amount of dependencies listed.
On of them being to never import from the root of @mui/utils and @mui/system.

[anthonyalayo]

@flaviendelangle that would be great!

[flaviendelangle]

I think Olivier added the icon label because it is the scenario with the most obvious gains.
An icon is a super small component and we are looking through a full package (@mui/system at least).
For regular component, I don't think you should expect the list of files visited to be small, but we can definitely make improvements.

By the way, on the X components (data grid and pickers), we are probably even worse than that.
But the components being big by themselves, it's less notceable.
We can still improve though.

[michaldudak]

Thanks for such a detailed report, @anthonyalayo! I agree with @flaviendelangle, we can start with creating an eslint rule to disallow imports from barrel files. It could be better than introducing a build transform, as anyone reading the source code will see the proper way of importing other modules.

@anthonyalayo, would you be interested in working on this?

[anthonyalayo]

@michaldudak thanks for going through it 😄 sure I'm interested, but I wanted to hit on that point you just mentioned:

we can start with creating an eslint rule to disallow imports from barrel files. It could be better than introducing a build transform, as anyone reading the source code will see the proper way of importing other modules.

I considered this (as it's the easier way to go code change wise), but I also noted in the proposed solution that the developer experience would be affected. A tangential discussion happened on a Next.js issue, and the audience was quite in favor of barrel files: vercel/next.js#12557 (comment)

On top of that, Next.js recently released modularizeImports for SWC, since babel-plugin-transform-imports was quite nice for developer experience.

While bundlers understand these barrel files and can remove unused re-exports (called "dead code elimination"), this process involves parsing/compiling all re-exported files. In case of published libraries some npm packages ship barrel files that have thousands of modules re-exported, which slows down compile times. These libraries recommended babel-plugin-transform-imports to avoid this issue, but for those using SWC, there was no previous support. We've added a new SWC transform built into Next.js called modularizeImports.

Leveraging this transform with @mui/icons-material or lodash allows skipping compilation of unused files.

In the absence of these features, I think the only solution would be an eslint rule to reject barrel file usage. But since we already are somewhat across the goal post for good DX and performance, why not take it all the way?

[michaldudak]

By "taking it all the way" you mean fixing the issues that prevent babel and modularizeImports transforms to work, right? I haven't looked into this much, but I fear we won't be able to change the structure of our packages without introducing breaking changes. We can certainly look into it for the next release, as we're going to change how things are imported anyway (by introducing ESM and import maps)

[anthonyalayo]

@michaldudak agreed, it would be a breaking change (since some import locations would move to be consistent), so I think looking into it for the next release sounds reasonable to me.

With that being said, I can definitely do the eslint rule and import fixes associated with it. I'll make a PR for it an attach it to this issue.

[anthonyalayo]

@michaldudak I did an initial attempt, but there's quite a bit of eslint configs/overrides setup already with 'no-restricted-imports': https://github.com/mui/material-ui/blob/master/.eslintrc.js

I also noticed that paths doesn't support wildcards, so it would have to look something like this (unless someone chimes in with a better solution):

    'no-restricted-imports': [
      'error',
      {
        "paths": ['@mui/material', '@mui/material/utils', '@mui/styles', '@mui/base', '@mui/utils'],
      }
    ],

Is there anyone at @mui that could join in the conversation for how they would like it ideally?

[anthonyalayo]

@michaldudak bumping the above message in case it got missed

[michaldudak]

Unfortunately, "patterns": ["@mui/*"] doesn't seem to do the trick here. Explicitly listing all the forbidden paths seems reasonable.

cc @mui/code-infra for visibility and perhaps other opinions

[anthonyalayo]

Sounds good, If no other opinions from @mui/code-infra i'll do that then

[oliviertassinari]

This problem isn't unique to @mui/icons-material either. Here's performing a path import of a button.

@anthonyalayo I think that the number of modules isn't this relevant. We should focus more on the metrics that directly impact developers:

1. built time in dev mode
2. load time in dev mode

I have added the icon's label because so far, I don't think that it was ever proven that the problem goes beyond icons. https://mui.com/material-ui/guides/minimizing-bundle-size/#development-environment mentions a build time of x6 for icons, but for a button, back then, it was like 50%, mostly negligible.

It could be great to measure again, it was a long time ago.

On of them being to never import from the root of @mui/utils and @mui/system.

👍 agree, to keep doing it (I think that we started doing this a long time ago).

[joshkel]

I've been looking into what I think is the same root issue: I'm trying to speed up our Jest test suite. Jest by default executes each of its suites in an isolated Node context, which means all of the tests' dependencies have to be re-loaded, parsed, and executed for every test module. Tree-shaking doesn't apply, and Babel typically isn't run on node_modules for Jest (so babel-plugin-import or babel-plugin-direct-import won't help), and the costs are incurred on every test execution (in watch mode, every time a file is saved, versus just when webpack-dev-server is starting up). MUI packages' use of barrel imports from other MUI packages seems to have a noticeable impact here, so I'm interested in this area of work as well.

Should @mui/base be added to the list of "don't do a root import" packages, especially as more of @mui/material and @mui/joy start using it? (In some crude local testing, replacing barrel imports of @mui/base speeds up a single test module by 2.5% - not much, but measurable, for an process that's run all the time during development.)

If this isn't the same issue or would be better tracked separately, please let me know.

[kevcmk]

As of right now, I would urge anyone considering using NextJS 13 to avoid Material UI v5

[oliviertassinari]

As of right now, I would urge anyone considering using NextJS 13 to avoid Material UI v5

@kevcmk Which issue are you facing?

[kevcmk]

Material UI documentation will send you down the complete wrong path if you're using next.js.

@oliviertassinari After looking through your posts on other code reviews, I tracked down this code (specific to Next.js), which finally did the trick. Thank you for this ↓

const nextConfig = {
...
modularizeImports: {
    "@mui/material": {
      transform: "@mui/material/{{member}}",
    },
    "@mui/icons-material": {
      transform: "@mui/icons-material/{{member}}",
    },
  },
}

module.exports = nextConfig

and, if you're like me, you ran into issues with, for example

import { useTheme } from "@mui/material"; # Will fail given the above next.config modification.

Use this instead

import { useTheme } from "@mui/material/styles";

Also, thank you for your work on this project and your effort on the issues board.

[oliviertassinari]

This issue is up for grabs for the community. It should be relatively easy to make progress. It's a matter of have deeper imports, avoiding barrel index files.

[Gu7z]

We could enforce a few rules to improve the amount of dependencies listed.
On of them being to never import from the root of @mui/utils and @mui/system.

Has the change to avoid importing directly from the root of @mui/utils and @mui/system been implemented yet?

I'm experiencing issues with exports as mentioned in the issue description. By merely importing an icon with `import Abc from '@mui/icons-material/Abc', the build time in a simple React + Webpack app increased from 3 seconds to 17 because Material adds roughly 500 internal packages.

"@emotion/react": "11.10.6",
"@emotion/styled": "11.10.6",
"@mui/icons-material": "5.11.16",
"@mui/material": "5.11.16",
"@mui/styles": "5.11.16",

Stuck at "react": "17.0.2" 😅

[Janpot]

Is there anyone at https://github.com/mui that could join in the conversation for how they would like it ideally?

@anthonyalayo or if anyone is still interested in tackling this. I believe a good starting point would be to update .eslintrc.js with

diff --git a/.eslintrc.js b/.eslintrc.js
index 038be57d26..c178890cb3 100644
--- a/.eslintrc.js
+++ b/.eslintrc.js
@@ -457,7 +457,17 @@ module.exports = {
         'no-restricted-imports': [
           'error',
           {
-            paths: NO_RESTRICTED_IMPORTS_PATHS_TOP_LEVEL_PACKAGES,
+            paths: [
+              ...NO_RESTRICTED_IMPORTS_PATHS_TOP_LEVEL_PACKAGES,
+              {
+                name: '@mui/system',
+                message: OneLevelImportMessage,
+              },
+              {
+                name: '@mui/utils',
+                message: OneLevelImportMessage,
+              },
+            ],
           },
         ],
         // TODO: Consider setting back to `ignoreExternal: true` when the expected behavior is fixed:

And then address the ~250 warnings this generates. Most of them will be about rewriting things from e.g.

import { getDisplayName } from '@mui/utils';

to

import getDisplayName from '@mui/utils/getDisplayName';

Some of them may be a bit more challenging such as

import { Interpolation } from '@mui/system';

There we may first have to push those re-exports deeper in @mui/system so that we can import them as

import { Interpolation } from '@mui/system/styled';

[jaydenseric]

I'm working on this, but firstly, it's good to establish the ideal end state. The imports in the MUI codebase are a hot mess of anti-patterns…

1. Type imports/exports should use the TypeScript type syntax.

   • https://typescript-eslint.io/blog/consistent-type-imports-and-exports-why-and-how/

   • Should be enforced by:

     • https://typescript-eslint.io/rules/consistent-type-imports
     • https://typescript-eslint.io/rules/consistent-type-exports

   • Example of incorrect:

     material-ui/packages/mui-utils/src/appendOwnerState/appendOwnerState.ts

     Line 2 in e0d43b4

     import { Simplify } from '@mui/types';

   • While rewriting type imports to deep imports, the type syntax should be manually added in the new import statement. But later, there should be another PR that implements the ESLint rule enforcing the type syntax and a project wide auto fix.

2. Internal imports within a package should be relative (not using the packages' own name).

   • Example of incorrect:

     material-ui/packages/mui-utils/src/capitalize/capitalize.test.js

     Line 2 in e0d43b4

     import capitalize from '@mui/utils/capitalize';

3. Cross-package imports should use the package name (no relative imports across packages).
4. Cross-package imports should be direct from the source, and not via a re-export.

   • Examples of incorrect:

     material-ui/packages/mui-material/src/utils/index.js

     Line 7 in 1a018b7

     export { default as debounce } from './debounce';

     material-ui/packages/mui-material/src/Slide/Slide.js

     Line 9 in 1a018b7

     import debounce from '../utils/debounce';

5. Import paths should be fully specified (including the file extension, .js).

   • This is for conformance to real ESM in browsers, Deno, Node.js, TypeScript in module mode nodenext, and bundlers like webpack and ESBuild.

   • [core] True ESM support #43938

   • Should be enforced by TypeScript in module mode nodenext.

   • Example of incorrect:

     material-ui/packages/mui-utils/src/ownerWindow/ownerWindow.ts

     Line 1 in e0d43b4

     import ownerDocument from '../ownerDocument';

6. Imports should be sorted.

   • The consistency accross modules makes comparison easier, and reduces conflicts.

   • Can be enforced by one of these ESLint plugins:

     • Used this for years, works great: https://www.npmjs.com/package/eslint-plugin-simple-import-sort
     • Not tried this yet: https://perfectionist.dev/rules/sort-imports

   • Example of incorrect:

     material-ui/packages/mui-material/src/Accordion/Accordion.js

     Lines 2 to 16 in e0d43b4

     	import * as React from 'react';
     	import { isFragment } from 'react-is';
     	import PropTypes from 'prop-types';
     	import clsx from 'clsx';
     	import chainPropTypes from '@mui/utils/chainPropTypes';
     	import composeClasses from '@mui/utils/composeClasses';
     	import { styled } from '../zero-styled';
     	import memoTheme from '../utils/memoTheme';
     	import { useDefaultProps } from '../DefaultPropsProvider';
     	import Collapse from '../Collapse';
     	import Paper from '../Paper';
     	import AccordionContext from './AccordionContext';
     	import useControlled from '../utils/useControlled';
     	import useSlot from '../utils/useSlot';
     	import accordionClasses, { getAccordionUtilityClass } from './accordionClasses';

   • While rewriting imports to deep imports, the imports should not be re-sorted to minimise diff for review. But later, there should be another PR that implements the ESLint rule enforcing sorted imports and a project wide auto fix.

7. Imports should be deep; never via a barrel module.

   • https://jaydenseric.com/blog/optimal-javascript-module-design

   • This can be enforced by:

     • Using the ESLint rule no-restricted-imports to disallow non-deep imports accross packages and to disallow import paths ending with index.js (assuming the best practice of fully specified import paths is already enforced).

   • Ideally in the future MUI would delete all barrel modules, and enforce optimal module design using eslint-plugin-optimal-modules. There is no good reason to offer users multiple suboptimal ways to consume the same API.

   • Examples of incorrect:

     material-ui/packages/mui-material/src/utils/index.js

     Line 2 in 1a018b7

     import { unstable_ClassNameGenerator as ClassNameGenerator } from '@mui/utils';

     Here two different deep and non-deep ways to import from the same module are used at the same time!

     material-ui/packages/mui-system/src/colorManipulator/colorManipulator.test.js

     Lines 2 to 17 in 1a018b7

     	import { blend } from '@mui/system';
     	import {
     	recomposeColor,
     	hexToRgb,
     	rgbToHex,
     	hslToRgb,
     	darken,
     	decomposeColor,
     	emphasize,
     	alpha,
     	getContrastRatio,
     	getLuminance,
     	lighten,
     	colorChannel,
     	} from '@mui/system/colorManipulator';

8. There should not be multiple import statements when importing from the same module.

   • Example of incorrect:

     material-ui/packages/mui-utils/src/index.ts

     Lines 2 to 3 in 1a018b7

     	export { default as deepmerge } from './deepmerge';
     	export { isPlainObject } from './deepmerge';

Some of the above anti-patterns I have listed how tooling can be used to enforce good patterns, but others I still have to figure out the right ESLint/TS config.

[flaviendelangle]

Internal imports within a package should be relative (not using the packages' own name).

This is a test file, it's not inside the package

I agree with some of the proposals (sorting the imports is something I would love to for quite some time but never took the effort to do so.

For others I have by doubts tbh

[jaydenseric]

This is a test file, it's not inside the package

It's a module within a package boundary; it just happens to not be published.

I agree with some of the proposals (sorting the imports is something I would love to for quite some time but never took the effort to do so.

For others I have by doubts tbh

Sorting imports has the least functional improvement of all the recommendations. What do you doubt about the other points? Do you just doubt some, but not all points?

[flaviendelangle]

It's a module within a package boundary; it just happens to not be published.

Then I'm not following what is the point of your whole post.
If it's to improve the life our the developers using the MUI packages, then how do it matter how the test files are importing what they need?

Sorting imports has the least functional improvement of all the recommendations. What do you doubt about the other points? Do you just doubt some, but not all points?

I have doubts about the viability of something like There should not be multiple import statements when importing from the same module. in a codebase with as many exports as the MUI packages.
For @mui/utils I'm sure it's viable (and most exports already only have one element). But for the main packages I think it would make the end DX worse. When we have two elements that are always used together, exporting them from the same file makes that clear. We always export the component and the method to build classes for this component together for example and I think we should continue to do so.

[oliviertassinari]

@jaydenseric Thanks for looking into it!

On the labeling for "hot mess of anti-patterns", for each point:

1. Mixed feelings: "some projects use transpilers such as Babel, SWC, or Vite that don't have access to type information" https://typescript-eslint.io/blog/consistent-type-imports-and-exports-why-and-how/. I can see an example of those, https://unpkg.com/browse/@mui/[email protected]/Divider/index.js with DividerProps. What we do here is wrong, we should import type. However, doing this systematically sounds like a distraction. Ignoring this and using a TypeScript aware transpiler sounds better.
2. Disagree, this is by design. It guarantees that those test can't test private APIs.
3. Example?
4. Agree, we started to reorganize the code but we never finished the work.
5. Agree for what is publish on npm, e.g. [core] Run @mui/icon-material src:icons #44097.
   For the source, not clear it's better https://devblogs.microsoft.com/typescript/announcing-typescript-5-7-beta/#path-rewriting-for-relative-paths, sounds like a distraction.
6. Disagree, distraction
7. ✅ Agree, what this issue is about 👍 (though it's not about removing barrel index, but not using them everywhere possible)
8. Agree, it looks like it won't make a difference https://unpkg.com/browse/@mui/[email protected]/index.js but it's signals lack of care.

[flaviendelangle]

6. Disagree, distraction

I tend to disagree on that one, I find that codebase with organized imports are more readable and maintainable, and this is easily handled by ESLint or even Prettier.

But for me it's totally out of the scope of this issue though

[oliviertassinari]

6. Disagree, distraction

@flaviendelangle we used to sort all the imports on the codebase. I think that we should continue to do it.

We would sort them in 3 buckets:

• dependencies, e.g. react
• local dependencies, e.g. @mui/utils
• relative imports

However, under "hot mess of anti-patterns" I disagree, I think it would be a distraction to work on this first. I also fail to see how its not already sorted in the example provided. Meaning, why would it be better with a different sorting?

But for me it's totally out of the scope of this issue though

👍