Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dart Analyzer very slow/stuck #55281

Open
JonasJW opened this issue Mar 22, 2024 · 109 comments
Open

Dart Analyzer very slow/stuck #55281

JonasJW opened this issue Mar 22, 2024 · 109 comments
Labels
analyzer-stability area-analyzer Use area-analyzer for Dart analyzer issues, including the analysis server and code completion. P2 A bug or feature request we're likely to work on type-performance Issue relates to performance or code size

Comments

@JonasJW
Copy link

JonasJW commented Mar 22, 2024

Dart & Flutter has suddenly become unusable slow in VSCode. Intellisense won't load within 30+ seconds, syntax highlighting won't update, jumping to code definitions loads infinitely, etc.

This seems to be especially the case on bigger projects, on a newly created project it works fine. I tried switching to Android Studio which seems to behave similarly (maybe a little better).

I tried uninstalling all extensions, even completely removing VSCode with any data stored and reinstalling it. As soon as I install the Dart extension, it is unusable. I also tried setting the following settings:

"dart.previewLsp": true,
"dart.useLegacyAnalyzerProtocol": true,
"dart.useLsp": false,
"dart.autoImportCompletions": false,

I'm on macOS 14.4, VSCode Version 1.87.2 (Universal),

In the Activity Monitor, there is the dart:analysis_server.dart.snapshot process running with around 1 GB of used memory. I'm not sure if this is normal.

Here is a screenshot from the Dart Analyzer (which I'm also having trouble opening)

Screenshot 2024-03-22 at 20 47 42

dart info output:

General info

  • Dart 3.3.2 (stable) (Tue Mar 19 20:44:48 2024 +0000) on "macos_arm64"
  • on macos / Version 14.4 (Build 23E214)

Project info

  • sdk constraint: '>=2.17.0 <3.0.0'

Process info

Memory CPU Elapsed time Command line
6 MB 0.0% 11:53 dart devtools --machine --allow-embedding
393 MB 100.6% 11:53 dart language-server --protocol=lsp --client-id=VS-Code --client-version=3.85.20240318
29 MB 0.0% 11:53 flutter_tools.snapshot daemon
35 MB 0.1% 00:23 flutter_tools.snapshot debug_adapter
121 MB 10.8% 00:23 flutter_tools.snapshot run --machine --start-paused -d B95DD896-131D-4163-A83C-1AC8A70E8560 --devtools-server-address http:/ --target /main.dart --web-renderer canvaskit --web-port 5000

Any recommendations on how I could possible fix this would be very welcome!

@JonasJW JonasJW changed the title Dart Dart Analyzer very slow/stuck Mar 22, 2024
@lrhn lrhn added the area-analyzer Use area-analyzer for Dart analyzer issues, including the analysis server and code completion. label Mar 24, 2024
@keertip keertip added type-performance Issue relates to performance or code size P1 A high priority bug; for example, a single project is unusable or has many test failures labels Mar 26, 2024
@keertip
Copy link
Contributor

keertip commented Mar 26, 2024

@DanTup , do you see this? Have there been more reports of this kind?

@keertip
Copy link
Contributor

keertip commented Mar 26, 2024

/ cc @bwilkerson

@DanTup
Copy link
Collaborator

DanTup commented Mar 26, 2024

I've not seen any other reports like this. 100% CPU certainly does not look healthy if it's persisting for a long period after initial analysis.

@JonasJW a few questions:

  • How big is the open workspace (code --status I think should give a summary of the number of each file type)
  • Might it contain any symlinks (that could result in analyzing more than just this folder, or even cycles?)
  • You noted that this issue started recently - are there any things that you updated around the time it started (VS Code, the VS Code extensions, the Dart/Flutter SDKs)?
  • Are you able to get a screenshot of the servers Timing page (the one you gave above was the Completion page) which will include timings for a larger set of requests

@MaximeRougieux
Copy link

We were having the same issue and using the custom_lint package (v0.5.7).

Removing it completely seems to have solved the issue for us, are you using it too, and can you try to remove it to confirm it comes from there if you are ?

@keertip
Copy link
Contributor

keertip commented Mar 27, 2024

/cc @scheglov

@keertip keertip added P2 A bug or feature request we're likely to work on and removed P1 A high priority bug; for example, a single project is unusable or has many test failures labels Mar 27, 2024
@JonasJW
Copy link
Author

JonasJW commented Mar 29, 2024

@DanTup Thanks for the response and sorry for the late reply.

code --status output:

Version:          Code 1.87.2 (863d2581ecda6849923a2118d93a088b0745d9d6, 2024-03-08T15:21:31.043Z)
OS Version:       Darwin arm64 23.4.0
CPUs:             Apple M1 Pro (8 x 24)
Memory (System):  16.00GB (0.06GB free)
Load (avg):       5, 5, 6
VM:               0%
Screen Reader:    no
Process Argv:     --crash-reporter-id 0c0cb32f-913b-418e-b987-7370929419d6
GPU Status:       2d_canvas:                              enabled
                  canvas_oop_rasterization:               enabled_on
                  direct_rendering_display_compositor:    disabled_off_ok
                  gpu_compositing:                        enabled
                  multiple_raster_threads:                enabled_on
                  opengl:                                 enabled_on
                  rasterization:                          enabled
                  raw_draw:                               disabled_off_ok
                  skia_graphite:                          disabled_off
                  video_decode:                           enabled
                  video_encode:                           enabled
                  webgl:                                  enabled
                  webgl2:                                 enabled
                  webgpu:                                 enabled

CPU %   Mem MB     PID  Process
    0      115   38907  code main
    7       66   38910     gpu-process
    0       16   38911     utility-network-service
    0       33   38923  shared-process
    0       33   82195  ptyHost
    0        0   83841       /bin/zsh -il
    0        0   90120         bash /usr/local/bin/code --status
    9       66   90129           electron-nodejs (cli.js )
    5      246   83837  window [5] (auto_router.gr.dart — exercisable_flutter)
    0       98   83839  extensionHost [5]
    0       33   83879       electron-nodejs (config.js )
    0        0   83961         /usr/bin/script -t 0 /dev/null /usr/bin/arch -arm64e xcrun xcdevice observe --usb
    0       16   83962           /Applications/Xcode.app/Contents/Developer/usr/bin/xcdevice observe --usb
    0        0   83963         /usr/bin/script -t 0 /dev/null /usr/bin/arch -arm64e xcrun xcdevice observe --wifi
    0       16   83964           /Applications/Xcode.app/Contents/Developer/usr/bin/xcdevice observe --wifi
    0      426   83884       /Users/jonas/Documents/flutter/bin/cache/dart-sdk/bin/dart language-server --protocol=lsp --client-id=VS-Code --client-version=3.85.20240327
    0        0   83886       /Users/jonas/Documents/flutter/bin/cache/dart-sdk/bin/dart devtools --machine --allow-embedding
    0       33   89935       electron-nodejs (config.js )
    0       33   89961         electron-nodejs (config.js )
  170     1081   90000           electron-nodejs (config.js )
    0       33   83840  fileWatcher [5]

Workspace Stats: 
|  Window (auto_router.gr.dart — exercisable_flutter)
|    Folder (exercisable_flutter): more than 20346 files
|      File types: transitive_digest(3856) json(1582) png(1354) flat(1004)
|                  jar(976) xml(843) webp(646) ttf(344) stamp(334)
|                  len(275)
|      Conf files: launch.json(1)
|      Launch Configs: dart(13)

To be honest, I'm not quite sure what you mean by your question about symlinks or how I can find out.

Yes, this issue seems to have started just recently without making any noticeable changes such as updating SDKs, installing extensions, etc.

Here are further screenshots, I hope this is the server timing page you meant.

Screenshot 2024-03-29 at 09 58 32

Screenshot 2024-03-29 at 09 59 31

If you have any recommendations what I can try to fix this issue, I'm happy to try that. As mentioned, I have already tried uninstalling Flutter/Dart, VSCode, etc. at the moment I don't know what else I could try.

@lulupointu
Copy link

For us updating custom_lint to 0.6.4 fixed the issue 🎉

@DanTup
Copy link
Collaborator

DanTup commented Mar 29, 2024

@JonasJW thanks - the page I meant was the one marked "Timing" on the left.

Can you also confirm whether you're using any analyzer plugins? A few comments above suggest custom_lint might have have issues (that may be fixed in 0.6.4).

You should be able to check for symlinks by running find . -type l -ls in your project folder.

@JonasJW
Copy link
Author

JonasJW commented Mar 29, 2024

@DanTup thanks for the clarification!

I don't use custom_linter or any analyzer plugins.

When I run find . -type l -ls I get a long list of I suppose symlinks that probably just belong the the packages I use, for example:

111156169        0 lrwxr-xr-x    1 jonas            staff                  29 Mar  8 17:37 ./build/macos/Build/Products/Debug-platform/exercisable_flutter.app/Contents/Frameworks/FirebaseCore.framework/FirebaseCore -> Versions/Current/FirebaseCore

Is that anything out of the ordinary?

Here the screenshots of the "Timing" page:

Screenshot 2024-03-29 at 19 47 54

Screenshot 2024-03-29 at 19 48 15

@DanTup
Copy link
Collaborator

DanTup commented Apr 4, 2024

When I run find . -type l -ls I get a long list of I suppose symlinks that probably just belong the the packages I use, for example:

111156169        0 lrwxr-xr-x    1 jonas            staff                  29 Mar  8 17:37 ./build/macos/Build/Products/Debug-platform/exercisable_flutter.app/Contents/Frameworks/FirebaseCore.framework/FirebaseCore -> Versions/Current/FirebaseCore

Is that anything out of the ordinary?

That one doesn't look like a problem, but are there any that point back up the tree (eg. creating cycles or including other large parts of the disk in the the path)?

It may be useful to enable the analyzer instrumentation log (this file can get very large - be sure to turn it off afterwards) and reproduce the issue, and see if there is anything in the log file that looks out of place while this happens (for example exceptions, or paths you would not expect to be analyzed with this project open being analyzed).

You mentioned this only happens on large projects - are any of them public projects that I could test with (or that sharing logs from would not include anything sensitive)?

@JonasJW
Copy link
Author

JonasJW commented May 8, 2024

Hi @DanTup,

I apologize for taking so long to respond. I still have this issue and it's really slowing down my development process.

are there any that point back up the tree (eg. creating cycles or including other large parts of the disk in the the path)?

Here are a few symlinks, some point up but it doesn't appear to me that they would create a cycle

./windows/flutter/ephemeral/.plugin_symlinks/firebase_auth -> /Users/jonas/.pub-cache/hosted/pub.dev/firebase_auth-4.19.1/

./build/macos/Build/Intermediates.noindex/EagerLinkingTBDs/Debug/just_audio.framework/just_audio.tbd -> /Users/jonas/Documents/project_name/build/macos/Build/Intermediates.noindex/EagerLinkingTBDs/Debug/just_audio.framework/Versions/A/just_audio.tbd

./macos/Pods/Headers/Public/Firebase/Firebase.h -> ../../../Firebase/CoreOnly/Sources/Firebase.h

./ios/.symlinks/plugins/path_provider_foundation -> /Users/jonas/.pub-cache/hosted/pub.dev/path_provider_foundation-2.3.1/

./build/macos/Build/Products/Debug/FirebaseSharedSwift/FirebaseSharedSwift.framework/FirebaseSharedSwift -> Versions/Current/FirebaseSharedSwift

In total there are over 550 symlinks. Does any of this sound problematic?

I also tried the analyzer instrumentation logs but I can't really find anything that appears out of place. However, this document seems to log all kinds of stuff and I'm not sure how to analyze it or what to look for.

Yes, unfortunately, this project is private and can't be shared. I could probably share the analyzer instrumentation logs privately with you, if that would help.

I recently tested this project on a windows laptop and the issue occurs as well. (Thus, I think completely reinstalling my mac won't even help fix it).

These issues don't appear on new Flutter projects. Maybe I could try to download a big public project and see if I have problems with other bigger projects as well? If not it must be something specific with my project but I'm clueless about where the issue could be.

@DanTup
Copy link
Collaborator

DanTup commented May 8, 2024

In total there are over 550 symlinks. Does any of this sound problematic?

Nothing above looks like an issue to me - the issue would be if the link points further up the tree so that walking the tree could cause endless cycles.

I also tried the analyzer instrumentation logs but I can't really find anything that appears out of place

The most useful thing would be knowing what's being logged during the periods where performance is bad. So if the bad performance is during startup, it would be the start of the logs (until the first progress event with "kind": "end"). If it's during completion, it would be what occurs between the textDocument/completion request and the corresponding response.

Yes, unfortunately, this project is private and can't be shared. I could probably share the analyzer instrumentation logs privately with you, if that would help.

I can't accept anything confidential (instrumentation logs can contain contents of opened files and paths/errors for other files in the workspace), but if you're able to reproduce this on a copy of the project with anything confidential removed (and perhaps only a single test file open), you might be able to get a log that doesn't contain anything confidential you can share.

Something I forgot to ask earlier - can you confirm whether you have this option ticked in the VS Code (User or Workspace) settings?

image

@JonasJW
Copy link
Author

JonasJW commented May 8, 2024

Understood, I will go back to the logs and look for these progress events. If I have something to share I will follow up on it here.

The Dart: Only Analyze Projets With Open Files was disabled. I'm trying to enable it to see it that improves it, I will update you on this.

@DanTup
Copy link
Collaborator

DanTup commented May 8, 2024

The Dart: Only Analyze Projets With Open Files was disabled. I'm trying to enable it to see it that improves it, I will update you on this.

FWIW, my recommendation would be to not use that setting (and I should update the text). Although it sounds better for large projects, it was really added for a fairly specific case (command line editors that would provide the current working directory - which may be the user home dir - as the workspace folder). While the server will start up faster for very large workspaces, opening and closing files from different projects will trigger re-creating the analysis roots which can trigger "initial" analysis for those projects.

It's beneficial if you're opening a folder that contains 100 projects and maybe only opening files from a handful, but if you're jumping between a large portion of the projects in the workspace, it's probably worse overall.

@mraleph
Copy link
Member

mraleph commented Oct 3, 2024

@JonasJW what is your current experience with analysis performance? Are you still experiencing this weird behavior?

@JonasJW
Copy link
Author

JonasJW commented Oct 3, 2024

@mraleph yes, I'm still experiencing performance issues

@DanTup
Copy link
Collaborator

DanTup commented Oct 3, 2024

@JonasJW are you able to reproduce this issue with any public projects? It might be easier to narrow down with an instrumentation log, but that log will contain parts of source code from the open projects.

If it only occurs with this one specific internal project, is it possible to make a copy of it and see if you can narrow down what causes it (for example if it's made up of many packages, can you remove some of them to narrow down if it's caused by a specific package/set of packages)?

Do you also know if this issue occurs when typing in any file, or just some subset? This issue could be similar to #56307 which may be caused by the number of files that need to be reanalyzed as you modify a file (see #56307 (comment) and #56307 (comment)).

@amrgetment
Copy link

amrgetment commented Oct 3, 2024

Maybe a related issue
Memory Leaks in the custom_lint library
invertase/dart_custom_lint#248

@larssn
Copy link

larssn commented Oct 30, 2024

We have a pretty large project (internal, 200k lines), and this issue is pretty hard on productivity.

The worst of it comes from typing new class properties. The analyzer will get stuck on these for up to 20-30 seconds.

Here's my code --status and a screenshot of top memory in Activity Monitor.

Version:          Code 1.95.0 (912bb683695358a54ae0c670461738984cbb5b95, 2024-10-28T20:16:24.561Z)
OS Version:       Darwin arm64 24.0.0
CPUs:             Apple M1 Max (10 x 2400)
Memory (System):  32.00GB (0.28GB free)
Load (avg):       4, 5, 6
VM:               0%
Screen Reader:    no
Process Argv:     --crash-reporter-id 4e581022-c36a-4fd6-8d5e-12c2b8c113c3
GPU Status:       2d_canvas:                              enabled
                  canvas_oop_rasterization:               enabled_on
                  direct_rendering_display_compositor:    disabled_off_ok
                  gpu_compositing:                        enabled
                  multiple_raster_threads:                enabled_on
                  opengl:                                 enabled_on
                  rasterization:                          enabled
                  raw_draw:                               disabled_off_ok
                  skia_graphite:                          disabled_off
                  video_decode:                           enabled
                  video_encode:                           enabled
                  webgl:                                  enabled
                  webgl2:                                 enabled
                  webgpu:                                 enabled
                  webnn:                                  disabled_off

CPU %	Mem MB	   PID	Process
    0	   197	 47912	code main
    0	    98	 47915	   gpu-process
    0	    33	 47916	   utility-network-service
    0	   623	 47918	window [1] (sales_page.dart — tillty_pos)
    0	   492	 48349	extensionHost [1]
    1	    66	 48582	     electron-nodejs (config.js )
    0	     0	 48894	       /usr/bin/script -t 0 /dev/null /usr/bin/arch -arm64e xcrun xcdevice observe --usb
    0	    33	 48896	         /Applications/Xcode.app/Contents/Developer/usr/bin/xcdevice observe --usb
    0	     0	 48895	       /usr/bin/script -t 0 /dev/null /usr/bin/arch -arm64e xcrun xcdevice observe --wifi
    0	    33	 48897	         /Applications/Xcode.app/Contents/Developer/usr/bin/xcdevice observe --wifi
    6	  3736	 77823	       /Users/lars/Library/Android/sdk/emulator/qemu/darwin-aarch64/qemu-system-aarch64 -avd Pixel_Tablet_API_35
    1	    33	 77829	         /Users/lars/Library/Android/sdk/emulator/netsimd --host-dns=8.8.8.8,8.8.4.4
    0	    33	 48584	     /Users/lars/workspace/flutter/bin/cache/dart-sdk/bin/dart tooling-daemon --machine
    0	  1147	 48593	     /Users/lars/workspace/flutter/bin/cache/dart-sdk/bin/dart language-server --protocol=lsp --client-id=VS-Code --client-version=3.98.1
    0	    33	 48836	     electron-nodejs (languageserver.js )
    0	    33	 48880	     /Applications/Visual Studio Code.app/Contents/Frameworks/Code Helper (Plugin).app/Contents/MacOS/Code Helper (Plugin) /Applications/Visual Studio Code.app/Contents/Resources/app/extensions/json-language-features/server/dist/node/jsonServerMain --node-ipc --clientProcessId=48349
    0	    98	 48350	shared-process
    0	    33	 48351	fileWatcher [1]
    0	    66	 59875	ptyHost

Workspace Stats: 
|  Window (sales_page.dart — tillty_pos)
|    Folder (tillty_pos): more than 20467 files
|      File types: transitive_digest(3880) class(1518) dex(782) xml(648)
|                  h(635) len(597) grpc_back(510) json(440)
|                  jar(368) c(325)
|      Conf files: launch.json(1) tasks.json(1)
|      Launch Configs: dart(5)
image

(The top consumer seems to be the Android Emulator, which is probably fine)

Here's the Dart process (if relevant):
image

Anything else I can do to help debug this issue?

@NijatTagizada
Copy link

I saw some talk about this issue being especially bad on Windows. I'll add that I'm working on Mac and it feels miserably slow at times.

It's the same to me. Dart is slow in VSCode, but when I switch to IntelliJ, it works fine.

@mustafa-fahimi
Copy link

What is the solution at last? Turning off real time protection didn't solve anything for me.

@BenevidesLecontes
Copy link

BenevidesLecontes commented Dec 17, 2024

In our company we have a super app with 108 packages and the analyzer is too slow with intelijj or vscode, the experience is the same.
Dart is using 6 to 22 gb of memory, even with the new feature workspaces.

image

@mraleph
Copy link
Member

mraleph commented Dec 17, 2024

@BenevidesLecontes is it possible for you to try analyzer from dev channel to see if it improves memory usage?

@BenevidesLecontes
Copy link

BenevidesLecontes commented Dec 17, 2024

@mraleph I tried with the dev version and with latest pub version this is the result

CleanShot 2024-12-17 at 10 18 21@2x

I think is worth to mentioning that we use flutter_lints package.

Soon I change something on a shared package, the usage grows
CleanShot 2024-12-17 at 10 35 43@2x

@mraleph
Copy link
Member

mraleph commented Dec 17, 2024

@BenevidesLecontes could you stay with dev channel for a bit and see if memory usage grows out of hand?

Then can you open Analysis Server Diagnostic page (Dart Analysis -> Gear menu, see screenshot) and there click Collect Report and upload that report.

Screenshot 2024-12-17 at 14 36 59

@BenevidesLecontes
Copy link

BenevidesLecontes commented Dec 17, 2024

@mraleph, when you mention “analyzer from dev channel”, are you suggesting that I need to download the Dart dev version?

If that’s the case, I downloaded sdk-3.7.0-259.0.dev, but the analyzer is crashing. If not, could you clarify what you mean?

@mraleph
Copy link
Member

mraleph commented Dec 17, 2024

@BenevidesLecontes I meant to keep using whatever version you were using to get the last screenshot (where you wrote "I tried with the dev version and with latest pub version this is the result"). I assumed you used flutter dev channel to get this? Otherwise I am not sure what "dev" refers to in that comment.

@BenevidesLecontes
Copy link

@mraleph here is the report
Dart Analyzer Diagnostics Report (1).json

@mcmah309
Copy link

mcmah309 commented Dec 17, 2024

Experiencing this issue on projects with SDK 3.7.0-232.0.dev. Even with optimizations like "dart.onlyAnalyzeProjectsWithOpenFiles": true, in the workspace. The following is from typing out builder. and waiting for completion.

(name: <scheduler>, count: 0, elapsed: 0:00:00.000000, elapsedSelf: -0:00:06.462091)
  (name: getUnitElement, count: 7, elapsed: 0:00:00.000453, elapsedSelf: 0:00:00.000235)
    (name: libraryContext, count: 7, elapsed: 0:00:00.000218, elapsedSelf: 0:00:00.000213)
      (name: libraryCycle, count: 7, elapsed: 0:00:00.000005, elapsedSelf: 0:00:00.000005)
  (name: analyzeFile, count: 18, elapsed: 0:00:06.461638, elapsedSelf: 0:00:00.540169)
    (name: libraryContext, count: 18, elapsed: 0:00:05.921469, elapsedSelf: 0:00:00.045051)(bytesGet: 50880, cycleCount: 10, libraryCount: 10, libraryLoadCount: 10)
      (name: libraryCycle, count: 18, elapsed: 0:00:00.000800, elapsedSelf: 0:00:00.000800)
      (name: macroCompileKernel, count: 4, elapsed: 0:00:05.875618, elapsedSelf: 0:00:05.875618)

dart_analyzer_diagnostics_report.json

@jensjoha
Copy link
Contributor

@BenevidesLecontes the uploaded report is from dart 3.6 not the dev build which includes some fixes.

It seems from your comment about "the new feature workspaces" that you are using pub workspaces - but from the uploaded report you are not.
There are 108 contexts, combined they have 33,564 ("added") files and combined the 108 contexts knows about 3,277,755 files, but only 36,329 of these are unique.
Using pub workspaces (https://dart.dev/tools/pub/workspaces) I believe should make the number of contexts go to 1 and the combined known files go to 36,329 --- and all in all it should help the memory usage a lot.

@mraleph
Copy link
Member

mraleph commented Dec 18, 2024

@mcmah309 are you using macros? If you do - could you try without macros?

I also notice that you don't use workspaces either - might be a good idea to adopt them because you also have huge number of contexts.

@DanTup
Copy link
Collaborator

DanTup commented Dec 18, 2024

@mcmah309

Even with optimizations like "dart.onlyAnalyzeProjectsWithOpenFiles": true, in the workspace.

FWIW, that flag can make performance worse as well as better. It was originally intended for command-line editors that might provide a large workspace directory when only ever actually editing a single file.

While it will make initial analysis faster (because when there are no open files we don't analyze anything), it can cause extra work as you open/close files. The first file you open in a project will cause the analysis roots to change (which is relatively expensive) and then closing the last file in a project does the same. If you're jumping between files across a lot of projects (as opposed to working in a very small number of projects in a large workspace), it will probably work out worse - generally I recommend not using it.

@BenevidesLecontes
Copy link

@BenevidesLecontes the uploaded report is from dart 3.6 not the dev build which includes some fixes.

It seems from your comment about "the new feature workspaces" that you are using pub workspaces - but from the uploaded report you are not. There are 108 contexts, combined they have 33,564 ("added") files and combined the 108 contexts knows about 3,277,755 files, but only 36,329 of these are unique. Using pub workspaces (https://dart.dev/tools/pub/workspaces) I believe should make the number of contexts go to 1 and the combined known files go to 36,329 --- and all in all it should help the memory usage a lot.

Hi @jensjoha I did all the steps to move to workspaces, I have only one pubspec.lock with all my dependencies and the pub get and dart pub workspace list are working accordingly. And about the dart version I downloaded the sdk-3.7.0-259.0.dev and when I'm point to this version using the intellij it's crashing the analysis server.

CleanShot 2024-12-18 at 07 31 01@2x

@jensjoha
Copy link
Contributor

With regards to the workspace feature, could it be that you have opened the folders separately instead of just opening the folder that contains the workspace? I'm not familiar with IntelliJ, so I don't know what the correct names are there, but in VSCode, if I without the workspace feature opened the main folder (with, for my test case "project1" and "project2" folders inside it) I got 3 contexts: The main folder, project1 and project2. With the workspace feature enabled, opening the main folder gives me just 1 context, whereas opening the projects individually (i.e. a VSCode 'workspace' with two folders (project1 and project2)) I get 2 contexts. If that's the case, could you try to just open that (in lack of better words) "main folder"?

As for the crashing, is it that it just doesn't launch? Again I'm not familiar with the IntelliJ UI, but I would assume you'd need a full path to the SDK (say something like /what/ever/it/is/sdk-3.7.whatnot/sdk).

@BenevidesLecontes
Copy link

@jensjoha I have the full path, I only I hid it to post here. About the workspace, I have this structure, I open the main folder containing all the packages, apps and so on.
CleanShot 2024-12-18 at 08 06 37@2x

@mraleph
Copy link
Member

mraleph commented Dec 18, 2024

@BenevidesLecontes can you check if you have any .dart_tool folders nested inside packages/apps? or any pubspec.yaml which do not specify resolution: workspace? e.g. what do you get if you run:

$ cd path-to-app_3.0
$ find . -iname '.dart_tool'
...
$ find . -iname 'pubspec.yaml' -exec grep -L 'resolution:.*workspace' '{}' ';'

Both find commands should find just one .dart_tool and just one pubspec.yaml without resolution: workspace.

@BenevidesLecontes
Copy link

@mraleph I removed all .dart_tool folders to test, when i do pub get, it creates .dart_tool for all packages with this structure:
CleanShot 2024-12-18 at 08 31 33@2x

These commands:
find . -iname '.dart_tool' is listing all packages
find . -iname 'pubspec.yaml' -exec grep -L 'resolution:.*workspace' '{}' ';' is listing 3 yaml files, two are from the example folders of plugins and one is the root

CleanShot 2024-12-18 at 08 35 39@2x

@mraleph
Copy link
Member

mraleph commented Dec 18, 2024

@BenevidesLecontes this looks correct. It is puzzling that you get so many contexts then. Could you open Analyzer Diagnostics page and see what it says about those contexts, e.g. when I create a test workspace I get 1 context with multiple packages inside:

Screenshot 2024-12-18 at 12 46 02

Can you look through contexts that you have? Is workspace recognized correctly (e.g. you should have at least one context listing all your packages)? If it is recognized correctly - try to look what other contexts do you have.

@BenevidesLecontes
Copy link

BenevidesLecontes commented Dec 18, 2024

@mraleph I blurred some paths but this is what i have. I have analysis_options.yaml for every packages (I read somewhere that this create contexts too), but i need them to linting every package.
CleanShot 2024-12-18 at 08 51 54@2x

@mraleph
Copy link
Member

mraleph commented Dec 18, 2024

@BenevidesLecontes this seems just one of the context, and it only has 7 packages inside. How do other contexts look like?

I think analysis_options.yaml should not create separate contexts anymore. @srawlins could you confirm?

@BenevidesLecontes
Copy link

@mraleph removed the analysis_options.yaml from some packages and the contexts decreased. Can i have one analysis_options.yaml for the entire workspace?

@BenevidesLecontes
Copy link

BenevidesLecontes commented Dec 18, 2024

I Removed all packages analysis_options.yaml file and put only one inside the root folder, now I have only one context (and the lint is working for the entire project). This is the memory usage
CleanShot 2024-12-18 at 09 38 18@2x

@DanTup
Copy link
Collaborator

DanTup commented Dec 18, 2024

@BenevidesLecontes do you have any analysis server plugins enabled?

I think analysis_options.yaml should not create separate contexts anymore. @srawlins could you confirm?

When that work was done, it could break plugins because plugins would no longer get the context roots they expected. To fix this, roots are still created for any folder where an analysis_options.yaml changes which plugins are enabled (see #56547). I'm not sure it explains this number of contexts though, unless plugins are enabled and disabled across many levels.

If plugins are enabled, it would be useful to try removing them from all analysis_options.yamls and see how many contexts there are then.

@BenevidesLecontes
Copy link

BenevidesLecontes commented Dec 18, 2024

@BenevidesLecontes do you have any analysis server plugins enabled?

I think analysis_options.yaml should not create separate contexts anymore. @srawlins could you confirm?

When that work was done, it could break plugins because plugins would no longer get the context roots they expected. To fix this, roots are still created for any folder where an analysis_options.yaml changes which plugins are enabled (see #56547). I'm not sure it explains this number of contexts though, unless plugins are enabled and disabled across many levels.

If plugins are enabled, it would be useful to try removing them from all analysis_options.yamls and see how many contexts there are then.

@DanTup I have this in my analysis_options.yaml

include: package:flutter_lints/flutter.yaml

analyzer:
  errors:
    one_member_abstracts: ignore
    todo: ignore
    dead_code: error
    invalid_annotation_target: false
  exclude:
    - build/**
    - "**/*.g.dart"
    - "**/*.freezed.dart"
    - "**/*.mocks.dart"
linter:
  rules:
    public_member_api_docs: true
    directives_ordering: true
    prefer_relative_imports: true
    library_private_types_in_public_api: false
    require_trailing_commas: true
    prefer_single_quotes: true
    unnecessary_lambdas: true
    use_if_null_to_convert_nulls_to_bools: true
    prefer_final_locals: true

@srawlins
Copy link
Member

I think analysis_options.yaml should not create separate contexts anymore.

I think this is either true or a qualified true. I'm not sure if specifying plugins or enabling experiments still requires a new analysis context. @pq @keertip can you say?

@hasimyerlikaya
Copy link

Lately, working with Flutter has been really tough and frustrating. Most of the time, the Dart analyzer freezes after a hot restart, and I can’t do anything. Code completion becomes extremely slow, and undefined errors occur frequently.

I even reinstalled macOS to resolve this issue, but the situation remains the same. We need to find a solution to this problem because it’s almost impossible to work like this right now.

image image
Click to see the log ``` #0 ElementImpl2.enclosingElement2 (package:analyzer/src/dart/element/element.dart:3106:38) #1 _ReferenceFinder.visitAssignmentExpression (package:analysis_server/src/services/correction/dart/import_add_show.dart:71:51) #2 AssignmentExpressionImpl.accept (package:analyzer/src/dart/ast/ast.dart:954:15) #3 ExpressionStatementImpl.visitChildren (package:analyzer/src/dart/ast/ast.dart:6366:17) #4 RecursiveAstVisitor.visitExpressionStatement (package:analyzer/dart/ast/visitor.dart:1099:10) #5 ExpressionStatementImpl.accept (package:analyzer/src/dart/ast/ast.dart:6362:50) #6 NodeListImpl.accept (package:analyzer/src/dart/ast/ast.dart:13009:20) #7 BlockImpl.visitChildren (package:analyzer/src/dart/ast/ast.dart:2051:17) #8 RecursiveAstVisitor.visitBlock (package:analyzer/dart/ast/visitor.dart:883:10) #9 BlockImpl.accept (package:analyzer/src/dart/ast/ast.dart:2047:50) #10 IfStatementImpl.visitChildren (package:analyzer/src/dart/ast/ast.dart:9730:20) #11 RecursiveAstVisitor.visitIfStatement (package:analyzer/dart/ast/visitor.dart:1273:10) #12 IfStatementImpl.accept (package:analyzer/src/dart/ast/ast.dart:9724:50) #13 NodeListImpl.accept (package:analyzer/src/dart/ast/ast.dart:13009:20) #14 BlockImpl.visitChildren (package:analyzer/src/dart/ast/ast.dart:2051:17) #15 RecursiveAstVisitor.visitBlock (package:analyzer/dart/ast/visitor.dart:883:10) #16 BlockImpl.accept (package:analyzer/src/dart/ast/ast.dart:2047:50) #17 BlockFunctionBodyImpl.visitChildren (package:analyzer/src/dart/ast/ast.dart:2007:12) #18 RecursiveAstVisitor.visitBlockFunctionBody (package:analyzer/dart/ast/visitor.dart:889:10) #19 BlockFunctionBodyImpl.accept (package:analyzer/src/dart/ast/ast.dart:1999:50) #20 FunctionExpressionImpl.visitChildren (package:analyzer/src/dart/ast/ast.dart:8624:11) #21 RecursiveAstVisitor.visitFunctionExpression (package:analyzer/dart/ast/visitor.dart:1213:10) #22 FunctionExpressionImpl.accept (package:analyzer/src/dart/ast/ast.dart:8613:50) #23 NodeListImpl.accept (package:analyzer/src/dart/ast/ast.dart:13009:20) #24 ArgumentListImpl.visitChildren (package:analyzer/src/dart/ast/ast.dart:476:16) #25 RecursiveAstVisitor.visitArgumentList (package:analyzer/dart/ast/visitor.dart:823:10) #26 ArgumentListImpl.accept (package:analyzer/src/dart/ast/ast.dart:472:50) #27 InstanceCreationExpressionImpl.visitChildren (package:analyzer/src/dart/ast/ast.dart:10518:19) #28 RecursiveAstVisitor.visitInstanceCreationExpression (package:analyzer/dart/ast/visitor.dart:1309:10) #29 InstanceCreationExpressionImpl.accept (package:analyzer/src/dart/ast/ast.dart:10507:15) #30 VariableDeclarationImpl.visitChildren (package:analyzer/src/dart/ast/ast.dart:18577:19) #31 RecursiveAstVisitor.visitVariableDeclaration (package:analyzer/dart/ast/visitor.dart:1806:10) #32 VariableDeclarationImpl.accept (package:analyzer/src/dart/ast/ast.dart:18572:50) #33 NodeListImpl.accept (package:analyzer/src/dart/ast/ast.dart:13009:20) #34 VariableDeclarationListImpl.visitChildren (package:analyzer/src/dart/ast/ast.dart:18701:16) #35 RecursiveAstVisitor.visitVariableDeclarationList (package:analyzer/dart/ast/visitor.dart:1812:10) #36 VariableDeclarationListImpl.accept (package:analyzer/src/dart/ast/ast.dart:18695:15) #37 FieldDeclarationImpl.visitChildren (package:analyzer/src/dart/ast/ast.dart:7042:16) #38 RecursiveAstVisitor.visitFieldDeclaration (package:analyzer/dart/ast/visitor.dart:1135:10) #39 FieldDeclarationImpl.accept (package:analyzer/src/dart/ast/ast.dart:7037:50) #40 NodeListImpl.accept (package:analyzer/src/dart/ast/ast.dart:13009:20) #41 ClassDeclarationImpl.visitChildren (package:analyzer/src/dart/ast/ast.dart:2939:13) #42 RecursiveAstVisitor.visitClassDeclaration (package:analyzer/dart/ast/visitor.dart:937:10) #43 ClassDeclarationImpl.accept (package:analyzer/src/dart/ast/ast.dart:2929:50) #44 NodeListImpl.accept (package:analyzer/src/dart/ast/ast.dart:13009:20) #45 CompilationUnitImpl.visitChildren (package:analyzer/src/dart/ast/ast.dart:3621:21) #46 RecursiveAstVisitor.visitCompilationUnit (package:analyzer/dart/ast/visitor.dart:961:10) #47 CompilationUnitImpl.accept (package:analyzer/src/dart/ast/ast.dart:3614:50) #48 ImportAddShow.compute (package:analysis_server/src/services/correction/dart/import_add_show.dart:48:10) #49 AssistProcessor._addFromProducers.compute (package:analysis_server/src/services/correction/assist_internal.dart:207:24) #50 AssistProcessor._addFromProducers (package:analysis_server/src/services/correction/assist_internal.dart:228:15) #51 AssistProcessor.compute (package:analysis_server/src/services/correction/assist_internal.dart:181:5) #52 DartCodeActionsProducer.getAssistActions (package:analysis_server/src/lsp/handlers/code_actions/dart.dart:133:21) #53 OperationPerformanceImpl.runAsync (package:analyzer/src/util/performance/operation_performance.dart:174:14) #54 CodeActionHandler.handle.. (package:analysis_server/src/lsp/handlers/handler_code_actions.dart:235:20) #55 ErrorOrRecord2Extension.mapResults (package:analysis_server/src/lsp/error_or.dart:125:12) #56 ErrorOr.mapResult (package:analysis_server/src/lsp/error_or.dart:94:11) #57 ServerStateMessageHandler.handleMessage (package:analysis_server/src/lsp/handlers/handlers.dart:452:11) #58 LspAnalysisServer._handleRequestMessage (package:analysis_server/src/lsp/lsp_analysis_server.dart:1068:18) #59 LspAnalysisServer.handleMessage.. (package:analysis_server/src/lsp/lsp_analysis_server.dart:509:15) #60 OperationPerformanceImpl.runAsync (package:analyzer/src/util/performance/operation_performance.dart:174:14) #61 LspAnalysisServer.handleMessage. (package:analysis_server/src/lsp/lsp_analysis_server.dart:491:11)

@hasimyerlikaya
Copy link

In my tests conducted on the beta channel and a new Flutter project, the app randomly freezes during a hot restart.

After a long period, the error below occurs, and the restart process completes.

This issue happens randomly—sometimes there’s no problem even after several restarts, while other times it happens on the first attempt.

[+60373 ms] Error writing "main.dart.dill" to DevFS: HttpException: Request has been aborted

[+4206 ms] Skipping target: gen_localizations
[        ] Skipping target: gen_dart_plugin_registrant
[        ] Skipping target: _composite
[        ] complete
[   +3 ms] Scanned through 644 files in 2ms
[        ] <- reset
[        ] Compiling dart to kernel with 0 updated files
[        ] Processing bundle.
[        ] <- recompile package:flutter_analyzer/main.dart c9a01cee-4e05-4db5-8f4c-d2c2778fe0f9
[        ] <- c9a01cee-4e05-4db5-8f4c-d2c2778fe0f9
[   +1 ms] Bundle processing done.
[  +36 ms] Updating files.
[        ] Pending asset builds completed. Writing dirty entries.
[+60373 ms] Error writing "main.dart.dill" to DevFS: HttpException: Request has been aborted
[        ] trying again in a few - 9 more attempts left
[ +993 ms] DevFS: Sync finished
[        ] Synced 53.6MB.
[        ] <- accept

@srawlins
Copy link
Member

@hasimyerlikaya it sounds like your problems are related to running several processes at once. Or at least, we can't tell if there is one culprit, from your description. Can you confirm that you are having a problem with the Dart Analyzer, meaning things like your IDE's Problems panel is slow to refresh, code completion is slow (which you do show in your screenshot), syntax highlight gets stale, things like that?

What is your IDE?

It sounds like you might be running your app alongside your IDE in a simulator or an emulator. Can you confirm that? What if you code for a long session in your IDE without running the simulator or emulator? Does the Dart Analyzer perform better then?

In my tests conducted on the beta channel and a new Flutter project, the app randomly freezes during a hot restart.

This is not a problem with the Dart Analyzer. Please open a new issue if you wish to report a bug with the Dart VM or with hot restart.

@hasimyerlikaya
Copy link

@srawlins Apologies for the incomplete information. I should have provided a more detailed explanation.

Development Environment:
• IDE: VSCode 1.96.1
• Dart Extension: 3.103.20241211
• Computer: iMac M1 16GB - macOS 15.2 (24C101)
• Device: iPhone Xs Max - iOS 18.2
• Flutter: 3.28.0-0.1.pre
• Dart: 3.7.0-209.1.beta

Issues Encountered After Flutter 3.27 Update:

1. Hot Restart Issue:
When performing a hot restart, the process hangs and never completes. Upon investigating, I found that the issue occurs at the line I mentioned in my previous comment. Since I suspected this was related to the Dart analyzer, I brought it up here, although it might not be the right place to report this.

2. Slow Code Completion:
The second and more significant issue is the extremely slow code completion. Whenever I write code, I have to wait several seconds for suggestions to appear. Additionally, the Dart analyzer underlines the code with red wavy lines, which disappear only after a delay of several seconds. I switched to the beta version to mitigate this issue, and although it performs slightly better than the stable version, it is still slow.
Upon checking the Analysis Server Diagnostics screen, I noticed two errors. The data there also highlights how slow code completion is.

Next Steps:
I plan to record a video demonstrating the issue and share it with you. Meanwhile, I am testing my project by disabling packages to identify if any specific package might be causing these problems.

Let me know if you need further details!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
analyzer-stability area-analyzer Use area-analyzer for Dart analyzer issues, including the analysis server and code completion. P2 A bug or feature request we're likely to work on type-performance Issue relates to performance or code size
Projects
None yet
Development

No branches or pull requests