DevTools automatically resumes after current breakpoint if paused when connecting

[jakemac53]

Repro instructions

Create any app, and set a breakpoint on the first line inside of main:

void main() {
  print('hello'); // Set breakpoint here
}

In VsCode, debug this app (click the debug tooltip above the main function).

It should stop at the breakpoint you set.

Click the button to open up the CPU profiler

The app will now advance past the breakpoint you set, and exit (unless you have more breakpoints set).

Expected behavior

I expect it to stay stopped at a breakpoint - I set it here in order to be able to resume it once I have connected the profiler and started recording frames.

[kenzieschmoll]

Possibly related to #8789.

[kenzieschmoll]

Possibly related to flutter/flutter#158958. @jakemac53 do you have VM Developer Mode enabled in DevTools? (Open DevTools and check the top level settings)

[jakemac53]

I do not have VM Developer Mode enabled, should I?

[kenzieschmoll]

No, you don't need to. In flutter/flutter#158958, the workaround was to disable VM Developer Mode.

I'm suspicious that this is a problem with the CPU profiler itself, but rather a problem with DevTools automatically resuming upon connecting to the app. I know @elliette worked on some breakpoint-related logic last year that could be related to this bug.

[jakemac53]

but rather a problem with DevTools automatically resuming upon connecting to the app

Yes this seems to be exactly what is happening

[jakemac53]

Indeed if I even just do the "open devtools" command pallete option - it resumes as soon as that connects.

[jakemac53]

Is this because devtools assumes it is connecting to something which hasn't actually started yet? Like an app which set pause-isolates-on-start?

[elliette]

Adding some notes here. This is something @bkonyi @DanTup and I worked on last year.

A high level summary is that DDS, which sits between our debugging tools (e.g. DevTools, IDEs) and the VM Service is responsible for deciding when to forward a resume event to the VM Service. It does so by waiting for the a readyToResume event to be sent from each of the attached debugging tools (which is the event sent by the debugging tools once they have set their breakpoints) and a normal resume event sent by the tools in response to the user triggering a resume.

What we would expect to happen here:

• User launches their app in debug mode
• DAP sets requireUserPermissionToResume. This tells DDS that it needs to wait for a resume event triggered by the user along with the readyToResume events sent from the tool

It's possible requireUserPermissionToResume wasn't correctly set, or when clicking on the CPU profiler a resume event is erroneously being sent (which would indicate to DDS that the user had clicked "resume"). Depending on what is going wrong, the fix might not be in DevTools.

[jakemac53]

I think the issue might be that this is only for the pause on start and pause on exit states? (see https://github.com/dart-lang/sdk/blob/b7a9e2b5d90807c137832c221f50d29f53d107c0/pkg/dds/lib/src/isolate_manager.dart#L446 and https://github.com/dart-lang/sdk/blob/b7a9e2b5d90807c137832c221f50d29f53d107c0/pkg/dds/lib/src/isolate_manager.dart#L99).

So, if it is paused at a user breakpoint then there isn't logic in there to not resume when it connects? Unless the _IsolateState.pauseStart state handles all pauses and not just the initial pause on startup.

[elliette]

So, if it is paused at a user breakpoint then there isn't logic in there to not resume when it connects? Unless the _IsolateState.pauseStart state handles all pauses and not just the initial pause on startup.

To clarify, by when "it connects" do you mean when DevTools connects?

I think that is possible, I think it's also possible that we are sending a resume event from DevTools when it's launched that we shouldn't be sending.

Side note: If I remember correctly, the pause start state means that isolates are initially paused on start-up and also paused after any hot-restarts.

[jakemac53]

To clarify, by when "it connects" do you mean when DevTools connects?

Yes, I think it isn't getting connected until I either open the devtools web app or open the CPU profiler, so it is connecting once the isolate is already running, but doesn't recognize/handle that?

[mkorbel1]

Is there any known workaround for this issue (other than setting two breakpoints)?

[jakemac53]

Is there any known workaround for this issue (other than setting two breakpoints)?

Not that I know of, I have just been setting multiple breakpoints.

[elliette]

I ran into this running a Flutter app from VS Code, and re-reading the issue description here, it seems like this also is occurring when setting a breakpoint in VS Code, not DevTools.

I suspect there has been a regression from when #7231 was completed.

Writing down some notes before I forget on how to narrow this down:

• Does this occur for both web and non-web apps? (if both, likely an issue at DDS/DAP level, if non-web only, then issue in VM Service)
• Does this occur in DevTools when starting an app with --start-paused, opening DevTools, setting a breakpoint, then resuming? (If yes, then issue might be in DDS, if no, issue might be in DAP).

[jakemac53]

I ran into this running a Flutter app from VS Code, and re-reading the issue description here, it seems like this also is occurring when setting a breakpoint in VS Code, not DevTools.

Correct - you have to already be stopped at a breakpoint, and then connect with devtools, once devtools connects the app will resume (but it shouldn't).

You probably could reproduce this with two devtools instances as well, by setting a breakpoint in one, then connecting with another to the same app.

[DanTup]

I haven't tried debugging, but searching the code I see that BreakpointManager has this flag to control whether it runs switchToIsolate when it's created:

devtools/packages/devtools_app/lib/src/screens/debugger/breakpoint_manager.dart

Line 18 in 5252a88

BreakpointManager({this.initialSwitchToIsolate = true});

And at the end of switchToIsolate is this:

devtools/packages/devtools_app/lib/src/screens/debugger/breakpoint_manager.dart

Lines 90 to 93 in 5252a88

	// Resume the isolate now that the breakpoints have been set:
	await serviceConnection.serviceManager.isolateManager.resumeIsolate(
	isolateRef,
	);

Assuming this is the path that's triggering it, I wonder if that resume call should be wrapped in a check that the isolate is paused for one of the start reasons and not a breakpoint? Scanning through the DAP code, I think it only does the equivalent if the existing isolates have kPausePostRequest or kPauseStart pause events.

[kenzieschmoll]

Raising the priority of this bug from P2 to P1.

[elliette]

Assigning to myself so we don't have multiple folks working on this at once 😄

Assuming this is the path that's triggering it, I wonder if that resume call should be wrapped in a check that the isolate is paused for one of the start reasons and not a breakpoint? Scanning through the DAP code, I think it only does the equivalent if the existing isolates have kPausePostRequest or kPauseStart pause events.

Yes, just did a little investigation and that is indeed where it's being called. Wrapping in a check to make sure we aren't paused at a breakpoint works (yay!). But I'm wondering if DevTools should even be trying to manage breakpoints in this environment...

I was looking back at the meeting notes from last year, and I'm pretty sure we are in this situation (copy-pasted from notes):

DAP launches app with command line flag --pause-isolates-on-start
By default, requireUserPermissionToResume is true
DAP knows that it started the process, so it immediately calls requireUserPermissionToResume = false
DAP calls requirePermissionToResume(onStart: true)
DAP calls readyToResume when it resumes
After restart, DDS does not wait for the user’s resume event

I don't think we (or at least I) ever considered that DevTools would be sending resume requests if a user was debugging from their IDE, and hadn't attached/launched a separate DevTools instance. However, because the BreakpointManager is being initialized in framework_core, it is being triggered even when a user is debugging from their IDE.

The only places it is being used are in the debugger panel (hidden when a user is in their IDE) and the VM developer tools panel. Potentially we should either not initialize it if DevTools embedded in IDE and the connected app is not a VM app, or move initialization out of framework_core.

I think for now @DanTup's suggestion (only resume here for kPausePostRequest or kPauseStart events) is sufficient, but wonder if folks (cc @kenzieschmoll @bkonyi) have thoughts about initializing BreakpointManager when embedded in an IDE.

[DanTup]

have thoughts about initializing BreakpointManager when embedded in an IDE.

When reproducing this issue, I'm not using DevTools embedded, I'm opening it in a browser from the IDE.

I believe part of the goal of the readyToResume work was so that we could stop hiding the DevTools debugger when there's already another IDEs debugger attached, so I feel like it would be better to continue in the direction or trying to make multiple debuggers play nice together, rather than back towards only allowing one.

Although, I will note that the idea of "only resuming if the pause event is x" is also somewhat racy, because it's not guaranteed that the isolate is still in that state when the resume is sent (ideally we would probably send a "resume this isolate if it is in pauseStart/pausePostRequest state", but whether in practice it's likely to happen and worth supporting that, I don't know).