Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failure to debug a suite test that is in a different file than the caller test #2414

Closed
nirhaas opened this issue Aug 15, 2022 · 20 comments · May be fixed by #2415
Closed

Failure to debug a suite test that is in a different file than the caller test #2414

nirhaas opened this issue Aug 15, 2022 · 20 comments · May be fixed by #2415
Labels
Debug Issues related to the debugging functionality of the extension. go-test issues related to go test support (test output, test explorer, ...)
Milestone

Comments

@nirhaas
Copy link
Contributor

nirhaas commented Aug 15, 2022

What version of Go, VS Code & VS Code Go extension are you using?

Version Information
  • Run go version to get version of Go from the VS Code integrated terminal.
go version go1.18.1 darwin/arm64
  • Run gopls -v version to get version of Gopls from the VS Code integrated terminal.
Build info
----------
golang.org/x/tools/gopls v0.9.3
    golang.org/x/tools/[email protected] h1:wfh4cfJAKwOG49sCE4ldafYeD5rlejE/gJQ7JAR5yX0=
    github.com/BurntSushi/[email protected] h1:Rt8g24XnyGTyglgET/PRUNlrUeu9F5L+7FilkXfZgs0=
    github.com/google/[email protected] h1:e6P7q2lk1O+qJJb4BtCQXlK8vWEO8V1ZeuEdJNOqZyg=
    github.com/sergi/[email protected] h1:we8PVUC3FE2uYfodKH/nBHMSetSfHDR6scGdBi+erh0=
    golang.org/x/exp/[email protected] h1:7Xs2YCOpMlNqSQSmrrnhlzBXIE/bpMecZplbLePTJvE=
    golang.org/x/[email protected] h1:6zppjxzCulZykYSLyVDYbneBfbaBIQPYMevg0bEwv2s=
    golang.org/x/[email protected] h1:uVc8UZUe6tr40fFVnUP5Oj+veunVezqYl9z7DYw9xzw=
    golang.org/x/[email protected] h1:v4INt8xihDGvnrfjMDVXGxw9wrfxYyCjk0KbXjhR55s=
    golang.org/x/[email protected] h1:olpwvP2KacW1ZWvsR7uQhoyTYvKAupfQrRGBFM352Gk=
    golang.org/x/[email protected] h1:yg8cGxL0g/WIlkF25aKuIAZacqXROXw4Dt57iKLWg8w=
    golang.org/x/[email protected] h1:BkeW9/QJhcigekDUPS9N9bIb0v7gPKKmLYeczVAqr2s=
    honnef.co/go/[email protected] h1:ytYb4rOqyp1TSa2EPvNVwtPQJctSELKaMyLfqNP4+34=
    mvdan.cc/[email protected] h1:avhhrOmv0IuvQVK7fvwV91oFSGAk5/6Po8GXTzICeu8=
    mvdan.cc/xurls/[email protected] h1:tzxjVAj+wSBmDcF6zBB7/myTy3gX9xvi8Tyr28AuQgc=
go: go1.18.1
  • Run code -v or code-insiders -v to get version of VS Code or VS Code Insiders.
1.70.1
6d9b74a70ca9c7733b29f0456fd8195364076dda
arm64
  • Check your installed extensions to get the version of the VS Code Go extension
v0.35.1
  • Run Ctrl+Shift+P (Cmd+Shift+P on Mac OS) > Go: Locate Configured Go Tools command.
Checking configured tools....
GOBIN: undefined
toolsGopath: 
gopath: /Users/nhaas/go
GOROOT: /opt/homebrew/Cellar/go/1.18.1/libexec
PATH: /Users/nhaas/go/bin:/Users/nhaas/Library/Python/3.8/bin:/opt/homebrew/opt/libpq/bin:/Users/nhaas/.tgenv/bin:/Users/nhaas/.tfenv/bin:/opt/homebrew/opt/make/libexec/gnubin:/opt/homebrew/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/go/bin:/Users/nhaas/.cargo/bin:/Users/nhaas/.fzf/bin

	go:	/opt/homebrew/bin/go: go version go1.18.1 darwin/arm64

	gotests:	/Users/nhaas/go/bin/gotests	(version: v1.6.0 built with go: go1.18.1)
	gomodifytags:	/Users/nhaas/go/bin/gomodifytags	(version: v1.16.0 built with go: go1.18.1)
	impl:	/Users/nhaas/go/bin/impl	(version: v1.1.0 built with go: go1.18.1)
	goplay:	/Users/nhaas/go/bin/goplay	(version: v1.0.0 built with go: go1.18.1)
	dlv:	/Users/nhaas/go/bin/dlv	(version: v1.8.3 built with go: go1.18.1)
	golangci-lint:	/Users/nhaas/go/bin/golangci-lint	(version: v1.48.0 built with go: go1.18.1)
	gopls:	/Users/nhaas/go/bin/gopls	(version: v0.9.3 built with go: go1.18.1)

go env
Workspace Folder (vscode-go): /Users/nhaas/Temp/vscode-go
	GO111MODULE=""
	GOARCH="arm64"
	GOBIN=""
	GOCACHE="/Users/nhaas/Library/Caches/go-build"
	GOENV="/Users/nhaas/Library/Application Support/go/env"
	GOEXE=""
	GOEXPERIMENT=""
	GOFLAGS=""
	GOHOSTARCH="arm64"
	GOHOSTOS="darwin"
	GOINSECURE=""
	GOMODCACHE="/Users/nhaas/go/pkg/mod"
	GONOPROXY=""
	GONOSUMDB=""
	GOOS="darwin"
	GOPATH="/Users/nhaas/go"
	GOPRIVATE=""
	GOPROXY="https://proxy.golang.org,direct"
	GOROOT="/opt/homebrew/Cellar/go/1.18.1/libexec"
	GOSUMDB="sum.golang.org"
	GOTMPDIR=""
	GOTOOLDIR="/opt/homebrew/Cellar/go/1.18.1/libexec/pkg/tool/darwin_arm64"
	GOVCS=""
	GOVERSION="go1.18.1"
	GCCGO="gccgo"
	AR="ar"
	CC="clang"
	CXX="clang++"
	CGO_ENABLED="1"
	GOMOD="/Users/nhaas/Temp/vscode-go/go.mod"
	GOWORK=""
	CGO_CFLAGS="-g -O2"
	CGO_CPPFLAGS=""
	CGO_CXXFLAGS="-g -O2"
	CGO_FFLAGS="-g -O2"
	CGO_LDFLAGS="-g -O2"
	PKG_CONFIG="pkg-config"
	GOGCCFLAGS="-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/t8/83vffqv11_bbfxbbdc3fsfx40000gn/T/go-build4218516277=/tmp/go-build -gno-record-gcc-switches -fno-common"

Share the Go related settings you have added/edited

  "go.lintTool": "golangci-lint",
  "go.formatTool": "gosimports",
  "go.formatFlags": ["-local", "github.com/company"],
  "gopls": {
    "ui.semanticTokens": true
  },

Describe the bug

Terminology: When using testify suite, there is

  1. The suite itself (struct)
  2. The suite tests (test functions)
  3. A normal test that runs suite.Run with the suite struct.

Following #1130, when clicking the button to "Debug test" for a suite test that is in a different file than the "normal test", we never stop at the breakpoint, and get testing: warning: no tests to run printed on the debug console.

I suspect that the reason for this behavior is that the function findAllTestSuiteRuns (

export function findAllTestSuiteRuns(
) only has access to the document symbols when trying to figure out whether there is a suite.Run. If the "normal test" is in a different file (which is different document), it returns an empty array, which would later be parsed to a wrong -test.run flag to dlv.

From dlv log output (note that ^$):

2022-08-15T21:51:57+03:00 debug layer=dap parsed launch config: {
	"mode": "test",
	"program": ".",
	"args": [
		"-test.run",
		"^$",
		"-testify.m",
		"^TestSomething$"
	],
	"env": {
		"GOPATH": "/Users/nhaas/go"
	},
	"backend": "default",
	"stackTraceDepth": 50
}

Steps to reproduce the behavior:

Follow the steps of #1130 and try to click "Debug test" for a suite test in the other file.

@gopherbot
Copy link
Collaborator

Change https://go.dev/cl/424094 mentions this issue: fix multifile suite test fails to debug

@alankritjoshi
Copy link

I’ve been facing this issue for the last two weeks.

A workaround for me for now was to begin debug test from the file where the test suite is declared or use the Testing side panel.

Thanks for identifying it and the fix!

@jamalc jamalc added the Debug Issues related to the debugging functionality of the extension. label Aug 29, 2022
@jamalc jamalc modified the milestones: Untriaged, vscode-go/later Aug 29, 2022
@divmgl
Copy link

divmgl commented Oct 24, 2022

A workaround for me for now was to begin debug test from the file

This actually doesn't work for me at all. Running debug test on individual Golang tests for files that are outside of where the test suite is declared simply doesn't work.

The only fix I've found is to create a test suite per file using the main test suite embedded.

For instance, say I have a test suite in main_test.go named APISuite, in tenant_test.go I have to use:

type TenantSuite struct {
	APISuite
}

func TestTenantSuite(t *testing.T) {
	suite.Run(t, new(TenantSuite))
}

func (s *TenantSuite) TestUserHasTenant() {
	accessToken := s.registerUser()

Unfortunate that this issue is still happening. It's been several years.

@Altiano
Copy link

Altiano commented Nov 4, 2022

I just hope they prioritize this one..

@shimonacarvalho
Copy link

Also having this issue in my tests.

@divmgl
Copy link

divmgl commented Nov 9, 2022

It does look like there's some movement on a fix for this: #2415 (comment)

@esdrasbeleza
Copy link

Also having this issue :(

@denis-sazonov-gaembla
Copy link

doesn't work for me as well :(

et added a commit to highlight/highlight that referenced this issue Feb 22, 2023
## Summary

<!--
Ideally, there is an attached Linear ticket that will describe the
"why".

If relevant, use this section to call out any additional information
you'd like to _highlight_ to the reviewer.
-->

Currently, we can only do a direct match for key attributes (e.g.
`workspace_id:1`).

This PR improves the matching, namely:

* Add wildcard support (`service:*foo*` will match `service:barfoobaz`)
* Add multi-word support `service:'image processor'` will match
`service:image processor`)

Additionally this PR improves code quality:

* Use a true query builder instead of rolling our code. I chose to use
[squirrel](https://github.com/Masterminds/squirrel). This makes
conditional queries a lot easier and removes usages of `fmt.Sprintf`
which is just ripe for a sql injection.
* Drop support for [testify
suite](https://github.com/stretchr/testify#suite-package). This is
currently unusable with vscode
(golang/vscode-go#2414).

## How did you test this change?

Verified space search works
![Screenshot 2023-02-21 at 10 50 49
AM](https://user-images.githubusercontent.com/58678/220422792-fb2dcd79-5ad8-453f-9feb-81f354f539ab.png)

Verified wildcard search works
![Screenshot 2023-02-21 at 10 56 00
AM](https://user-images.githubusercontent.com/58678/220422942-34fdf347-268a-4372-8fee-19280e2ef39b.png)


Verified we can search with body as well
![Screenshot 2023-02-21 at 10 56 36
AM](https://user-images.githubusercontent.com/58678/220423058-e8770d97-7a12-437a-bbce-b3a3a340895c.png)



## Are there any deployment considerations?

<!--
 Backend - Do we need to consider migrations or backfilling data?
-->

N/A
@jdahm
Copy link

jdahm commented Aug 29, 2023

I want to use vscode-go for developing where possible, but I use this pattern too often to make this a smooth experience. Really looking forward to this getting resolved soon.

@hendrics
Copy link

is there a work around for this? Or there's no way to debug a test function from viscose? What folks do instead?

@piavgh
Copy link

piavgh commented Sep 25, 2023

is there a work around for this? Or there's no way to debug a test function from viscose? What folks do instead?

Same, I really want to remove Goland from my development environment (it eats up too much system resource), but the developer experience in Goland is so great (especially in this individual test case).

@MortenGerdes
Copy link

Would love to see this fixed. I've also been using this testing suite quite a lot and have tilted towards using Goland (or IntelliJ with Go plugin) because it just allowed for debugging this so seemless.

Is there anything we can do to see the fix for this merged?

@makeitraina
Copy link

Would love to see this fixed! Any update?

@hyangah hyangah added the go-test issues related to go test support (test output, test explorer, ...) label Dec 6, 2023
@gopherbot
Copy link
Collaborator

Change https://go.dev/cl/555676 mentions this issue: src/goTest: fix multifile suite test fails to debug

@morremeyer
Copy link

Just verified that this is fixed in the nightly¹ version. Thanks @Cr4zySheep for taking this on!

This makes my (and I assume many others) dev experiences much more smooth.

¹If you want to get this asap, disable the Go extension and install Go Nightly instead

@hyangah hyangah removed this from the vscode-go/backlog milestone Mar 14, 2024
@hyangah hyangah added this to the v0.41.2 milestone Mar 14, 2024
@matsaleh13
Copy link

I'm running v0.41.4 of the extension and this problem is happening for me now. I saw from the changelog that the fix was applied to v0.41.2, however. Any chance there's been a regression? AFAIK, I shouldn't have to do anything special to make use of this fix, right? Thanks.

@marcelo-rocha
Copy link

I'm running v0.41.4 of the extension and this problem is happening for me now. I saw from the changelog that the fix was applied to v0.41.2, however. Any chance there's been a regression? AFAIK, I shouldn't have to do anything special to make use of this fix, right? Thanks.

I've noticed the test function that creates the test suite cannot have anything else than the test suite creation itself, so I suggest checking if your function is like this:

func TestMySuite(t *testing.T) {
	suite.Run(t, new(MySuite))
}

@matsaleh13
Copy link

I've noticed the test function that creates the test suite cannot have anything else than the test suite creation itself, so I suggest checking if your function is like this:

Oh wow... that's very specific, isn't it?

My code (not working):

func TestRunSuite(t *testing.T) {
	batchSuite := new(KVDatastoreSuite)
	suite.Run(t, batchSuite)
}

... and now, fixed:

func TestRunSuite(t *testing.T) {
	suite.Run(t, new(KVDatastoreSuite))
}

Thanks very much for that suggestion! I would never have thought of that.

I guess it makes sense that the extension would have to do some kind of code parsing to be able to figure out what's going on, right?

Thanks again!

@adityachowdhry-probo
Copy link

Having the same issue on version v0.41.4

My test suite structure looks like this

func TestMySuite(t *testing.T){
  t.Run("TestA", func(t *testing.T){
    t.Run("TestASubTest", func(t *testing.T){
    })
  })
  t.Run("TestB", func(t *testing.T){
  })
}

Am I doing something wrong here?

@firelizzard18
Copy link
Contributor

@adityachowdhry-probo This issue is specifically about test suites built with stretchr. Generic subtest support is tracked in #2445.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Debug Issues related to the debugging functionality of the extension. go-test issues related to go test support (test output, test explorer, ...)
Projects
Status: Done