x/mobile: binding doens't seem to reuse build cache for modules

[karalabe]

Apologies for not adhering to the default issue template, the issue is not a "bug" per se and not an issue with Go itself, so all those version and env infos would be useless.

gomobile added module support recently, which works spectacularly. One thing, however, that I did notice, is that if I call gomobile bind on a package that has heavy dependencies, those get rebuilt from scratch for every build, making the build very painful.

E.g. Binding https://github.com/coronanet/go-coronanet will take 30 mins as it needs to build a dependent package that contains a huge C library (Tor). However, trying to call bind again with zero modifications will again take 30 mins. This is in contrast with building simply with Go locally, which will take 5 mins for the first run, but will be ~instantaneous afterwards (only link time is left).

My best guess until now is that gomobile bind creates its own temporary working directory /tmp/gomobile-work-xxxxxxxx and thus all code operations are short circuited into that folder, away from the system caches. This results in all object files produced by one build run to be deleted at the end.

Issue opened per @hajimehoshi 's request in #27234 (comment).

---

Continuing a thread from the above issue:

Aha, gobind, which is invoked by gomobile, generates a source code from Go packages for binding, and they are not cached indeed. Did you mean 'cache' of them?

No, I'm not referring to caching the generated files, rather the module dependencies of them, which should most definitely be cacheable.

[hajimehoshi]

CC @hyangah

[MariusVanDerWijden]

The temporary directory is created in the following line: https://github.com/golang/mobile/blob/0df4eb2385467a487d418c6358313e9e838256ae/cmd/gomobile/env.go#L61
Maybe you could add a flag that allows to change this line to an existing directory e.g.
gomobile bind -dep /home/user/mobile-deps

[sunny-lan]

The solution mentioned by @MariusVanDerWijden seems to work. Is this going to be added anytime soon?

[hajimehoshi]

My understanding is that Go's build cache is created somewhere, wherever you build your application. Then the temporary directory should not matter. Is that correct? I believe gomobile does not delete Go's cache explicitly.

If so, is the culprit building Java or C files?

[hajimehoshi]

@karalabe I'm sorry for the delay. Is this problem happening in Android or iOS, or both?

[spoukke]

Hi @hajimehoshi

I'm having the same issue here. Indeed, I specify the workdir using the flag -cache when I use gomobile bind in Github Actions. I put this directory in cache with actions/[email protected] as long as the generated .aar file for android, and the .framework file for ios. However, the build starts over each time.

[fortuna]

Any chance we can get the solution in https://go-review.googlesource.com/c/mobile/+/273406 merged?

PR on Github: golang/mobile#58 (though the discussion is on googlesource)

[modasi]

you can try this :
https://github.com/modasi/gomobile-cache-mod

[hajimehoshi]

This is not related to the cache, but I hope https://go-review.googlesource.com/c/mobile/+/426274 would mitigate the slowness of Android building. I'll try iOS later.

[hajimehoshi]

@modasi What change did you add to gomobile?

EDIT: I think this is it (initializing tmpdir)

	if buildN {
		tmpdir = "$WORK"
		cleanupFn = func() {}
	} else if buildWork {
		tmpdir = "/tmp/gomobilework"
		fmt.Fprintln(xout, "WORK="+tmpdir)

	} else {
		tmpdir, err = ioutil.TempDir("", "gomobile-work-")
		if err != nil {
			return nil, err
		}
	}

[hajimehoshi]

gomobile added module support recently, which works spectacularly. One thing, however, that I did notice, is that if I call gomobile bind on a package that has heavy dependencies, those get rebuilt from scratch for every build, making the build very painful.

Did this issue exist before Go modules support was introduced? If not, I was wondering how supporting Go modules triggerred this...

[gopherbot]

Change https://go.dev/cl/442195 mentions this issue: cmd/gomobile: concurrent build for iOS archive files

[hajimehoshi]

Can anyone test the latest gomobile?

[hajimehoshi]

Let me close this. Feel free to reopen this when you find this unresolved.