[WASI] experiment - tracking #65895
Comments
|
Tagging subscribers to 'arch-wasm': @lewing Issue DetailsDescriptionThis issue is to track known issues in the early WASI-enabled runtime builds. These need to be resolved in order to have a proper supportable WASI-ready release.
Reproduction StepsExpected behaviorActual behaviorRegression?No response Known WorkaroundsNo response ConfigurationNo response Other informationNo response
|
dotnet-issue-labeler
bot
added
the
untriaged
lambdageek
removed
the
untriaged
lambdageek
added
the
tracking
|
Is exporting functions to wasi tracked here too? |
Added |
|
@pavelsavara Thanks! We also need the opposite: Exporting functions so that you can create a .NET library and compile it to wasm and consume it from the other languages. It's already possible from Go, C, C++, and Rust. See this sample: https://github.com/extism/csharp-pdk/blob/master/samples/SamplePlugin/Functions.cs#L9 Currently you have to write the C glue code manually: https://github.com/extism/csharp-pdk/blob/master/samples/SamplePlugin/native.c#L11 |
That would be Also there trouble is sharing memory between caller and called code. We will implement this assuming shared memory (with the host engine) and passing pointers. And the "WASI component model" with isolated memory, lowering/lifting etc will need different attributes in the future future. |
|
Is there any way to produce raw Wasm MVP files from .NET 9 after #104683 has landed? We've been "piggy-backing" on .NET's Wasi target to produce Wasm files targeted for custom hosts, as, unlike C/C++/Rust/etc .NET doesn't have support for raw ("hostless") Wasm output and WASI is the closest target where we could simply stub out all the imports during compilation and rely on This has been possible starting from the original @SteveSandersonMS' Wasi.Sdk and up until and including .NET 8. I now wanted to try and test how hard it would be to migrate to .NET 9 preview as it has some exciting Wasm features - in particular, AOT and Which brings me to the crux of my question - is there any way path to opt-out from Wasm component output or, even better, an official example that shows how to produce a raw "hostless" Wasm in plain old MVP format, going to be provided in .NET 9 or does this mean we're stuck with .NET 8 with its slow interpreter mode for IL bytecode for the foreseeable future? UPD: I should probably add that, by decomposing components into individual Wasm files using |
|
We switched to WASI preview 2 and component model in Net9 Preview 7. We now depend on wasi:http component import directly. We don't have dev bandwidth to support multiple build configurations, I'm sorry. |
|
I imagine that's going to break quite a few users in similar positions to us. Out of curiosity, what was the driver behind that decision? Wasm components are still in rather raw stages, and the engine support is pretty poor compared to regular Wasm. While it's interesting to watch its development, it seems rather too early to switch to it as the primary output format for stable .NET version, cutting off majority of engines other than I guess Wasmtime. Perhaps there is some So, while I totally understand lack of bandwidth for actual development & support, perhaps there is a way to just provide a different link-stage flag so that .NET would produce just this one single file, which already has the required shape? We can do the rest of the stubbing ourselves. |
|
You could try disable We are also passing Here is WASM & WASI ABI, I'm not sure which of them would apply.
"Skate to where the puck is going, not where it has been." |
Thanks, this is helpful, going to take a look.
Yeah I suspect this is going to be a problem, too. I looked at it briefly before, and I think I could hack around it with MSBuild's
That is fine, since we are going to stub them out anyway, it doesn't matter which version is used. On the C ABI side it still behaves like a regular C import, after all, so declaring stubs isn't very different from the original preview. It's really just the components + WIT that cause issues, which is why I'm hopeful that what we want is achievable with purely a link-time change.
Heh, I agree with the sentiment, just saying it feels... risky / early for a switch, given the very limited native engine support. I liked WASI p1 for its simplicity and compatibility with any regular Wasm, and was one of probably dozens of people who made their own in-browser implementations at the time because it was just a matter of loading same Wasm file but with custom imports. Oh well, either way that's beside the point for this particular usecase. |
For Net10 possibly yes. But I'm afraid we will break it again ... let's give it a try. |
|
@RReverser Did you ever get this workaround working? I'm in the same boat where I'm trying to code to an existing host that only supports modules, not components. .NET 9 preview 6 was working for me, but for some reason I couldn't get it to include native assets. |
|
This definitely seems like a misstep. There is practically no use of WASI Preview 2[0] in the ecosystem, and very few runtime maintainers are interested in supporting this moving target. WASI Preview 1 is still in significant demand, and it would be a shame to have to drop .NET support in our projects because of this decision.
The puck definitely does not seem to be moving in the direction you're alluding to with this change. Based on the component model implementation across the runtime landscape, this severley limits the utility of .NET in the Wasm ecosystem. If that is the intention of the team here, then I suppose you are making the right call. [0]: from https://crates.io/crates/wasi
|
|
My personal position as one of the original creators of WebAssembly is that "skating towards where the puck is going" has always been necessary to deliver a good developer experience, and it will continue to be necessary. It's not always possible to predict where the puck is going to end up, but following the standards process is decent way to go about it.
By the same logic used here to argue for staying on p1 forever (look at this linear scale graph showing usage of a widely deployed piece of technology vs an emerging technology), it would i.e. be appropriate to only support a single dominant web browser and ignore every other browser, based on current usage statistics. At one point, the proven technology was asm.js - widely supported and widely used - while the emerging technology was WASM. In the same way, one could argue that WASI itself shouldn't be supported, since WASI usage is tiny in comparison to usage of client web browsers. Surely that's not the goal? Given the amount of time it takes for core runtime changes to percolate through the ecosystem into developers' hands, and then the time it takes for developers to adopt a new core runtime version and deploy it in their applications, it's not sufficient to ask "what does WASM look like today? what does WASI look like today?" because by the time your answer(s) end up on an end user's computer or on a server in the cloud, years have passed. It's not sufficient to just say "this is the wrong answer right now" and use that to close off any forward-looking development. Some key questions are:
Just off the top of my head, I don't see a good solution for HTTP in WASI p1, so I feel those three questions are illuminating in this context. My personal advice in this context would be to consider stating what the user needs are and how those needs are best served by staying still instead of moving forward, or alternately, suggest which direction the forward movement should be aimed in and how that will serve users. |
|
@nilslice Hi there, Rust Project member here. I'd be careful not to read too much into the download data of one specific Rust package. The WASI 0.1 (nee This is relevant because in order for crates to support both WASI 0.1 and also introduce support for WASI 0.2, a compiler target #[cfg(all(target_os = "wasi", target_env = "p1"))]
mod wasip1 { /* WASI 0.1 items go here */ }
#[cfg(all(target_os = "wasi", target_env = "p2"))]
mod wasip2 { /* WASI 0.2 items go here */ }In order for crates to test whether they work on CI they require a working WASI 0.2 toolchain, which in turn requires having a prebuilt compiler toolchain available. While it was technically possible to build your own WASI 0.2 toolchain for Rust before Rust 1.82, we've recommended against people doing that. That has meant that targeting WASI 0.2 from existing libraries essentially hasn't been possible. This will change with the upcoming Rust 1.82 release, at which point we will likely see a gradual roll-out of support for WASI 0.2 throughout the ecosystem. I hope that provides some insight into the evolution of Rust's WASI targets. If you'd like to read more, please see the post: Changes to Rust's WASI targets, as well as the draft for the upcoming release: The wasm32-wasip2 Target has Reached Tier 2 Support. Footnotes |
|
We use Wazero as our WASM host runtime. I was planning to offer C# as a target guest language, but I guess since Wazero has voted no to WASI-p2 and .NET has voted no to WASI-p1, that sadly we'll just have to tell our customers to pick a different language. See |
|
@yoshuawuyts, hi - I don't think the point you're making really lands here or is related to the .NET support concern, but thanks!
I have been reading and following these closely for a few years, it's always a good proxy to measure interest. Considering it's my job to know what users want from Wasm, I read a lot of crates.io and other package manager data.
Wasip2 launched in January of this year (WebAssembly/WASI#577), so I'd put the starting line for adoption metrics there. It's had nearly zero uptick, except from a few insiders within the Bytecode Alliance. This is despite countless strong pushes, conferences, and other evangelism. It's just not sticking. I deeply care about WebAssembly, and have its best interest in mind, and I do not see the ecosystem having voted with their usage of Wasip2. Now, maybe if it really is that the Rust target alone is the winning solution, then that will change. But it still does NOT address anything to do with this thread, which is squarely about .NET tooling compiling to core standard Wasm Modules vs. non-standard unadopted Wasm components. This is where the problem lays, and I worry will further fragment the ecosystem and at the same time strip .NET languages from their utility to Wasm projects.
I have read this, and saw the problematic release and reversal of this announcement. It's not the most reassuring state of things to now think about depending on this target in any way. |
@nilslice ? - I was directly responding to your comment in #65895 (comment), where you claim that the downloads of a single Rust ecosystem crate are indicative of broader WASI 0.2 ecosystem adoption. You use that to further the argument that the dotnet maintainers should shy away from supporting WASI 0.2. I didn't make the connection between Rust and dotnet, you did. Posting just the screenshot on its own can be misleading as it omits context. If you're going to discuss download counts, it seems relevant to explain that library authors can't practically add a dep on |
|
I'm going to try and steer this thread back to reality, and to the core concern of neutering .NET's utility to the greater Wasm ecosystem. @kg, much respect for your work on Wasm and continued interest, but:
This is precisely a reason not to drop targeting standardized core modules... it's widely deployed piece of technology that .NET will now lost traction with. I don't understand why this is the way you would approach the rollout of an emerging technology that is unproven, undesired, and unnecessary to a significant portion of the Wasm ecosystem. By all means, go ahead and keep experimenting, but it would appear that the only reason you'd do this is to force components and wasip2 on unsuspecting users. It feels very hostile that if you can't lure users with quality, you force it this way.
Again, that amount of time equals the amount of time the latest versions of .NET become effectively useless to the greater Wasm ecosystem. It seems rather self-destructive to go this route, no?
I think this is untrue, unless you'd care to specify that you mean "a good solution for running a virtual HTTP server in WASI p1"? So:
First, users want their code to be useful, and usefulness starts with being able to run. This should make it obvious that supporting standard core modules which run everywhere is preferable to users, vs. running a non-standard component, which runs basically nowhere.
Largely like they do today, and the abstraction already provided by Wasm removes the need to worry. Perhaps I don't follow what this has to do with the overall concern though.
None. However, I'm not saying that we stop trying to innovate and improve. I'm not saying the .NET team shouldn't support components. I'm just saying don't severely limit .NET's reach and utility to the vast existing ecosystem of Wasm users and runtimes that have no need or interest in components. |
|
As a user of dotnet and the wasip1 target. I'd ask for a reconsideration on components as well, as we won't be able to use this if it requires component model. A lot of the debate in this thread seems to be about wasp1 vs wasip2. I don't think people have a problem with the system interface evolving. It's actually a good thing and is very welcome. The concern is that, as I currently understand it, wasip2 requires the component model which locks it out of most of the runtimes we use. I seem to recall some wasi maintainers talk about not needing to couple both wasip2 and component model, but i'm not sure the status of that. But I think if you could just make the component model piece optional, then it would satisfy all parties. |
|
I've read the thread and spend a bit of time to reflect before adding my comments. I appreciate the time that has been put into this discussion. I think for those who want it or need it, WASIp2 might be great. However, based on the the original post, it seems support for multiple runtimes was one of the original goals behind creating the issue:
WASIp2 is supported in only one Wasm runtime, while WASIp1 is supported in almost the totality of server-side Wasm runtimes. I'm afraid that the decision of not supporting WASIp1 might decrease the overall usage of .NET in the broader WebAssembly ecosystem, which is also hinted by other comments on this thread. One of the concerns showcased is that only one target can be supported: either WASIp1 or WASIp2... which is turning the discussion into the wrong angle in my view. Please note that even the Bytecode Alliance (who are the champions behind WASIp2) have promised to keep updating and supporting WASIp1, specially because of industry needs. As an example, if supporting the Personally, I think the best outcome would be to support both targets: WASIp1 and WASIp2. After doing a bit of research, I don't see a good reason why the two targets can't be supported, specially as the changes done in the PR to support WASIp2 (or p1) on the codebase are rather trivial: #104683
Of course, I don't think is reasonable to push the maintenance cost into someone else. Given the concerns presented, I think is only fair to put that maintenance cost on the interested parties rather than someone else. Given that, I'm proposing the following:
We will eat the burden and bandwidth for supporting an extra build configuration so the Wasm community is able to use .NET in runtimes where WASIp2 is not supported, so .NET supports also WASIp1. For those that wants to use WASIp1 in most of the runtimes, they could. For those that want to use WASIp2 with the new binary format... they also could. Will that work for you @pavelsavara @kg? Thanks a lot for your great work! |
@enghch Yes, it's pretty ugly (as expected), but the core of it is <Target Name="RemoveWasmComponents" BeforeTargets="_WasmWriteRspForLinking">
<ItemGroup>
<!-- hardcoded path because only pure paths can be excluded by wildcard in `Remove=...` while we want to remove a custom concatenated string -->
<_WasmLinkStepArgs Remove="-Wl,--component-type,"$([MSBuild]::NormalizePath('$(MicrosoftNetCoreAppRuntimePackRidNativeDir)', 'WasiHttpWorld_component_type.wit').Replace('\','/'))""/>
<_WasmLinkStepArgs Include="-fuse-ld=lld" />
</ItemGroup>
</Target>This disables components from being linked in and produces raw Wasm MVP library - although you still need to deal with unknown imports depending on what you are doing. It's far from ideal (and I don't have much to add to the discussion about the ideal state of things, aside from what has already been said above), but at least it hacks around the immediate problem. |
|
|
|
For others trying to build Wasm libraries, #101434 is another issue that can be confusing to debug until you realise what's going on (it prevents Luckily, the workarounds are relatively simple - either use Other than these two issues + a need for a proper component model opt-out flag (I'll still try to get around to making that PR for .NET 10), it's possible to make everything work at least as well as in .NET 8. |
|
One more pretty obscure issue: exports with 9+ arguments compile but crash in runtime. #109338 |
|
Also running into #101276 when trying to use AOT (the main motivation for upgrading to .NET 9). Looks like there are more issues for now than I had hoped, so maybe we'll postpone the upgrade. |
|
@RReverser any conclusion regarding .NET 9? |
Well, I did get it to work, but there are a bit too many hacks and workarounds for the bugs mentioned above. At least a bit faster than .NET 8 thanks to AOT (around 20% in benchmarks that need to do a lot of work in C# itself rather than in the host imports). |
|
@RReverser ah I see, are those hacks open source? also, did it impact binary size? |
I'm going to write something up on my personal blog post, but we decided not to proceed with the upgrade after all. Way too hacky to work around the listed issues, for relatively little benefit. The potential breakage risks don't make it worth it yet. Pinning to .NET 8 for now and hoping that by .NET 10 the situation will improve, maybe even with decidated pure Wasm support. |
and others
Description
This issue is to track known issues in the early WASI-enabled runtime builds. These need to be resolved in order to have a proper supportable WASI-ready release.
wasi-wasmRIDOperatingSystem.IsWasi()#78389 API changeSystem.Nativeusing WASI SDK instead of using the Emscripten-built binary.wasm_m2n_invoke.g.hSystem.Native. Currently there's a small hardcoded list.dotnet/wasi-sdk)dotnet/wasi-sdkpackages #95086mono_gc_finalize_notifyingc.c).synthetic-pthread.ccode to support having separate pthread keys for each thread instead of a single global list.ves_icall_System_Reflection_Assembly_InternalLoad. #95137[DLLImport]which doesn't match known static native symbols #86984[UnmanagedCallersOnly]#86985Build
and a public distribution mechanism.wasi-experimentalworkload,wasiconsoletemplate, andWasi.Build.Tests#81849)wasi-experimentalworkload,wasiconsoletemplate, andWasi.Build.Tests#81849)wasi-experimentalworkload,wasiconsoletemplate, andWasi.Build.Tests#81849)wasi-experimentalworkload,wasiconsoletemplate, andWasi.Build.Tests#81849)wasi-experimentalworkload,wasiconsoletemplate, andWasi.Build.Tests#81849)The text was updated successfully, but these errors were encountered: