Handling of stdcall (and other x86-32-specific ABIs) on non-x86-32 Windows targets is inconsistent

[RalfJung]

We generally accept stdcall on non-x86-32 Windows targets (while rejecting it on other non-x86-32 targets). See the discussion about this at #86231 (comment) and #86231 (comment). However, @ChrisDenton points out that the raw-dylib feature is more strict here. That's an inconsistency that we should probably fix?

@ChrisDenton can you give an example of what exactly is being rejected by raw-dylib here?

Cc @workingjubilee @rust-lang/lang @nagisa @petrochenkov

[ChrisDenton]

It's just generally more strict. As far as I recall this is intentional, cc @dpaoliello. For example:

#[link(name = "library", kind = "raw-dylib")]
extern "cdecl" { // or stdcall
    fn test();
}

This is rejected on x86_64-pc-windows-msvc and aarch64-pc-windows-msvc. But remove kind = "raw-dylib" and it works.

[RalfJung]

I think we should make up our mind: either "stdcall" is a legal ABI string on these targets, or it is not.

[RalfJung]

@Aditya-PS-05 currently this issue is about discussing what to do next; there is no agreed-upon plan for what to do yet.

---

Reading the past comments, it seems like ~noone argued in favor of the status quo. So maybe the best next step is to have a FCW lint against

• stdcall, fastcall on everything but 32-bit x86
• vectorcall on everything but x86 (32bit or 64bit)
• EDIT: cdecl on everything but 32-bit x86

This should probably re-introduce the lint that was removed in #129935.

@ChrisDenton also mentioned cdecl above; I did not have that on my radar yet. Currently, that one is accepted everywhere. Which targets should it support?

Either way, raw-dylib shouldn't have its own "does this ABI make sense" logic (currently located here); it should call is_abi_supported like everything else. (But that function might return a three-state value to indicate "not really but for backwards compatibility we accept this in some cases; and raw-dylib can then make a hard error rather than an FCW for those.)

[dpaoliello]

It's just generally more strict. As far as I recall this is intentional, cc @dpaoliello.

Yes, it was intentional, although I would describe it more as "let's start restrictive and we'll loosen things later" rather than "this is the absolute correct behavior": #86419 (comment)

Either way, raw-dylib shouldn't have its own "does this ABI make sense" logic (currently located here); it should call is_abi_supported like everything else.

raw-dylib has its own logic for historical reasons: primarily that it wasn't supported for all calling conventions or all arches for a while, hence its own logic to filter to what it did support.

Personally, I'm going to bow out of this discussion. I previously said that having this behavior would cause a proliferation of cfg_attr attributes without a huge benefit to safety or correctness but was told that #87678 would eventually be an error and so this would be moot: https://rust-lang.zulipchat.com/#narrow/channel/233931-t-compiler.2Fmajor-changes/topic/Add.20import_name_type.20parameter.20to.20.23.5Blink.5D.20compiler-team.23525

[RalfJung]

I previously said that having this behavior would cause a proliferation of cfg_attr attributes without a huge benefit to safety or correctness

Not sure what exactly you are referring to with "this behavior". Maybe this part of your Zulip message?

If using stdcall is an error on x86_64, then I have no idea how you'd write this without duplicating the entire extern block...

For the case of "stdcall on win32, C everywhere else", we have the system ABI, so for this case forbidding "stdcall" on non-x86 would not cause any duplication.

[RalfJung]

"cdecl" is an ABI that we have supported since pre-1.0, but it is implemented as a complete alias for C. Nominally this is the name of a particular x86 ABI, so we should probably deprecate this on all non-x86 targets together with the others mentioned above.

[RalfJung]

@ChrisDenton

Knowing the history here and the reasons behind those decisions is the first step.

I'm pretty sure the history is

• way before Rust 1.0 we didn't think about this very carefully so we just allowed all ABIs everywhere
• at some point we started having a notion of some ABIs only working on specific architectures / targets, but for backwards compatibility all the existing ABIs were grandfathered in
• Replace per-target ABI denylist with an allowlist #86231 started cleaning this up for some ABIs, but @nagisa didn't want to reject more cases since "I feel like a change like this will want an input from T-lang and perhaps some sort of an FCP" (source)

So... I would propose we ask t-lang to approve the intention of completing what was started in #86231. The goal would be:

• thiscall, stdcall, fastcall, cdecl should only be accepted on x86-32
• vectorcall should only be accepted on x86-32 and x86-64

The status quo is:

• thiscall is only accepted on x86-32 (this one is already correct 🎉 )
• cdecl is accepted everywhere
• stdcall, fastcall are accepted on x86-32 as well as on all Windows targets
• vectorcall is accepted on x86-32 and x86-64 as well as on all Windows targets (this ABI is still gated behind abi_vectorcall, so changing it is not a breaking change)

If people want the behavior of "stdcall on win32, C everywhere else", they can use system. "cdecl" can unconditionally be replaced by "C". If they have different reasons to use any of these ABIs on targets where they don't make sense, we do not have a direct replacement (so we might learn about good use-cases for this when we start linting against this).

@ChrisDenton @workingjubilee any objections to this plan?

[ChrisDenton]

I have no objections.

[nagisa]

Yeah, a few years sounds like a reasonably long timeline to start thinking about the next steps here. I do recall there being some push-back against making thiscall/stdcall/fastcall work on x86-32 only on the grounds of "MSVC accepts everything everywhere and I don't want to think when rewriting my windows C code to Rust", but I personally think the world is going to be a better place if we take a principled stance here.

[RalfJung]

I do recall there being some push-back against making thiscall/stdcall/fastcall work on x86-32 only on the grounds of "MSVC accepts everything everywhere and I don't want to think when rewriting my windows C code to Rust"

Do you have a link to that? In the final PR, so far I only saw @petrochenkov also asking for the ABIs to be rejected on other targets.

Also note that thiscall is already x86-32-only. I couldn't find a record in our bugtracker of anyone complaining about that.

[ChrisDenton]

I do wish we had a nicer way for users to select ABI based on target that doesn't involve either duplicating a lot of unrelated code or putting the entire extern block in a macro. I think that would have made the decision here a lot easier.

[RalfJung]

I seem to recall there was a proposal once for allowing macros in the ABI string position, which would make this slightly less annoying albeit still a bit awkward.