MSDTC With SQL Server and Containers

[SRJames]

Is it at all possible to have the following scenario:

An instance of SQL Server running on an EC2 and connected to an AD Domain somedomain.com.

A Windows Container running in EKS with a client application that connects to the SQL server (outside EKS) using MSDTC ?

I have ensured that the node and the SQLServer can participate in transactions, but I am unable to get the pod to work. I receive the error :

"The MSDTC transaction manager was unable to pull the transaction from the source transaction manager due to communication problems. Possible causes are: a firewall is present and it doesn't have an exception for the MSDTC process, the two machines cannot find each other by their NetBIOS names, or the support for network transactions is not enabled for one of the two transaction managers. (Exception from HRESULT: 0x8004D02B)
at System.Transactions.Oletx.IDtcProxyShimFactory.ReceiveTransaction(UInt32 propgationTokenSize, Byte[] propgationToken, IntPtr managedIdentifier, Guid& transactionIdentifier, OletxTransactionIsolationLevel& isolationLevel, ITransactionShim& transactionShim)
at System.Transactions.TransactionInterop.GetOletxTransactionFromTransmitterPropigationToken(Byte[] propagationToken)"

Thanks

[Howard-Haiyang-Hao]

I’m curious about the network being used here. Is it necessary to enable port forwarding? Thank you.

[SRJames]

Using hostaliases on the client deployment and wireshark I can see that the requests on port 135 are being made to the SQL server host machine. But the response back to the ipaddress of the pod fails.
I assume this is because it should be going via a Service.
But how, if needed, should I approach redirecting traffic to a service and then making all msdtc port range available.
I have done a lot of searching over the last few weeks and haven't found any examples so I'm not sure if this is possible.

If I used a nodeport service is it possible to redirect the traffic for the SQL server to the pod to go the service?

[ntrappe-msft]

Hi @SRJames. Could you try to repro this issue using AKS instead of EKS? It would be helpful for us to know whether or not the platform is influencing the behavior you're seeing or if it's truly a Windows Containers problem.

[SRJames]

I will hopefully be getting some assistance from AWS on this and will update with any findings. However could anyone confirm if it is necessary to have gmsa configuredt in order for this to work. Thanks

[ntrappe-msft]

Ok thanks for letting us know and please keep us updated! We've contacted someone on the gMSA team to provide clarity. We'll let you know when we hear something.

[SRJames]

An update and a question.
Update: no luck as yet.
Question: I am basing my image on the windowsservercore ltsc2022 image.
As part of the image creation I am setting msdtc to use a fixed port by setting the registry as detailed here Single Port

However this seems to be completely ignored and the call to port 135 always returns the same port range 49152 - 49156.
Have these ports been fixed in the base image?

[microsoft-github-policy-service]

This issue has been open for 30 days with no updates.
no assignees, please provide an update or close this issue.

[microsoft-github-policy-service]

This issue has been open for 30 days with no updates.
no assignees, please provide an update or close this issue.

[microsoft-github-policy-service]

This issue has been open for 30 days with no updates.
no assignees, please provide an update or close this issue.

[efbird]

Hey, curious if you have any update on this? We have encountered the exact same issue, Kube pods (not on domain) connecting to SQL (on domain) getting the same error message.

Did you got anywhere with AWS? Looks like they have some blogs for using gMSA as a potential work around:
https://aws.amazon.com/blogs/containers/windows-authentication-on-amazon-eks-windows-pods/
https://aws.amazon.com/blogs/containers/domainless-windows-authentication-for-amazon-eks-windows-pods/

[SRJames]

No, we didn't get it to work. After spending approx 3 months trying we gave up. We could find nobody at AWS that had ever got it to work. We're now ripping msdtc out of our product.

[efbird]

Tried with gMSA, both with hosts on the domain and the domainless version, neither seemed to work, continued to get the same error.

We spoke with a EKS SA from AWS and stepped through some troubleshooting with him, and it looks like this is more of an issue with MSDTC being containerised, then anything specific to EKS.

I noticed that MS mention its not supported in AKS: https://learn.microsoft.com/en-us/virtualization/windowscontainers/quick-start/lift-shift-to-containers#msdtc

And it mentions "If using gMSA the name must match the hostname which must match the gMSA account name." Which I did have set up, but it still didn't work.

We are also now looking to remove MSDTC.

[microsoft-github-policy-service]

This issue has been open for 30 days with no updates.
no assignees, please provide an update or close this issue.

[microsoft-github-policy-service]

This issue has been open for 30 days with no updates.
no assignees, please provide an update or close this issue.

[microsoft-github-policy-service]

This issue has been open for 30 days with no updates.
no assignees, please provide an update or close this issue.

[microsoft-github-policy-service]

This issue has been open for 30 days with no updates.
no assignees, please provide an update or close this issue.

[microsoft-github-policy-service]

This issue has been open for 30 days with no updates.
no assignees, please provide an update or close this issue.

[microsoft-github-policy-service]

This issue has been open for 30 days with no updates.
no assignees, please provide an update or close this issue.

[microsoft-github-policy-service]

This issue has been open for 30 days with no updates.
no assignees, please provide an update or close this issue.

[vrapolinario]

Getting VERY late to this, but better late than never.
Unfortunately, I don't think we have good news on this. We have documented that MSDTC is not supported on AKS here: https://learn.microsoft.com/en-us/virtualization/windowscontainers/quick-start/lift-shift-to-containers#msdtc

The main thing is that we realized that some networking components would have be changed for MSDTC to work in a Kubernetes environment such as AKS - and I'm assuming it's what you are seeing in EKS - that we don't have enough signals to prioritize. For now, this is not supported as we can't repro your issue on a non-supported environment. With that said, I'd also check if you can go back to basics on this - does your environment work in a non K8s set up?

[vrapolinario]

Also, I'm going ahead and close this - since it's not supported according to our docs. Happy to keep the conversation going, though.