Allow bots to join friend worlds. #473
Comments
|
@LucienHH could you point to your current WIP work on this? |
|
Yeah sure, here's what I've found so far, joining a peer to peer session is explained here by a support rep: https://educommunity.minecraft.net/hc/en-us/articles/360047118992-FAQ-IT-Admin-Guide-
First part to joining a friend's session is by fetching all sessions currently available to join by using the https://sessiondirectory.xboxlive.com API: POST Body {
"type": "activity",
"scid":"4fc10100-5f7a-4470-899b-280835760c07",
"owners": {
"people": {
"moniker": "people",
"monikerXuid": "<xuid>"
}
}
}This will give you a list of sessions to join, within the session's body there will be an object called SupportedConnections which could contain one or more connection types: {
"SupportedConnections": [
{
"ConnectionType": 3,
"HostIpAddress": "",
"HostPort": 0,
"WebRTCNetworkId": 15778080650974306186
}
]
}Here we have a supported connection of type 3 where We now need to connect to the signalling channel where we'll send and receive offers/answers. Mojang have a WebSocket connection available at To connect to the WebSocket we first have to get an MCToken, we can do so from the endpoint below, this will most likely need implementing into prismarine-auth: POST Body {
"device": {
"applicationType": "MinecraftPE",
"capabilities": ["RayTracing"],
"gameVersion": "1.20.51",
"id": "<device_id>",
"memory": "34185707520",
"platform": "Windows10",
"playFabTitleId": "20CA2",
"storePlatform": "uwp.store",
"treatmentOverrides": null,
"type": "Windows10"
},
"user": {
"language": "en",
"languageCode": "en-GB",
"regionCode": "GB",
"token": "<playfab_token>"
"tokenType": "PlayFab"
}
}Response {
"result": {
"authorizationHeader": "<MCTOKEN>",
"validUntil": "2024-01-14T23:43:10Z",
"treatments": [
// bunch of flags
],
"configurations": {
"minecraft": {
"id": "Minecraft",
"parameters": {
// bunch of key values
}
}
}
}
}With the MCToken we can now create a connection to the WebSocket service. The server will immediately respond with ICE server credentials upon connecting to the WebSocket. Client <-- Server {
"Type": 2,
"From": "Server",
"Message": "{\"Username\":\"\",\"Password\":\"\",\"ExpirationInSeconds\":172799,\"TurnAuthServers\":[{\"Username\":\"\",\"Password\":\"\",\"Urls\":[\"stun:relay.communication.microsoft.com:3478\",\"turn:relay.communication.microsoft.com:3478\"]}]}"
}The client then responds with multiple requests which are documented below
Client --> Server {
"Message": "CONNECTREQUEST 9806856835729287287 v=0\r\no=- 5487540117316254753 2 IN IP4 127.0.0.1\r\ns=-\r\nt=0 0\r\na=group:BUNDLE 0\r\na=extmap-allow-mixed\r\na=msid-semantic: WMS\r\nm=application 9 UDP/DTLS/SCTP webrtc-datachannel\r\nc=IN IP4 0.0.0.0\r\na=ice-ufrag:GTsq\r\na=ice-pwd:<pass>\r\na=ice-options:trickle\r\na=fingerprint:<fingerprint>\r\na=setup:actpass\r\na=mid:0\r\na=sctp-port:5000\r\na=max-message-size:262144\r\n",
"To": 15778080650974306186,
"Type": 1
}Client --> Server {
"Message": "CANDIDATEADD 9806856835729287287 candidate:4152947846 1 udp 2122265344 <IPV6> 49370 typ host generation 0 ufrag GTsq network-id 4 network-cost 10",
"To": 15778080650974306186,
"Type": 1
}Client --> Server {
"Message": "CANDIDATEADD 9806856835729287287 candidate:3061656305 1 udp 2122199808 <IPV6> 49371 typ host generation 0 ufrag GTsq network-id 5 network-cost 10",
"To": 15778080650974306186,
"Type": 1
}Client --> Server {
"Message": "CANDIDATEADD 9806856835729287287 candidate:2426374300 1 udp 2122134272 <IPV6> 49372 typ host generation 0 ufrag GTsq network-id 8 network-cost 10",
"To": 15778080650974306186,
"Type": 1
}Client --> Server
{
"Message": "CANDIDATEADD 9806856835729287287 candidate:737026903 1 udp 41558015 20.202.1.9 52312 typ relay raddr <public-IPV4> rport 49375 generation 0 ufrag GTsq network-id 1 network-cost 10",
"To": 15778080650974306186,
"Type": 1
}After this we then receive the response from the server and 5 other CANNIDATEADD responses, I've only documented the first response and the first CANNIDATEADD response below: Client <-- Server {
"Type": 1,
"From": "15778080650974306186",
"Message": "CONNECTRESPONSE 9806856835729287287 v=0\r\no=- 6021002969452554251 2 IN IP4 127.0.0.1\r\ns=-\r\nt=0 0\r\na=group:BUNDLE 0\r\na=extmap-allow-mixed\r\na=msid-semantic: WMS\r\nm=application 9 UDP/DTLS/SCTP webrtc-datachannel\r\nc=IN IP4 0.0.0.0\r\na=ice-ufrag:ji75\r\na=ice-pwd:<pass>\r\na=ice-options:trickle\r\na=fingerprint:<fingerprint>\r\na=setup:active\r\na=mid:0\r\na=sctp-port:5000\r\na=max-message-size:262144\r\n"
}Client <-- Server {
"Type": 1,
"From": "15778080650974306186",
"Message": "CANDIDATEADD 9806856835729287287 candidate:148451967 1 udp 2122129151 <local_world_ipv4> 51125 typ host generation 0 ufrag ji75 network-id 1 network-cost 10"
}In the above response was the local IP for the world that was hosted on my mobile which allowed me to connect to the sesssion. I haven't been able to see what joining a session hosted outside of my local network would look like but I imagine it'd be a fairly similar process. The part that needs working on is the connection within the signalling channel, sending the correct SDP request to the host to receive the public IP. I haven't been able to spend much more time on this but if anyone's well experienced with this protocol please chime in. |
|
Thanks for the write up. I don't have experience with WebRTC, so I can't comment much on this protocol. Do you think there is a difference in implementation here as opposed to other WebRTC peer-to-peer application protocols? I'd bet that they did not invent anything here, more likely it seems they are using some existing libraries to facilitate the communication, https://github.com/zenomt/rtmfp-cpp/tree/main seems interesting and maybe related. As for the 20 digit integer: if you can, route the domain to a local WebSocket server that acts as a proxy and try erasing this field when forwarding request to the remote server and check if the peer to peer connection still works or not. That will give an idea on the significance/randomness of the ID. As for the session description Message's, this should be normal WebRTC comms, no? It should be possible to either use some lib to handle the protocol here or just replay vanilla messages with small modifications as necessary. |
|
No worries, thanks for jumping in on this. I'll have a look at figuring out the 20 digit int later tonight, upon looking at Mojangs telemetry they call this value In this scuffed POC we connect to the signalling channel, TURN/STUN servers and send the data between. I haven't been able to get this in a polished state yet. const WebSocket = require('ws');
const { RTCPeerConnection } = require('werift');
const { Authflow, Titles } = require('prismarine-auth')
// webrtcid
// 5696196935114111266
// ws
// https://signal.franchise.minecraft-services.net/ws/v1.0/signaling/15859775084651201335
//https://authorization.franchise.minecraft-services.net/api/v1.0/session/start
// post {"device":{"applicationType":"MinecraftPE","capabilities":["RayTracing"],"gameVersion":"1.20.51","id":"c1681ad3-415e-30cd-abd3-3b8f51e771d1","memory":"34185707520","platform":"Windows10","playFabTitleId":"20CA2","storePlatform":"uwp.store","treatmentOverrides":null,"type":"Windows10"},"user":{"language":"en","languageCode":"en-GB","regionCode":"GB","token":"","tokenType":"PlayFab"}}
const main = async () => {
// WORLDID 16920563543604857349
const ws = new WebSocket('wss://signal.franchise.minecraft-services.net/ws/v1.0/signaling/5301802620021606730', {
headers: { Authorization: `MCToken <token>` }
});
let resolve
const promise = new Promise((res) => {
resolve = res
})
ws.on('message', (data) => {
// Potentially need to set remoteDescription here
const message = JSON.parse(String(data));
console.log(message)
const payload = JSON.parse(message.Message);
resolve(payload);
})
const data = await promise
console.log(data)
const peer = new RTCPeerConnection({
iceServers: [
{
urls: 'stun:relay.communication.microsoft.com:3478',
credential: data.Password,
username: data.Username
},
{
urls: 'turn:relay.communication.microsoft.com:3478',
credential: data.Password,
username: data.Username
},
],
iceTransportPolicy: "all"
})
const dc = peer.createDataChannel('test')
peer.oniceconnectionstatechange = () => {
console.log(
"oniceconnectionstatechange",
peer.iceConnectionState
);
};
peer.onicecandidate = function (evt) {
console.log(evt)
if (evt.candidate) {
// Send the candidate to the other party via signaling channel
const message = `{
"Type": 1,
"To": 15341643931879376243,
"Message": "CANDIDATEADD 9806856835729287287 ${evt.candidate.candidate}"
}`
console.log(message)
ws.send(message)
}
};
const offer = await peer.createOffer()
console.log(offer)
const message = `{
"Type": 1,
"To": 15341643931879376243,
"Message": "CONNECTREQUEST 9806856835729287287 ${offer.sdp}"
}`
console.log(message)
ws.send(message)
await peer.setLocalDescription(offer)
}
main() |
|
If it's a consistent number, then it's either retrieved from the network or stored somewhere in the file system. In case of latter I'd think it could be a hash of the world ID. So not really retrievable, but doesn't sound important either. As for the implementation, you can ask ChatGPT to write some simple back and forth protocol code based on that |
|
So I just wanted to ask on how POST I've tried getting the data off it however the ip and port are empty strings just the same way as your response is however another friend has managed to directly get the IP and port off that request itself? What is the signaling and everything with the WS for? I'm a little lost but is that authentication? |
|
@zLevii It depends on the session you're connecting to, if the session is a server then it will contain the IP/port and connect using that, however if the connection is P2P then it will need to establish a connection via the signalling channel. It's briefly explained in my initial comment. |
|
Yeah it's P2P and it makes sense as to why I cannot connect with a direct IP/port. I would have to tell xbox to add me etc. What if I invite the account to the world? Can I skip the signaling etc? |
|
Inviting a player to the session adds them, allowing them to get the 'WebRTCNetworkId' from the connection details. You'd still need to establish a connection with the host via WebRTC. |
|
In the below example is a working script which establishes a connection with a peer via Minecraft's signalling channel and successfully returns the hosts connection details. https://gist.github.com/LucienHH/dab431394dabc38026ee1dd81cd0cdbc This returns 2 IPv6 addresses which the client then uses to connect to the remote peer, using DTLS v1.2. From signalling channel: Packets from wireshark I'm not sure that bedrock-protocol supports IPv6 and if that we'd need to maintain the connection through WebRTC? |
Makes sense. |
Interesting! I assume the gist you linked is everything after getting the WebRTC isn't it? I'll follow through that example and see how I get along. @rom1504 Is this something bedrock protocol could support? |
|
Why not |
For sure is possible to however we would need your help to chip in as this stuff is kind of beyond my understanding so it would be nice if the team could work alongside @LucienHH to implement this |
@extremeheat any thoughts on the protocol change when it's a peer to peer session, bit out of my realm of knowledge? |
|
Is all the normal minecraft game traffic over that DTLS protocol? Or is that just an intermediate step? Would need more information on that. |
|
All of the peer-to-peer session is over DTLS compared to connecting to a Realm / external server that's over Raknet |
|
I've written docs and done various tests with this a couple of months back. More info: |
Bedrock protocol supports joining realms however I assume there is no current support to join friend worlds.
It would be great if that could be added.
The text was updated successfully, but these errors were encountered: