Introductions and Planning Our First Meeting

[robertlong]

Welcome, we're just getting started! This group is intended to explore the realm of adding support for scripting for user generated content (UGC) in "metaverse" applications. What all that means is still yet to be defined.

In this discussion we'd like to hear from you, what are your needs for interactive user generated content in metaverse worlds or other applications of a WASM API for 3D content? We'd also like to hear what your availability is like for the first two weeks of January. Let us know your timezone and what times you are generally available during the week.

OMI Background

For those of you who are new to the Open Metaverse Interoperability Group (OMI), here's a basic summary of the organization.

We are a community-led organization focused on bridging virtual worlds by designing and promoting protocols for identity, social graphs, inventory, and more. Our members include businesses and individuals working towards this common goal. In practice this has led to members creating self-organized groups, such as this one or the glTF Extensions Subgroup where we can openly discuss interoperability projects. These projects aren't all technical projects and also include things like content discovery / metaverse mapping.

So far the glTF Extensions Subgroup is our only standards-based group. It operates under our W3C Community Group where members must first sign the W3C Contributor License Agreement in order to participate. This assures that intellectual property is not lost to participants or third parties and can remain open for all of us to benefit from.

OMI has weekly general meetings every Wednesday at 9:00 AM Pacific Time in this Discord server. This meeting is open to the public and topics vary week to week. You can find the past agendas / notes here.

WASM UGC Scripting Background

I originally proposed this group in the Third Room GitHub Discussions as a Entity Component System specific API. It immediately got some attention from various parties and it became clear that parties that weren't using ECS were interested in WASM UGC Scripting.

This led to the proposal in OMI for this group which we voted unanimously to establish. At this point we have a number of interested parties and need to gather goals/requirements from those parties. We'll do our best to invite people from a broad set of backgrounds to participate. In this discussion we can collect those goals/requirements as well as introductions. Then, in our first meeting, we can go over this thread and start forming a more complete picture of what is missing in this space and what we can do to fill that gap.

I'm looking forward to working with you all! Thanks and hope to talk to you all soon!

[KooIaIa]

Hello everyone. My interest is in creating a WASM UGC programming language for creating WASM UGC programming languages. I have a pretty good grasp how to do it but love talking about it and am open to suggestions or new ideas. The 'metaverse need' I am trying to solve is that XR doesn't have any 'real' UGC programming languages yet that are real languages. I'm not sure how to define what's 'real' or a language succinctly but the dim view is that virtual programing languages in every example today are bespoke and tied to specific engines which means your content can't leave the world its created in. I'm excited for this group to address that problem of WASM content interoperability.

WASM Interoperability wise, I am very interested in the 'module-linking' component of the new WASM Component Model

The module-linking proposal will initialize the Component Model specification, adding the ability for WebAssembly to import, nest, instantiate and link multiple Core WebAssembly modules without host-specific support.

It is very hard to find good information on this new model so I would love some help with learning it and teaching others. According to this interview the WASI folks should be explaining it simply soon. Right now so much WASI stuff seems undefined and hard to wrangle on my own. With things in the air I feel like a group wanting to make new interoperability patterns for WASM would need to become current in this field and I'm excited if OMI could feel connected to the WASM/WASI developer network and the state of the art.

I am EST USA and free typically after work at 5:30 PM EST.

[robertlong]

My name is Robert Long. I'm a software engineer at Element working on decentralized communications and leading a project called Third Room which aims to build a decentralized metaverse platform on top of Matrix, the decentralized communications protocol. At AltspaceVR I worked on the SDK team, helping build out their platform for user-created worlds. At Mozilla Hubs, I helped create their glTF pipeline and the Mozilla Spoke editor. One of the biggest requests we received while I was on the team was for scripting support. Now on Third Room, I hope we can create a platform for people to create dynamic worlds with programmable objects, scenes, and avatars.

My goals for this project would be to create a lightweight and extensible API that enables dynamic UGC. Ideally it would have a relatively small API footprint which will hopefully make implementing it easier. It would be great if we could integrate this API into existing Rust and C++ libraries such that you can cross compile certain projects to run in this WASM sandbox, but that's probably going to be a lot to ask.

More realistically, I see this as an opportunity to create something of a standard "mod" SDK. In that any application could embed this API to support a common set of interactive 3D content. In my use case, I'd like to embed these scripts in a glTF/glb file to make self-contained portable assets.

Here are some samples of interactive content I'd like to enable:

• Reading / modifying transform properties
• Collision / user input triggers
• Dynamically modifying material properties
• Controlling animations and media playback
• Spawning / despawning objects
• Applying physics properties (velocity, friction, gravity, etc.)
• Raycasting
• Networked properties

This is a lot of stuff and clients may not want all of it. What I'd like to determine is if it is possible to implement this in a modular way, where clients can opt-in / out of various models. I'd also like to figure out a permissions model for UGC to ask for things like raycasting access or head transform, etc.

What may come out of this is a need for a much more opinionated and specialized API such as the web's DOM APIs. Perhaps a generalized mod API can't be designed and instead we should be talking about basically a new "metaverse browser" API.

Or perhaps we can carve out a generalized 3D content API and the "metaverse" specific APIs can be split into a separate module, making interactive glTF content separate from the inclusion of APIs for defining an immersive world.

Whatever path we take, I'm excited to explore this problem space with you all!

I'm in PST and free Monday afternoons, Tuesday mornings / afternoons, all day Wednesdays besides our weekly OMI meeting time, Thursday late mornings / afternoons, and Friday late mornings. I work for a London-based company so mornings are a bit tricky for me as that is the time for me to meet with my co-workers. If we choose a morning time, after 10AM is best for me.

[ashconnell]

Is there a particular reason that WASM is part of the requirements here?

I’ve been using an implementation of SES (Secure EcmaScript) to compartmentalise UGC with great success. One major benefit of this is that everything runs in the host process, which provides much more capability than a WASM bridge API would ever be able to do, such as using React.

If interested, you can see how I’m using this here: https://twitter.com/ashconnell/status/1470218044681031681?s=21

[cdata]

This seems off-topic for this issue. There is a good discussion to be had about the topic you are raising, but perhaps consider filing a dedicated top-level issue for it?

[ashconnell]

Good point: #6 (comment)

[robertlong]

Whoops, I created a whole discussion on it: #7

[cdata]

Thanks for starting this discussion @robertlong !

Introduction / background

I'm a software engineer who is building metaverse stuff. I previously worked at Google where I did some work on Chrome and WebXR. I created the open source <model-viewer> project used by a lot of websites to display glTFs: http://modelviewer.dev/. Now I'm off on my own, hoping to contribute to the ecosystem in a more self-directed fashion.

I've also spent some time participating in Khronos glTF and 3D Commerce groups, and I chaired the 3D Commerce Certification working group for some time.

Use case

A few years ago, I set out to build a runtime scene graph API to support interactive use cases for 3D models loaded by <model-viewer>. We never fully realized its potential, but it is notable in that it demonstrates how business logic can be shipped alongside a model to enable interactivity in both inline and immersive contexts without changes.

This work and interactions with <model-viewer>'s users in industry has left me with the belief that a sadboxed runtime environment that operates on 3D scenes is inevitable. Here is what I posted on the topic elsewhere in this Github org:

An ubiquitous portable, sandboxed runtime environment for interactive content seems inevitable. My main uncertainty is whether we can agree on the shape of such an environment as a community, or if one will end up being thrust upon us unilaterally from [pick your source].

WASM gives us a convenient substrate for experimentation and discovery of a plausible standard. I'm interested in investigating its strengths and weaknesses - and the opportunities it presents - with this group.

There are a lot of use cases that fall out from the umbrella of "3D content interactivity," but to me the most powerful thing worth reaching for is content portability. Content portability means that as a content author, I can ship business logic around to different runtime environments the same way that I ship a 3D model asset: I shouldn't have to worry too much about each environment's internal details. Content portability will be a corner stone of a web-like metaverse, which is what I aim to help build. I hope to learn from this group where the "sweet spot" is for a portable format that may not be ideal at the extremes but can hit at least 80% or so of the use cases.

Availability

My timezone is AMER / PST.
I have a fairly flexible schedule right now, and I'm happy to meet at very early or very late times if it helps to accomodate EMEA or APAC participants. Middle of the day would work for me too.

[RangerMauve]

Hey folks, I'm Mauve and I mostly do consulting on decentralized technologies like IPFS and Hypercore-Protocol.

One of the projects I'm working is HyperGodot which is a set of libraries for doing Peer to Peer stuff in the Godot game engine.

Our project is expanding and we're now also looking at making it easy to build virtual worlds on top by using peer to peer multiplayer and loading/uploading GLTF objects from peer to peer archives.

We're interested in making the GLTF objects scriptable, and WASM seems like a good bet for making those scripts cross-platform.

At the moment, we're mostly interested in basic stuff like triggering animations in the GLTF and swapping out materials (with other materials in the gltf). Though having the engine notify the script of stuff like collission detection and ray intersection is going to be important for triggers and might be hard to detect.

Longer term it'd be cool to provide APIs along the lines of WASI to do graphics calls to render to textures and some basic file/network IO using URLs (e.g. making HTTP requests to URLs to load and save data).

We're still scoping out what we want, and don't have anything existing constraining us, so it'd be cool to co-create the interfaces as we're integrating them into our code.

Regarding Availability, I'm generally around between 11:00 EST and 23:00 EST, I'm busy on the 3rd and 5th, and on 7/14th I have meetings on Friday at 13:00 EST.

[AdamFrisby]

Hi all, I think it's clear there's plenty of interest in this, I'd like to propose doing the first meeting sometime after the tenth just to allow everyone to get back from their holidays. Would the morning of the 12th work for people, in a similar time to the normal meeting?

…

________________________________ From: RangerMauve ***@***.***> Sent: Wednesday, December 29, 2021 7:27:31 AM To: omigroup/omi-wasm-group ***@***.***> Cc: Subscribed ***@***.***> Subject: Re: [omigroup/omi-wasm-group] Introductions and Planning Our First Meeting (Discussion #4) Hey folks, I'm Mauve and I mostly do consulting on decentralized technologies like IPFS and Hypercore-Protocol. One of the projects I'm working is HyperGodot<https://github.com/HyperGodot> which is a set of libraries for doing Peer to Peer stuff in the Godot game engine. Our project is expanding and we're now also looking at making it easy to build virtual worlds on top by using peer to peer multiplayer and loading/uploading GLTF objects from peer to peer archives. We're interested in making the GLTF objects scriptable, and WASM seems like a good bet for making those scripts cross-platform. At the moment, we're mostly interested in basic stuff like triggering animations in the GLTF and swapping out materials (with other materials in the gltf). Though having the engine notify the script of stuff like collission detection and ray intersection is going to be important for triggers and might be hard to detect. Longer term it'd be cool to provide APIs along the lines of WASI to do graphics calls to render to textures and some basic file/network IO using URLs (e.g. making HTTP requests to URLs to load and save data). We're still scoping out what we want, and don't have anything existing constraining us, so it'd be cool to co-create the interfaces as we're integrating them into our code. Regarding Availability, I'm generally around between 11:00 EST and 23:00 EST, I'm busy on the 3rd and 5th, and on 7/14th I have meetings on Friday at 13:00 EST. — Reply to this email directly, view it on GitHub<#4 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AAA326L5FBALDERE5J6BJ5LUTIMTHANCNFSM5KTI3SQQ>. Triage notifications on the go with GitHub Mobile for iOS<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[robertlong]

@AdamFrisby @cdata @RangerMauve @KooIaIa is there any way we could do the meeting on January 11th at 9:00 AM PST instead? I'd like to have a little more buffer time afterwards and not have to worry about going into our weekly meeting.

[RangerMauve]

11th works fine for me 👍

[cdata]

That date and time works for me

[AdamFrisby]

Works for me too

…

________________________________ From: Christopher Joel ***@***.***> Sent: Wednesday, January 5, 2022 10:04:03 AM To: omigroup/omi-wasm-group ***@***.***> Cc: Adam Frisby ***@***.***>; Mention ***@***.***> Subject: Re: [omigroup/omi-wasm-group] Introductions and Planning Our First Meeting (Discussion #4) That date and time works for me — Reply to this email directly, view it on GitHub<#4 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AAA326IRIFXH24XH2UD7NK3UUN4GHANCNFSM5KTI3SQQ>. Triage notifications on the go with GitHub Mobile for iOS<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you were mentioned.Message ID: ***@***.***>