Weekly Meeting 7/19/22

[madjin]

Agenda

• Utility of what this group can provide in short term
• Meeting times
  • Decided to leave as is (1600 UTC)
• Meet n greet
• Otherside litepaper discussion

[madjin]

Short term utility ideas:

• Actual case studies with 3-4 platform / engine participation
• Use popular / standard assets
  • https://github.com/KhronosGroup/glTF-Sample-Models/tree/master/2.0
  • https://github.com/madjin/interop-assets (WIP)
• Post sample scripts for the same thing in different platforms
  • Example: chair / pickupable objects
    • Chairs seem simple, but actually complex, good for demonstrating challenges
  • VRChat world SDK / Webaverse mount / Hubs anchor points / Second Life / OpenSim LSL scripts / Sinespace: Big bunch of example scripts for a VW platform / simple script generator for SL / Opensim
• How does the scripting sandbox sorta work? What are high level ideas we want these diff resources / components / gameobjects be defined? Focus on how do we move around an object, get a reference to the object, change its transform, create a cube creating mesh data, things like that. How do I do the things that glTF or basic game engine scene graph can do in sandbox form.
  • Adam: First is approaching platform from script that has wide view of the scene authorized to manipulate, other is create every object as island, allow some kinda RPC gotta signal script on another object to get

[madjin]

Adam: What scale of each script do we anticipate running in a scene? 10k or 10?
Small ones: time slicing / thousand scripts in same frame / keep moving counter forward

Humbletim: Got list of scripting use-cases? Seeing a real big challenge across platforms. Group in hubs had issues with textures loading in hubs. Assumptions about getting into scenes in stable predictable state, start with simplest use-cases, charting it out for getting effort to make things actually true with getting platform participation. Getting model + script can represent significant challenges.

jin: We should do a figma exercise with buncha platform logos and scope out effort to doing things between them as an exercise.
Robert: Gunna be implementing stuff, should get simple stuff filled out - basic stuff like create cube with physics enabled on it and make sure it falls.

Introspection: self check model, ask from script what is the scale, embed some kinda check logic - autodetect some kinda API

[madjin]

Notes from Otherside litepaper: https://otherside.xyz/litepaper#open-object-standards
Highlights

• Examples: tagging an object as a chair allows it to be sat on, tagging an object as a ball allows it to be picked up and thrown. Metadata tags will follow the ERC-721 metadata JSON standard.
• 3D Objects - We support 3D models through the popular glTF format (which includes meshes, materials, cameras, textures, animations, and everything you need to make a complete model). These objects can respond to in-engine lighting and physics.
• Universal scripting system: The current prototype uses JavaScript and is undergoing large-scale testing.
  
  they're using remixable templates (free) and in-game resources for crafting objects
  Jin: can make a github repo and mint some example testnet 3D objects with metadata tags like chair, ball, etc

[madjin]

I made a figma to start mapping out early case studies / short term projects for interoperable scripting across different platforms / engines: https://www.figma.com/file/WaMxYjv0avBIq1dnh3QMdC/OMI-Scripting-Group?node-id=0%3A1

[AdamFrisby]

My notes:

Decisions:
Time of Meeting: Reconfirmed as 1600 UTC.
Next Week Discussion: Developer Hour / Demos

Topics of Discussion:
General meta-discussion around the groups purpose; there was a general consensus that this group should be more 'deliverables focused' and less theory focused - with an emphasis on prototyping and demos.

Scope of early examples: it was discussed that a few scripts may be good for getting 'generalised scoping' of scripting runtimes, chairs were a particular point of discussion - however read-only access was also discussed as a possible debugging aid for things like glTF format failure (e.g. scale).

Small discussion of built-in components vs. external components (e.g. do we provide HTML-tag like building blocks, or do we assume everything is built through Javascript). Example libraries of scripts we can look at were posted into chat.

Robert is now at the point of needing to make API-design decisions and is interested in a conversation on those. Adam offered help to review any APIs that people want to bring to the table (done it before, cocked it up, learn from my failures please!)

This lead into a discussion of some of the basic scoping before the API can even be designed:

• How do we read a scene graph: do we provide universal access to the entire scene, or do we provide access to just 'an objects view of the world'; the former is better for complicated scripts, but the second provides a built-in RPC which can handle scenes distributed across multiple network nodes.
• What are our common constraints? Do we have a list (no), how many scripts do we reasonably expect to execute concurrently? is it 100, or is it 10,000?
• We should accommodate "bad" content too - e.g. 2 billion objects in a scene, how do we make sure the API is resistant to these extreme use cases?
• Guarding execution: how do we handle bad scripts, fuel v. permissions, quick discussion of ways of doing inter-script communication, and permission models.
• Micro v. Monolithic scripts: can we use shared libraries in WASM (no?), then how do we handle common use cases (e.g. calling a sin() function), what about more esoteric, but common ones (e.g. vector math).
• This lead into a discussion of the limitations of WASM, and WASI - which is going to be implementing Interface types.
• WASM bodies: do we have any good contacts with the WASM implementers (no.), suggestion we should join the WASM mailing lists / discord, and announce our presence once we have a bit more to show.
• Discussion around custom tooling: do we need custom tooling, can we get away with just the standard tooling? (mixed views shared)

[kettle11]

I enjoyed listening in today! I'll try to make some future meetings and jump on the mic. Some thoughts:

Guarding execution: how do we handle bad scripts, fuel v. permissions,

As far as I know Wasm fuel (also known as 'metering' or 'gas') works by pre-processing the code to count how many Wasm operations occur between control flow. Wherever a loop or function all occurs code is injected to make sure the Wasm hasn't exhausted its fuel and if it has then the Wasm crashes. This pre-processing can't be done when the user compiles the code because the user could cheat and remove the checks. But it can be done once when the user uploads a script. Adding those checks does have a significant performance penalty. It'd be better if there were a way to yield the script instead of crash, but that's probably more involved.

This unmaintained project describes its approach to Wasm metering: https://github.com/ewasm/wasm-metering

Compiling Wasmtime to Wasm (meta!) was raised as an idea but I don't think that can work because wasmtime's compiler backend currently compiles only to native assembly code (ARM, x86, etc.).

As mentioned by @robertlong another approach is to run all untrusted scripts on a worker thread (or web worker) and end the thread if it takes too long.

Micro v. Monolithic scripts: can we use shared libraries in WASM (no?), then how do we handle common use cases (e.g. calling a sin() function), what about more esoteric, but common ones (e.g. vector math

From a code size perspective I think it's OK if each script has its own basic math statically included, at least to start. Math code like that will be a very tiny subset of a total script's code and not a significant binary-size burden. Compiled Wasm tends to be quite small so this may not become an issue even when including larger libraries.

@AdamFrisby's mentioned an approach of exposing global "services". That feels like a pretty decent way to handle shared functionality.

On "Micro vs Monolithic": I prefer tiny scripts that are coupled to individual items. If someone authors a laser gun I'd love to be able to reuse that in other scenes, but if someone implements a monolithic "laser tag game" then that's impossible to reuse the pieces of. I think developers will drift towards monolithic scripts for ease and performance but if users enjoy building scenes from scripted objects that will incentives script authors to write more modular scripts.

---

The reason I decided to listen in today is because I've recently started (not sure how far I'll get) prototyping a basic scripting system similar to what was discussed. This is the architecture I'm aiming for:

• Tiny per-object Wasm scripts
• The host / script communicate with a single function call that passes messages. Each message has a target (either a specific Entity or the host, a message type ID, and optionally raw bytes.
• When a script is initialized it send messages to the host that request subscriptions to events like key presses or collisions.
• Scripts can ask the host / world to do things like move an object, play a sound, or update a mesh.

Notably this uses a really simple communication layer: just messages with a target, message type, and bytes. I'm writing a tiny Rust library to make this nicer to interact with but if other languages were to be supported it wouldn't be hard to write a similar tiny abstraction library.

For context this is what my test script looks like currently:

use verse_entity::*;

#[no_mangle]
pub extern "C" fn start() {
    verse_entity::set_message_handler(handle_message);
}

fn handle_message(world: &mut World, _sender: Entity, message: Message) {
    match message {
        Message::Setup => {
            let entity = world.spawn_entity().unwrap();
            world.move_entity(entity, [1.0, 2.0, 3.0]);
            world.subscribe_to_events(SubscriptionType::KeyEvents);
        }
        Message::KeyDown(key) => {
            /* TODO: Do something with this event */
        }
        _ => {}
    }
}

That script spawns an entity, moves it, and begins listening for key events. It's compiled with the regular Rust compiler. The above example ends up as 27kb after compilation with debug-info stripped / wasm-opt.

I'm still mulling over how to persist script state (Save the entire Wasm memory? Force scripts to save state in "properties"?)

I'm not sure how far I'll go with this prototype but I want to try to implement a few items that are fun and relatively simple:

• A gun that shoots balls

  • Spawn an entity
  • Either give it a velocity or move it per frame
  • Eventually despawn it

• A fountain or bubble machine
  This is trickier because it could be synced by just sharing the particle system's random seed, but a naive approach might sync every particle which would be too much.