From e7014ad7e9f065b53375f73e7bb08f4b13f63a49 Mon Sep 17 00:00:00 2001
From: Mendy Berger <12537668+MendyBerger@users.noreply.github.com>
Date: Wed, 16 Oct 2024 13:02:25 -0400
Subject: [PATCH] Fill in full design explanation in README.md

---
 README.md | 41 ++++++++++++++++++++++++++++++++++++++---
 1 file changed, 38 insertions(+), 3 deletions(-)

diff --git a/README.md b/README.md
index 9d74954..b46a42f 100644
--- a/README.md
+++ b/README.md
@@ -47,17 +47,52 @@ This proposal is a collection of multiple packages.
 
 ### Goals [or Motivating Use Cases, or Scenarios]
 
-[What is the end-user need which this project aims to address?]
+- Bring some of the benefits that Wasm provides to CPU code, over to GPU code. 
+- Enable secure GPU compute.
+- Enable wasm-based UI applications.
+- Enable wasm-based graphics even in environments with no GPUs.
+- Enable wasm-based AI for cases not covered by wasi-nn.
 
 ### Non-goals
 
-[If there are "adjacent" goals which may appear to be in scope but aren't, enumerate them here. This section may be fleshed out as your design progresses and you encounter necessary technical and other trade-offs.]
+- A full windowing API.
+- VR/AR (subject to change, if we find contributors with expertise in the field).
 
 ### API walk-through
 
 The full API documentation can be found in [imports.md](imports.md).
 
-[Walk through of how someone would use this API.]
+#### `wasi:webgpu`
+
+`wasi:webgpu` is based on the official [webgpu spec](https://www.w3.org/TR/webgpu/).
+
+`wasi:webgpu` does deviate sometimes from the webgpu spec, namely, in cases where the spec makes assumptions about running in a web or JS environment. Wherever `wasi:webgpu` deviates from the spec, a clear explanation should be documented.
+
+#### `wasi:surface`
+
+`wasi:surface` is a basic surface API. It lets you create a surface you can draw to (similar to canvas on web). You can also get basic user events from the surface, like pointer and keyboard events.
+
+`wasi:surface` supports the following event types:
+- Pointer: For mouse/touch/pencil/etc.
+- Key: For keyboard events.
+- Resize: Surface was resized.
+- Frame: Fires on each frame. (similar to [`requestAnimationFrame`](https://developer.mozilla.org/en-US/docs/Web/API/window/requestAnimationFrame) on the web)
+
+Other user input, such as scroll wheel, clipboard, drag-over, device motion, device orientation, gamepad input, etc., are all useful in many applications, but since they're a bit less common, they're out of scope for now. However, they might be added eventually.
+
+#### `wasi:frame-buffer`
+
+A simple frame buffer API for CPU-based rendering.
+
+This might be especially useful in embedded.
+
+#### `wasi:graphics-context`
+
+`wasi:graphics-context` is the point of connection between a graphics API (`wasi:webgpu`, `wasi:frame-buffer`, future graphics API) to a windowing system (`wasi:surface`, future windowing system). See diagram.
+
+![image](https://github.com/user-attachments/assets/55bcc436-7bcd-4f02-9090-888106b889ef)
+
+`wasi:graphics-context` has a method to get the next frame as an `abstract-buffer`. Abstract buffer's contents can't be read. Instead, the graphics API in use can turn the `abstract-buffer` into something it can make use of. E.g. webgpu is able to turn the buffer into a `GPUTexture`. The runtime can easily, in the background, represent the `abstract-buffer` as a `GPUTexture`, and the conversion would just be a no-op.
 
 #### [Use case 1]