Project 5 : Xiang Deng

README.md

@@ -1,28 +1,99 @@
 WebGL Deferred Shading
 ======================
 
-**University of Pennsylvania, CIS 565: GPU Programming and Architecture, Project 5**
+**University of Pennsylvania, CIS 565: GPU Programming and Architecture, Project 4** 
+
+* Xiang Deng
+* Tested on:  Windows 10-Home, i7-6700U @ 2.6GHz 16GB, GTX 1060 6GB (Personal Computer)
 
-* (TODO) YOUR NAME HERE
-* Tested on: (TODO) **Google Chrome 222.2** on
-  Windows 22, i7-2222 @ 2.22GHz 22GB, GTX 222 222MB (Moore 2222 Lab)
 
 ### Live Online
 
-[![](img/thumb.png)](http://TODO.github.io/Project5B-WebGL-Deferred-Shading)
+[![](img/thumb.JPG)](https://dengxianga.github.io/Project5-WebGL-Deferred-Shading-with-glTF)
+
+For this project we used GLSL and WebGL to implement a deferred shading pipeline. Features include:
+
+## Features
+
+* Ambient light and Blinn-Phong light shading
+* Scissor test
+* Sphere model
+* Bloom & Bloom plus
+* G-buffer optimization 
+
+
 
 ### Demo Video/GIF
 
-[![](img/video.png)](TODO)
+[![](img/scissor.gif) ] 
+
+[![](img/scissor2.gif) ] 
+
+
+
+Here is the comparison between toon shading and lambert light:
+
+With toon |  No toon 
+:-------------------------:|:-------------------------:  
+![](img/toon.JPG) | ![](img/notoon.JPG) 
+
+With toon |  No toon 
+:-------------------------:|:-------------------------:  
+![](img/toon2.JPG) | ![](img/notoon2.JPG) 
+
+Box bloom is also implemented; (just for fun) the right picture correspondings to 
+alpha blending that is based on the blooming effect, I think it looks pretty cool when I zoom out and
+visualize the whole building.
+
+With Bloom |  With Bloom&lava
+:-------------------------:|:-------------------------:  
+![](img/bloomnolava.JPG) | ![](img/bloomlava.JPG) 
+
+This is everything..
+
+[![](img/boom.JPG) ] 
+
+
+Two models were used for light rendering, scissor and sphere, here is the visualization of the Two
+in debug model
+
+With Scissor Debug |  With Sphere Debug
+:-------------------------:|:-------------------------:  
+![](img/scissordebug.JPG) | ![](img/spheredebug.JPG) 
+
+
+
+#debug images
+
+Depth |  Position | Geometric normal
+:-------------------------:|:-------------------------: |:-------------------------:  
+![](img/debug1.JPG) | ![](img/debug2.JPG) | ![](img/debug3.JPG) 
+
+Colormap |  Normalmap | Surface normal
+:-------------------------:|:-------------------------: |:-------------------------:  
+![](img/debug4.JPG) | ![](img/debug5.JPG) | ![](img/debug6.JPG) 
+
+# Performance Analysis 
+
+![](img/chart1.JPG) 
 
-### (TODO: Your README)
+For differnt rendering effect, I am analyzing the change of performance based on the render time / total time ratio, which is 
+extracted from Chrome profiling.  Here it shows 1. the sphere model consumes improves the costs by scissor test; 2. as number of 
+lights increases, the the percentage of the time consumed by GPU rendering increases for all effects; for non-debug mode, adding shading effects
+does not significantly increases the time; and apprarently rendering significantly increases the time than pure debugging visualization.
+The spherical proxy renders the natural range of lights and saved wasted fragements by scissor squares, this helps with the improvement.
+As number of lights increases, the GPU becomes more saturated and more dominant in the overll computation time percentage, this also suggests that optimization techniques
+such as compaction might help improve the efficiency. 
 
-*DO NOT* leave the README to the last minute! It is a crucial part of the
-project, and we will not be able to grade you without a good README.
+ 
+![](img/chart2.JPG) 
 
-This assignment has a considerable amount of performance analysis compared
-to implementation work. Complete the implementation early to leave time!
+![](img/chart3.JPG)
 
+For G-buffer optimization, since the normal, geometric normal and position buffers all have one free slot,
+the three components was squeezed into the free slots and saves one buffer. Even though we only reduced the number of 
+G-buffer from 4 to 3, we are still able to see the signifant improvments. (Here I am using FPS as measurements because the ms measure appears to be inacurrate.)
+improvement by reducing G-buffer size by one also suggests we look at memory optimization, if time permits, will try Tiling for better improvments.
 
 ### Credits
 

glsl/clear.frag.glsl

@@ -3,7 +3,7 @@
 precision highp float;
 precision highp int;
 
-#define NUM_GBUFFERS 4
+#define NUM_GBUFFERS 3
 
 void main() {
     for (int i = 0; i < NUM_GBUFFERS; i++) {

glsl/copy.frag.glsl

@@ -10,11 +10,31 @@ varying vec3 v_position;
 varying vec3 v_normal;
 varying vec2 v_uv;
 
+vec3 applyNormalMap(vec3 geomnor, vec3 normap) {
+    normap = normap * 2.0 - 1.0;
+    vec3 up = normalize(vec3(0.001, 1, 0.001));
+    vec3 surftan = normalize(cross(geomnor, up));
+    vec3 surfbinor = cross(geomnor, surftan);
+    return normap.y * surftan + normap.x * surfbinor + normap.z * geomnor;
+}
+
+
 void main() {
     // TODO: copy values into gl_FragData[0], [1], etc.
     // You can use the GLSL texture2D function to access the textures using
     // the UV in v_uv.
 
     // this gives you the idea
     // gl_FragData[0] = vec4( v_position, 1.0 );
+    vec4 colormap = texture2D(u_colmap,v_uv);
+    gl_FragData[0] = vec4(v_position,colormap.x);
+
+    vec3 normap = texture2D(u_normap,v_uv).rgb;  // The raw normal map (normals relative to the surface they're on)
+    //vec3 nor = applyNormalMap (v_normal, normap);     // The true normals as we want to light them - with the normal map applied to the geometry normals (applyNormalMap above)
+
+    //gl_FragData[1] =  vec4(v_normal, 1); 
+    gl_FragData[1] =  vec4(v_normal, colormap.y); 
+    //gl_FragData[2] = colormap; 
+    //gl_FragData[3] =  vec4(normap, 1); 
+    gl_FragData[2] =  vec4(normap, colormap.z); 
 }

glsl/deferred/ambient.frag.glsl

@@ -3,7 +3,7 @@
 precision highp float;
 precision highp int;
 
-#define NUM_GBUFFERS 4
+#define NUM_GBUFFERS 3
 
 uniform sampler2D u_gbufs[NUM_GBUFFERS];
 uniform sampler2D u_depth;
@@ -13,15 +13,16 @@ varying vec2 v_uv;
 void main() {
     vec4 gb0 = texture2D(u_gbufs[0], v_uv);
     vec4 gb1 = texture2D(u_gbufs[1], v_uv);
-    vec4 gb2 = texture2D(u_gbufs[2], v_uv);
-    vec4 gb3 = texture2D(u_gbufs[3], v_uv);
+    //vec4 gb2 = texture2D(u_gbufs[2], v_uv);
+    vec4 gb3 = texture2D(u_gbufs[2], v_uv);
+    vec3 colmap = vec3(gb0.w,gb1.w,gb3.w);
     float depth = texture2D(u_depth, v_uv).x;
     // TODO: Extract needed properties from the g-buffers into local variables
 
     if (depth == 1.0) {
-        gl_FragColor = vec4(0, 0, 0, 0); // set alpha to 0
+        gl_FragColor = vec4(0, 0, 0, 1); // set alpha to 0
         return;
     }
 
-    gl_FragColor = vec4(0.1, 0.1, 0.1, 1);  // TODO: replace this
+    gl_FragColor = vec4(colmap.rgb/5.0, 1);  // TODO: replace this
 }

glsl/deferred/blinnphong-pointlight.frag.glsl

@@ -2,14 +2,20 @@
 precision highp float;
 precision highp int;
 
-#define NUM_GBUFFERS 4
+//#define NUM_GBUFFERS 4
+#define NUM_GBUFFERS 3
 
 uniform vec3 u_lightCol;
 uniform vec3 u_lightPos;
 uniform float u_lightRad;
 uniform sampler2D u_gbufs[NUM_GBUFFERS];
 uniform sampler2D u_depth;
 
+uniform vec3 u_camPos;
+uniform vec3 u_effects;
+uniform float u_width;
+uniform float u_height;
+
 varying vec2 v_uv;
 
 vec3 applyNormalMap(vec3 geomnor, vec3 normap) {
@@ -23,9 +29,18 @@ vec3 applyNormalMap(vec3 geomnor, vec3 normap) {
 void main() {
     vec4 gb0 = texture2D(u_gbufs[0], v_uv);
     vec4 gb1 = texture2D(u_gbufs[1], v_uv);
-    vec4 gb2 = texture2D(u_gbufs[2], v_uv);
-    vec4 gb3 = texture2D(u_gbufs[3], v_uv);
+    //vec4 gb2 = texture2D(u_gbufs[2], v_uv);
+    vec4 gb3 = texture2D(u_gbufs[2], v_uv);
     float depth = texture2D(u_depth, v_uv).x;
+
+    vec3 pos = gb0.xyz;     // World-space position
+    vec3 geomnor = gb1.xyz;  // Normals of the geometry as defined, without normal mapping
+    //vec3 colmap = gb2.rgb;  // The color map - unlit "albedo" (surface color)
+    vec3 normap = gb3.xyz;  // The raw normal map (normals relative to the surface they're on)
+
+    vec3 colmap = vec3(gb0.w,gb1.w,gb3.w);
+    vec3 nor = applyNormalMap (geomnor, normap);     // The true normals as we want to light them - with the normal map applied to the geometry normals (applyNormalMap above)
+
     // TODO: Extract needed properties from the g-buffers into local variables
 
     // If nothing was rendered to this pixel, set alpha to 0 so that the
@@ -35,5 +50,63 @@ void main() {
         return;
     }
 
-    gl_FragColor = vec4(0, 0, 1, 1);  // TODO: perform lighting calculations
+    float enableToon = u_effects.r;
+    if (enableToon == 1.0){
+        vec2 dx = vec2(1.0 / u_width,0.0);
+        vec2 dy = vec2(0.0, 1.0 / u_height);
+        float diff = abs(texture2D(u_depth, v_uv + dx).r - depth)
+                    +abs(texture2D(u_depth, v_uv + dy).r - depth)
+                    +abs(texture2D(u_depth, v_uv - dx).r - depth)
+                    +abs(texture2D(u_depth, v_uv - dy).r - depth);
+        if (diff > 0.01){
+            gl_FragColor = vec4(-1.0, -1.0, -1.0, 1.0);
+            return;
+        }
+    }
+
+    float dist_from_surface_to_light = length(u_lightPos-pos);
+    float attenuation = max(0.0, u_lightRad - dist_from_surface_to_light);
+
+    //diffuse section:
+    vec3 thiscolor = colmap.rgb*u_lightCol;
+
+    vec3 lightdir = normalize(u_lightPos-pos);
+    float diffusefact=dot(nor,lightdir);
+
+
+
+    if ( enableToon==1.0){
+        float diffusefact2=   abs(diffusefact) ;
+        if (diffusefact2>0.75){
+            thiscolor *= 1.0;
+        }
+        else if (diffusefact2>0.5){
+            thiscolor *= 0.5;
+        }
+        else if (diffusefact2>0.05){
+            thiscolor *= 0.25;
+        }
+        else{
+            thiscolor *= 0.1;
+        }
+        thiscolor *= sign(diffusefact)*attenuation*0.35;
+    }
+    else{
+        thiscolor *= attenuation*diffusefact*0.35;
+    }
+
+
+
+
+    //fill here
+
+    //specular
+    vec3 camdir = normalize(u_camPos-pos);
+    vec3 tmp = normalize(lightdir+camdir);
+    float specularfact = dot(nor,tmp);
+    thiscolor += colmap * specularfact * attenuation * 0.15;
+
+    gl_FragColor = vec4(thiscolor,1);
+
+
 }

glsl/deferred/bloom.frag.glsl

@@ -0,0 +1,50 @@
+#version 100
+precision highp float;
+precision highp int;
+
+#define sigma  10
+#define alpha  0.08
+#define cutoff 0.5
+
+uniform float u_width;
+uniform float u_height;
+
+uniform sampler2D u_color;
+
+uniform int u_whichway;
+
+
+varying vec2 v_uv;
+
+
+
+
+void main(){
+    vec4 thiscolor = texture2D(u_color, v_uv).rgba;
+
+    if (thiscolor.x == 0.0){
+        gl_FragColor = vec4(0, 0, 0, 0);
+        return;
+    }
+    float dx = 1.0/ u_width;
+    float dy = 1.0 / u_height;
+
+    vec2 dd = vec2(0.0, 0.0);
+    if (u_whichway==1){ // define dy as 1
+        dd = vec2(0.0, dy);
+    }
+    else if (u_whichway ==0){
+        dd = vec2(dx, 0.0);
+    }
+
+    vec3 result = vec3(0.0, 0.0, 0.0);
+    vec3 thres3 = vec3(cutoff);
+    for (int i= -sigma; i<=sigma; i++){
+        vec3 tempcol= texture2D(u_color, v_uv + dd).rgb;
+        result += max(tempcol-thres3, 0.0) *alpha;
+    }
+
+    gl_FragColor = vec4(result,1.0);
+}
+
+

glsl/deferred/debug.frag.glsl

@@ -2,7 +2,8 @@
 precision highp float;
 precision highp int;
 
-#define NUM_GBUFFERS 4
+//#define NUM_GBUFFERS 4
+#define NUM_GBUFFERS 3
 
 uniform int u_debug;
 uniform sampler2D u_gbufs[NUM_GBUFFERS];
@@ -23,30 +24,33 @@ vec3 applyNormalMap(vec3 geomnor, vec3 normap) {
 void main() {
     vec4 gb0 = texture2D(u_gbufs[0], v_uv);
     vec4 gb1 = texture2D(u_gbufs[1], v_uv);
-    vec4 gb2 = texture2D(u_gbufs[2], v_uv);
-    vec4 gb3 = texture2D(u_gbufs[3], v_uv);
+    //vec4 gb2 = texture2D(u_gbufs[2], v_uv);
+    vec4 gb3 = texture2D(u_gbufs[2], v_uv);
+
+    vec3 colmap = vec3(gb0.w,gb1.w,gb3.w);
+
     float depth = texture2D(u_depth, v_uv).x;
     // TODO: Extract needed properties from the g-buffers into local variables
     // These definitions are suggested for starting out, but you will probably want to change them.
     vec3 pos = gb0.xyz;     // World-space position
     vec3 geomnor = gb1.xyz;  // Normals of the geometry as defined, without normal mapping
-    vec3 colmap = gb2.rgb;  // The color map - unlit "albedo" (surface color)
+    //vec3 colmap = gb2.rgb;  // The color map - unlit "albedo" (surface color)
     vec3 normap = gb3.xyz;  // The raw normal map (normals relative to the surface they're on)
     vec3 nor = applyNormalMap (geomnor, normap);     // The true normals as we want to light them - with the normal map applied to the geometry normals (applyNormalMap above)
 
     // TODO: uncomment
     if (u_debug == 0) {
         gl_FragColor = vec4(vec3(depth), 1.0);
     } else if (u_debug == 1) {
-        // gl_FragColor = vec4(abs(pos) * 0.1, 1.0);
+         gl_FragColor = vec4(abs(pos) * 0.1, 1.0);
     } else if (u_debug == 2) {
-        // gl_FragColor = vec4(abs(geomnor), 1.0);
+         gl_FragColor = vec4(abs(geomnor), 1.0);
     } else if (u_debug == 3) {
-        // gl_FragColor = vec4(colmap, 1.0);
+         gl_FragColor = vec4(colmap, 1.0);
     } else if (u_debug == 4) {
-        // gl_FragColor = vec4(normap, 1.0);
+         gl_FragColor = vec4(normap, 1.0);
     } else if (u_debug == 5) {
-        // gl_FragColor = vec4(abs(nor), 1.0);
+         gl_FragColor = vec4(abs(nor), 1.0);
     } else {
         gl_FragColor = vec4(1, 0, 1, 1);
     }

glsl/red.frag.glsl

@@ -3,5 +3,5 @@ precision highp float;
 precision highp int;
 
 void main() {
-    gl_FragColor = vec4(1, 0, 0, 1);
+    gl_FragColor = vec4(1, 0, 0, 0.5);
 }