Use add_mount to store resources (#9)

* WIP Use add_mount

* Add `wipe_on_start` option

* WIP Mounts #2

* Update sh script

* Remove miniz

* Use `add_mount` to store resources

.defignore

@@ -1 +1,3 @@
 /build2
+/build-v1
+/build-alt

README.md

@@ -12,9 +12,8 @@ You can do the following things:
 
 Defold has the [Live Update](https://defold.com/manuals/live-update/) feature that we can use to implement these ideas, and the project aims to demonstrate the usage of it. The project contains:
 
-1. The Lua `liveupdate_reszip.reszip` module that downloads (with progress!) and extracts the missing resources.
-2. [The magic JS code](liveupdate_reszip/manifests/web/engine_template.html) that removes Live Update cache from IndexedDB before the start of your game. This step is based on our experience of Live Update usage in Defold. Defold often fails to use the mix of resources from different versions of the game and the best solution is to clear its cache on every launch.
-3. The Bash script (`example_build_script.sh`) shows you how to automatically build your game for the web and move the `resources.zip` file to your build result folder.
+- The Lua `liveupdate_reszip.reszip` module that downloads (with progress!) and mounts the missing resources.
+- The Bash script (`example_build_script.sh`) shows you how to automatically build your game for the web and move the `resources.zip` file to your build result folder.
 
 Check out the online demos:
 1. [**Demo 1**](https://indiesoftby.github.io/defold-liveupdate-reszip/latest/index.html) - this project. **Tap anywhere to load level 2.**
@@ -26,6 +25,7 @@ Check out the online demos:
 
 | Asset Version   | Defold Version | Status        |
 | --------------- | -------------- | ------------- |
+| 1.4.0           | 1.6.1          | Tested ✅     |
 | 1.3.0           | 1.5.0          | Tested ✅     |
 | 1.2.0           | 1.4.7-8        | Tested ✅     |
 | 1.2.0           | 1.4.6          | Doesn't work ❌ |
@@ -54,42 +54,55 @@ https://github.com/indiesoftby/defold-liveupdate-reszip/archive/main.zip
 2. Follow the [Live Update tutorial](https://defold.com/manuals/live-update/) on the Defold website and exclude chosen collections in proxies.
 3. Use the `Zip` mode for Live Update and publish the Live Update content through `Project / Bundle...` or using `bob.jar` (the arg is `--liveupdate yes`).
 4. Move the resulting .zip file with resources into your production build folder.
-5. Look at the `example/main.script` to learn how to check for the missing resources and how to load them from the `.zip` resources file.
+5. Look at the `example/main.script` to learn how to check for the missing resources and how to mount the `.zip` resources file:
+
+```lua
+local zip_filename = sys.get_config("liveupdate_reszip.filename", "resources.zip")
+local zip_file_location = (html5 and zip_filename) or ("http://localhost:8080/" .. zip_filename)
+local excluded_proxy_url = "/level2#collectionproxy"
+
+local missing_resources = collectionproxy.missing_resources(excluded_proxy_url)
+if next(missing_resources) ~= nil then
+    print("Some resources are missing, so download and mount the resources archive...")
+    assert(liveupdate, "`liveupdate` module is missing.")
+
+    reszip.load_and_mount_zip(zip_file_location, {
+        on_finish = finish_handler,
+        on_progress = http_loading_progress_handler
+    })
+else
+    -- All resources exist, so load the level:
+    print("Resources are already loaded. Let's load level 2!")
+    msg.post(excluded_proxy_url, hash("load"))
+end
+```
 
 ### Tips
 
 The easiest way to use ResZip in your project is to move some of your audio files (i.e. sound components) to a proxied collection and exclude the collection for the release build. To play these sounds, you should make an external script that acts as a sound manager of all your in-game audio and knows when proxied sounds are loaded from the `resources.zip` file.
 
-Also, you can remove an unused manifest from the `resources.zip` file to reduce its size: 
-
-```bash
-7z d -r resources.zip liveupdate.game.dmanifest
-```
-
 ### Advanced Usage
 
-ResZip can start preloading the `resources.zip` file as soon as game loading is finished. It's highly recommended to enable this option:
+#### Preload resources
+
+ResZip can start preloading the `resources.zip` file as soon as game loading is finished. It's highly recommended to enable this option because the engine initialisation takes some time, during which we can already start loading resources:
 
 ```ini
 [liveupdate_reszip]
 preload_file = your_resources_file_name.zip
 ```
 
-#### Deprecated options
-
-~~If the `resources.zip` file contains hundreds or thousands of resources, you can speed up the process of loading resources by enabling batching (only for HTML5!):~~
+If you suspect a bug in Live Update or ResZip, the following option can be used to force Live Update data to be completely cleared before the game starts:
 
-```lua
-reszip.RESOURCES_PER_BATCH = 10
-reszip.BATCH_MAX_TIME = 0 -- Seconds. Set 0 or less to disable.
+```ini
+[liveupdate_reszip]
+wipe_on_start = 1
 ```
 
 ## Credits
 
 This project is licensed under the terms of the CC0 1.0 Universal license. It's developed and supported by [@aglitchman](https://github.com/aglitchman). 
 
-Also, the project uses [miniz](https://github.com/richgel999/miniz), a MIT-licensed data compression library.
-
 The demo contains third-party music files which require attribution:
 ```
 Ethernight Club by Kevin MacLeod

dev_build_web.sh

@@ -1,16 +1,17 @@
 #!/bin/bash
 
 # Download bob.jar like that:
-# wget "https://github.com/defold/defold/releases/download/1.4.6/bob.jar" -O bob.jar
+# wget "https://github.com/defold/defold/releases/download/1.6.1/bob.jar" -O bob.jar
 
 # Plus, the script uses https://www.npmjs.com/package/http-server to serve local files.
 
 set -e
 
-PLATFORM=js-web
+PLATFORM=wasm-web
 
 rm -rf build
 mkdir -p build/public
-java -jar bob.jar --email [email protected] --auth 123 --texture-compression true --bundle-output build/bundle/${PLATFORM} --build-report-html build/public/build_report_latest.html --platform ${PLATFORM} --archive --liveupdate yes --variant debug resolve build bundle
+java -jar bob.jar --email [email protected] --auth 123 --texture-compression true --bundle-output build/bundle/${PLATFORM} --build-report-html build/public/build_report_latest.html --platform ${PLATFORM} --architectures ${PLATFORM} --archive --liveupdate yes --variant debug resolve build bundle
 mv build/liveupdate_output/*.zip build/bundle/${PLATFORM}/liveupdate_reszip_demo/resources.zip
-(cd build/bundle/${PLATFORM}/liveupdate_reszip_demo/ && http-server -c-)
+# (cd build/bundle/${PLATFORM}/liveupdate_reszip_demo/ && http-server -c-)
+http-server -c-

example/main.script

@@ -9,6 +9,17 @@ local ZIP_FILE_LOCATION = (html5 and ZIP_FILENAME) or ("http://localhost:8080/"
 local EXCLUDED_PROXY_URL = "/level2#collectionproxy"
 local LEVEL1_PROXY_URL = "/level1#collectionproxy"
 
+local function finish_handler(self, err)
+    if not err then
+        -- All resources are loaded, finally load the level:
+        print("Everything is OK, load level 2!")
+        msg.post(EXCLUDED_PROXY_URL, hash("load"))
+    else
+        -- Try again?...
+        print("ERROR: " .. err)
+    end
+end
+
 local function http_loading_progress_handler(self, loaded, total)
     msg.post("#loading_progress", "enable")
 
@@ -19,25 +30,16 @@ end
 local function load_level2(self)
     local missing_resources = collectionproxy.missing_resources(EXCLUDED_PROXY_URL)
     if next(missing_resources) ~= nil then
-        print("Resources are missing, downloading...")
-
-        -- TIP: You can pass the "missing_resources" table instead of "nil" to load only missing resources.
-        reszip.request_and_load_zip(ZIP_FILE_LOCATION, nil, function(self, err)
-            if not err then
-                -- All resources are loaded, finally load the level:
-                print("Everything is OK, load the level 2!")
-                msg.post(EXCLUDED_PROXY_URL, hash("load"))
-
-                -- Free the memory, if you are not going to use the `resources.zip` file later (i.e. you can )
-                reszip.clear_cache()
-            else
-                -- Try again?...
-                print("ERROR: " .. err)
-            end
-        end, http_loading_progress_handler)
+        print("Some resources are missing, so download the resources archive...")
+        assert(liveupdate, "`liveupdate` module is missing.")
+
+        reszip.load_and_mount_zip(ZIP_FILE_LOCATION, {
+            on_finish = finish_handler, 
+            on_progress = http_loading_progress_handler
+        })
     else
         -- All resources exist, so load the level:
-        print("Resources are already loaded. Let's load the level 2!")
+        print("Resources are already loaded. Let's load level 2!")
         msg.post(EXCLUDED_PROXY_URL, hash("load"))
     end
 end

game.project

@@ -29,4 +29,3 @@ scale_mode = fit
 
 [input]
 game_binding = /builtins/input/all.input_bindingc
-

liveupdate_reszip/manifests/web/engine_template.html

@@ -1,12 +1,15 @@
 <html>
     <body>
         <script id="engine-setup" type="text/javascript">
+            // {{#liveupdate_reszip.wipe_on_start}}
             // HashSHA1 implementation
             !function(){var r=function(r){for(var n="",t=7;t>=0;t--)n+="0123456789abcdef".charAt(r>>4*t&15);return n},n=function(r,n){var t=(65535&r)+(65535&n);return(r>>16)+(n>>16)+(t>>16)<<16|65535&t},e=function(r,n){return r<<n|r>>>32-n},o=function(r,n,t,e){return r<20?n&t|~n&e:r<40?n^t^e:r<60?n&t|n&e|t&e:n^t^e},u=function(r){return r<20?1518500249:r<40?1859775393:r<60?-1894007588:-899497514};window._HashSHA1=function(f){for(var a=function(r){for(var n=1+(r.length+8>>6),t=new Array(16*n),e=0;e<16*n;e++)t[e]=0;for(e=0;e<r.length;e++)t[e>>2]|=r.charCodeAt(e)<<24-e%4*8;return t[e>>2]|=128<<24-e%4*8,t[16*n-1]=8*r.length,t}(f),c=new Array(80),i=1732584193,h=-271733879,v=-1732584194,A=271733878,g=-1009589776,l=0;l<a.length;l+=16){for(var w=i,d=h,y=v,H=A,b=g,s=0;s<80;s++)c[s]=s<16?a[l+s]:e(c[s-3]^c[s-8]^c[s-14]^c[s-16],1),t=n(n(e(i,5),o(s,h,v,A)),n(n(g,c[s]),u(s))),g=A,A=v,v=e(h,30),h=i,i=t;i=n(i,w),h=n(h,d),v=n(v,y),A=n(A,H),g=n(g,b)}return r(i)+r(h)+r(v)+r(A)+r(g)}}();
+            // {{/liveupdate_reszip.wipe_on_start}}
 
-            // Delete all LiveUpdate files stored in IDBFS
             var old_preloadAndCallMain = Module._preloadAndCallMain;
             Module._preloadAndCallMain = function () {
+                // {{#liveupdate_reszip.wipe_on_start}}
+                // Delete all LiveUpdate files stored in IDBFS
                 var dir = DMSYS.GetUserPersistentDataRoot();
                 var resDir = _HashSHA1("{{{project.title}}}");
                 try {
@@ -18,6 +21,10 @@
                 try {
                     FS.unlink(dir + "/." + resDir + "/liveupdate.arci.tmp");
                 } catch (e) {}
+                try {
+                    FS.unlink(dir + "/." + resDir + "/liveupdate.mounts");
+                } catch (e) {}
+                // {{/liveupdate_reszip.wipe_on_start}}
 
                 if (Module._archiveLoaded) {
                     // {{#liveupdate_reszip.preload_file}}