feat: add cli support to build assets from executable

[brianmay]

See #3402 (reply in thread)

[brianmay]

Not sure I understand the requirements correctly for what I have to do.

The instructions I got said:

AssetManifest::load_from_file("app.exe")

This matches my understanding of reading the ocde

But If I try this, it fails with a UTF-8 error:

[brian@miacis:~/tree/3rdparty/dioxus]$ cargo run -p dioxus-cli -- build_assets  ~/tree/personal/dioxus-fs/target/server-dev/dioxus-fs-demo abcd
    Finished `dev` profile [unoptimized] target(s) in 0.34s
     Running `target/debug/dx build_assets /home/brian/tree/personal/dioxus-fs/target/server-dev/dioxus-fs-demo abcd`
thread 'main' panicked at packages/cli/src/cli/build_assets.rs:19:61:
Failed to load asset manifest: stream did not contain valid UTF-8
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Oh, wait I think I need to do this:

https://github.com/DioxusLabs/dioxus/blob/main/packages/cli/src/build/request.rs#L253C1-L257

Will try that.

[brianmay]

[brian@miacis:~/tree/3rdparty/dioxus]$ cargo run -p dioxus-cli -- build_assets  ~/tree/personal/dioxus-fs/target/server-dev/dioxus-fs-demo abcd
    Finished `dev` profile [unoptimized] target(s) in 0.32s
     Running `target/debug/dx build_assets /home/brian/tree/personal/dioxus-fs/target/server-dev/dioxus-fs-demo abcd`
Writing assets AssetManifest { assets: {} }

Oh, wait, I don't have any assets in this project. Oops. Will fix.

[brianmay]

What works

It creates the required assets, the filenames look correct.

This works:

curl  -v http://localhost:4000/header-73ca13e70f7867c1.svg

What doesn't work

Front end is using filesystem paths, which isn't going to work, e.g.:

<link rel="stylesheet" href="/home/brian/tree/personal/dioxus-fs/assets/main.css">
<link rel="icon" href="/home/brian/tree/personal/dioxus-fs/assets/favicon.ico">

This was from:

const HEADER_SVG: Asset = asset!("assets/header.svg");
const MAIN_CSS: Asset = asset!("assets/main.css");
const FAVICON: Asset = asset!("assets/favicon.ico");


#[component]
fn App() -> Element {
    rsx! {
        document::Link { rel: "icon", href: FAVICON }
        document::Link { rel: "stylesheet", href: MAIN_CSS }
        Router::<Route> {}
    }
}

I think that is correct?

Can't help but think I am doing something stupid, but can't think what.

And the unwraps need to be replaced with better error handling (not sure how to do this).

[brianmay]

I have identified two problems with the client/wasm:

1. is_bundled_app() is returning false, but it needs to return true. OK, so I need to set DIOXUS_CLI_ENABLED env on the server. Which fixes that. But confused, I thought this would be running on the client not the server.
2. The client is now generating paths like http://localhost:4000/assets/main-f85f02c167ed32dc.css but the server wants http://localhost:4000/main-f85f02c167ed32dc.css

Wonder if this is a sign something not right on the server. But out of time now.

[brianmay]

Oh, probably just that I needed to put the assets in ~/tree/personal/dioxus-fs/target/server-dev/public/assets not ~/tree/personal/dioxus-fs/target/server-dev/public. Now it is working fine.

[brianmay]

Next step: try to fix up the error handling...

[brianmay]

Error handling fixed. Odd I thought I tried that before and it didn't work.

[brianmay]

I just noticed when I build assets from the nix build, it get the source paths wrong.

e.g. it looks for "/build/lknys4lnckh88mxvi7pba1zsvgfyh1a1-source/assets/header.svg" which presumably was valid during the build but not anymore.

So I switched this back to draft.

Will think about this more tomorrow. Can't think of any good options right now.

[brianmay]

The solution is to take another argument specifying where the source of the asset files are. That is the easy part.

We then get a relative path to the asset. i.e. We need to turn /build/lknys4lnckh88mxvi7pba1zsvgfyh1a1-source/assets/header.svg into assets/header.svg and add this to the specified source path.

There are two ways we can do this:

1. The dodgy way: We look for the "assets" component and delete everything before that. But this makes assumptions that the build path doesn't contain "assets".
2. The better way: IMHO the embedded path should always be relative paths. It is easy to turn a relative path into an absolute path in the CLI, but it is not so easy going the other way. Plus the current behaviour means the builds are non-reproducible because they leak information about the build system.

I am going to start of with the first solution. But if there is any interest, I can change to the second solution. Although this is going to be more invasive.

As a compromise we could store the relative path as well as the absolute path. Not sure I see any benefit though.

[brianmay]

OK, so I need to set DIOXUS_CLI_ENABLED env on the server. Which fixes that. But confused, I thought this would be running on the client not the server.

I was forgetting, it is the server which generates the initial html, which is why these paths are being generated in the server. When you force paths to be generated in wasm, it is always correct (I just tested this).

[brianmay]

I imagine the 2nd option could be implemented with something like:

diff --git a/packages/cli/src/cli/build_assets.rs b/packages/cli/src/cli/build_assets.rs
index 3a6834659..b81c905ae 100644
--- a/packages/cli/src/cli/build_assets.rs
+++ b/packages/cli/src/cli/build_assets.rs
@@ -1,7 +1,4 @@
-use std::{
-    fs::create_dir_all,
-    path::{Path, PathBuf},
-};
+use std::{fs::create_dir_all, path::PathBuf};

 use crate::{Result, StructuredOutput};
 use clap::Parser;
@@ -27,8 +24,7 @@ impl BuildAssets {

         create_dir_all(&self.destination)?;
         for (path, asset) in manifest.assets.iter() {
-            let relative_path = turn_asset_path_into_relative_path(path);
-            let source_path = self.source.join(relative_path);
+            let source_path = self.source.join(path);
             let destination_path = self.destination.join(asset.bundled_path());
             debug!(
                 "Processing asset {} --> {} {:#?}",
@@ -42,22 +38,3 @@ impl BuildAssets {
         Ok(StructuredOutput::Success)
     }
 }
-
-/// Hack to turn an absolute path into a relative path.
-///
-/// For example, the executable path might have the absolute path:
-/// "/build/lknys4lnckh88mxvi7pba1zsvgfyh1a1-source/assets/header.svg
-///
-/// And we need a relative path to the source directory:
-/// "assets/header.svg"
-fn turn_asset_path_into_relative_path(asset_path: &Path) -> PathBuf {
-    let components = asset_path
-        .components()
-        .skip_while(|c| c.as_os_str() != "assets")
-        .collect::<Vec<_>>();
-
-    components.iter().fold(PathBuf::new(), |mut acc, c| {
-        acc.push(c);
-        acc
-    })
-}
diff --git a/packages/manganis/manganis-core/src/asset.rs b/packages/manganis/manganis-core/src/asset.rs
index 97ceb6ef7..e2a809ed0 100644
--- a/packages/manganis/manganis-core/src/asset.rs
+++ b/packages/manganis/manganis-core/src/asset.rs
@@ -48,9 +48,6 @@ impl BundledAsset {
     /// Create a new asset but with a relative path
     ///
     /// This method is deprecated and will be removed in a future release.
-    #[deprecated(
-        note = "Relative asset!() paths are not supported. Use a path like `/assets/myfile.png` instead of `./assets/myfile.png`"
-    )]
     pub const fn new_relative(
         absolute_source_path: &'static str,
         bundled_path: &'static str,
diff --git a/packages/manganis/manganis-macro/src/asset.rs b/packages/manganis/manganis-macro/src/asset.rs
index 5774c37d4..36eb92e72 100644
--- a/packages/manganis/manganis-macro/src/asset.rs
+++ b/packages/manganis/manganis-macro/src/asset.rs
@@ -40,14 +40,10 @@ fn resolve_path(raw: &str) -> Result<PathBuf, AssetParseError> {
         });
     };

-    // 4. Ensure the path doesn't escape the crate dir
-    //
-    // - Note: since we called canonicalize on both paths, we can safely compare the parent dirs.
-    //   On windows, we can only compare the prefix if both paths are canonicalized (not just absolute)
-    //   https://github.com/rust-lang/rust/issues/42869
-    if path == manifest_dir || !path.starts_with(manifest_dir) {
+    // 4. Strip the manifest dir from the path
+    let Ok(path) = path.strip_prefix(&manifest_dir).map(|x| x.to_path_buf()) else {
         return Err(AssetParseError::InvalidPath { path });
-    }
+    };

     Ok(path)
 }

Note: I obviously had to remove the deprecated warning, that checks the output of the macro not the input, and seems to be pointless.

[jkelleyrtp]

OK, so I need to set DIOXUS_CLI_ENABLED env on the server. Which fixes that. But confused, I thought this would be running on the client not the server.

I was forgetting, it is the server which generates the initial html, which is why these paths are being generated in the server. When you force paths to be generated in wasm, it is always correct (I just tested this).

Yes, was going to comment on this but you figured it out, when .resolve() is called with cargo run, the asset path is the absolute path on the user's filesystem. When dx runs your app, it unsets the cargo_manifest_dir env var since we want dx serve to emulate a being bundled.

I think, since we now force assets to be part of the crate they're defined in, we can standardize the asset path output to always be in the form of /crate/absolute/path and when under cargo run we use that path rather than the file system absolute paht.