Updated Artifacts Folder Structure #1843

mehsaandev · 2025-01-21T21:50:26Z

New folder structure for Local ensemble Apps:-

inside ensemble/apps/yourAppName, you should have the following structure for your app data:

asset
config
internal_script
internal_widget
screen
theme
.manifest.json

Currently Working Items:

Screens
widgets
scripts
themes
Assets
Env and Secret

Please change the following files in order to run this new structure:

Update `pubspec.yaml` of starter App:

Add all your apps folder in the assets section at the very bottom. For example if the name of your app is helloApp then add the following under assets

 - ensemble/apps/helloApp/
 - ensemble/apps/helloApp/.manifest.json
 - ensemble/apps/helloApp/asset/
 - ensemble/apps/helloApp/screen/
 - ensemble/apps/helloApp/internal_widget/
 - ensemble/apps/helloApp/internal_script/
 - ensemble/apps/helloApp/theme/
 - ensemble/apps/helloApp/config/appConfig.json
 - ensemble/apps/helloApp/secret/secrets.json

CC: @amin-nas

sharjeelyunus · 2025-01-27T13:10:05Z

Why do we still have these internal_script and internal_widget? We've been criticizing the Firebase structure for having these collections, and now we're expanding it to new features?

Shouldn't the folder structure for both Electron and the starter be something like this?

assets
fonts
scripts
widgets
screens
i8n
theme.yaml

CC: @kmahmood74 @evshi

sharjeelyunus · 2025-02-03T14:48:08Z

Why do we still have these internal_script and internal_widget? We've been criticizing the Firebase structure for having these collections, and now we're expanding it to new features?

Shouldn't the folder structure for both Electron and the starter be something like this?

assets

fonts

scripts

widgets

screens

i8n

theme.yaml

CC: @kmahmood74 @evshi

Just a had a convo with @evshi we will be renaming these to proper structure
@mehsaandev please discuss the latest structure with evan for consistency

mehsaandev · 2025-02-03T14:56:01Z

Why do we still have these internal_script and internal_widget? We've been criticizing the Firebase structure for having these collections, and now we're expanding it to new features?
Shouldn't the folder structure for both Electron and the starter be something like this?

assets

fonts

scripts

widgets

screens

i8n

theme.yaml

CC: @kmahmood74 @evshi

Just a had a convo with @evshi we will be renaming these to proper structure @mehsaandev please discuss the latest structure with evan for consistency

Sure

sharjeelyunus · 2025-02-04T20:20:12Z

modules/ensemble/lib/framework/definition_providers/local_provider.dart

+      final manifestContent =
+          await rootBundle.loadString(path + '.manifest.json');
+      final Map<String, dynamic> manifestMap = json.decode(manifestContent);


Hi @evshi,

This function is taking a lot amount of time to load and decode. Its whole job is to extract widget/script names, which are needed to fetch the content. But, the manifest file, for apps with around 100 screens and 100 widgets, can go up to 4-5 MB in size and takes 3-5 seconds to load and decode.

Why are we storing the YAML directly in the manifest file. When we remove the content keys/values from it, it works fine. The large content data is really slowing things down.

From the Electron app, could we either:

Remove the content keys/values from the manifest file, or

Provide a separate file containing only the widget and scripts names, which we can use here?

Yeah, we should not include the YAML in the manifest. @anserwaseem we discussed this previously can you address this now since it is causing this performance issue?

amin-nas · 2025-02-05T14:49:31Z

@mehsaandev @sharjeelyunus - is this ready to be merged?

sharjeelyunus · 2025-02-05T17:15:10Z

@mehsaandev @sharjeelyunus - is this ready to be merged?

this needs a lil bit of cleanup

sharjeelyunus

please make sure this is not breaking the ensemble or server provider. Also update the example helloApp structure with the updated structure.

sharjeelyunus · 2025-02-05T17:23:20Z

modules/ensemble/lib/util/utils.dart

+    String path = EnsembleConfigService.config["definitions"]?['local']?["path"];
+    return '${path}/asset/${stripQueryParamsFromAsset(asset)}';


shouldn't this be assets? instead of asset?

also the assets are stored locally in ensemble/assets folder in case of ensemble provider. wouldn't this break the assets for ensemble provider?

yes it should be assets

and for Ensemble provider, No it won't because for ensemble provider source of each assets is cloud-based. we are not storing assets locally in case of ensemble provider, so in case of ensemble provider this function will not be called to create a local path

NO, we are storing local assets in ensemble provider as well, for studio yes we provide the cloud-based urls, but when user creates a build, we store the assets in the old assets folder for ensemble provider

sharjeelyunus · 2025-02-05T17:23:30Z

modules/ensemble/lib/framework/secrets.dart

+        final secretsString =
+            await rootBundle.loadString('${path}/config/secrets.json');
+        final Map<String, dynamic> appSecretsMap = json.decode(secretsString);
+        // await dotenv.load();


remove this unnecessary comment

sharjeelyunus · 2025-02-06T15:45:10Z

modules/ensemble/lib/framework/secrets.dart

+        String path =
+            EnsembleConfigService.config["definitions"]?['local']?["path"];
+        final secretsString =
+            await rootBundle.loadString('${path}/config/secrets.json');
+        final Map<String, dynamic> appSecretsMap = json.decode(secretsString);
+        appSecretsMap["secrets"].forEach((key, value) {
+          secrets![key] = value;
+        });


how this will work for ensemble provider? why is this static to local only?

sharjeelyunus · 2025-02-06T17:20:51Z

starter/ensemble/apps/helloApp/scripts/common.yaml

+function sayHello(name) {
+  console.log('sayHello:'+name);
+}
+
+function saveName(first,last) {
+  ensemble.storage.helloApp = {name: {first: first, last: last}};
+}
+
+function getName() {
+  if ( ensemble.storage.helloApp != null ) {
+    return ensemble.storage.helloApp.name;
+  }
+  return null;
+}


shouldn't this be a JS file??? why scripts files are .yaml files?

mehsaandev · 2025-02-06T19:07:00Z

@sharjeelyunus Updated scripts extension to js, PTAL

…nfig' into artifacts-folder-config

Muhammad Ehsaan added 3 commits January 22, 2025 02:34

Updated artifacts folder config

5f58817

Added conditions for missing artifacts

9b61b18

Added secrets, env and assets support

5e6cfa0

mehsaandev requested a review from kmahmood74 January 26, 2025 12:51

mehsaandev changed the title ~~[WIP] Updated Artifacts Folder Structure~~ Updated Artifacts Folder Structure Jan 27, 2025

mehsaandev linked an issue Jan 28, 2025 that may be closed by this pull request

Update local definition provide to work with folder structured artifacts #1840

Open

mehsaandev self-assigned this Jan 28, 2025

updated new folder structure

300fce4

mehsaandev requested review from sharjeelyunus and removed request for kmahmood74 February 4, 2025 20:17

sharjeelyunus reviewed Feb 4, 2025

View reviewed changes

sharjeelyunus reviewed Feb 5, 2025

View reviewed changes

Updated HelloApp structure & fixed assets path

e90e572

mehsaandev requested a review from sharjeelyunus February 6, 2025 15:37

anserwaseem mentioned this pull request Feb 6, 2025

update folder structure when an app is pulled EnsembleUI/ensemble-react#1023

Merged

4 tasks

sharjeelyunus reviewed Feb 6, 2025

View reviewed changes

fixed ensemble secrets & added manifest.json

4d32ec2

mehsaandev requested a review from sharjeelyunus February 6, 2025 17:39

mehsaandev and others added 2 commits February 7, 2025 00:00

changed scripts extention to js

5d3e2b6

Merge branch 'main' into artifacts-folder-config

fa99b44

mehsaandev added 2 commits February 7, 2025 02:29

Updated Starter Readme

3b8d122

Merge remote-tracking branch 'refs/remotes/origin/artifacts-folder-co…

379f80b

…nfig' into artifacts-folder-config

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Updated Artifacts Folder Structure #1843

Updated Artifacts Folder Structure #1843

mehsaandev commented Jan 21, 2025 •

edited

Loading

sharjeelyunus commented Jan 27, 2025 •

edited

Loading

sharjeelyunus commented Feb 3, 2025

mehsaandev commented Feb 3, 2025

sharjeelyunus Feb 4, 2025

evshi Feb 4, 2025

amin-nas commented Feb 5, 2025

sharjeelyunus commented Feb 5, 2025

sharjeelyunus left a comment

sharjeelyunus Feb 5, 2025

mehsaandev Feb 6, 2025

sharjeelyunus Feb 6, 2025 •

edited

Loading

sharjeelyunus Feb 5, 2025 •

edited

Loading

sharjeelyunus Feb 6, 2025

sharjeelyunus Feb 6, 2025

mehsaandev commented Feb 6, 2025

		String path = EnsembleConfigService.config["definitions"]?['local']?["path"];
		return '${path}/asset/${stripQueryParamsFromAsset(asset)}';

Updated Artifacts Folder Structure #1843

Are you sure you want to change the base?

Updated Artifacts Folder Structure #1843

Conversation

mehsaandev commented Jan 21, 2025 • edited Loading

Update pubspec.yaml of starter App:

sharjeelyunus commented Jan 27, 2025 • edited Loading

sharjeelyunus commented Feb 3, 2025

mehsaandev commented Feb 3, 2025

sharjeelyunus Feb 4, 2025

Choose a reason for hiding this comment

evshi Feb 4, 2025

Choose a reason for hiding this comment

amin-nas commented Feb 5, 2025

sharjeelyunus commented Feb 5, 2025

sharjeelyunus left a comment

Choose a reason for hiding this comment

sharjeelyunus Feb 5, 2025

Choose a reason for hiding this comment

mehsaandev Feb 6, 2025

Choose a reason for hiding this comment

sharjeelyunus Feb 6, 2025 • edited Loading

Choose a reason for hiding this comment

sharjeelyunus Feb 5, 2025 • edited Loading

Choose a reason for hiding this comment

sharjeelyunus Feb 6, 2025

Choose a reason for hiding this comment

sharjeelyunus Feb 6, 2025

Choose a reason for hiding this comment

mehsaandev commented Feb 6, 2025

mehsaandev commented Jan 21, 2025 •

edited

Loading

Update `pubspec.yaml` of starter App:

sharjeelyunus commented Jan 27, 2025 •

edited

Loading

sharjeelyunus Feb 6, 2025 •

edited

Loading

sharjeelyunus Feb 5, 2025 •

edited

Loading