Skip to content

Commit d562214

Browse files
authored
feat: etag and gzip by compiler option (#21)
* feat: etag compiler option * feat: gzip compiler option * feat: order of directives * feat: created and version info * fix: separated demo env * feat: create output directory * feat: 2x9 build and test system * feat: detect precompressed files * feat: reduce .h file size * feat: skip gz under 1024 * fix: pipeline node version * fix: proper svelte demo app * feat: warning if not usable directive used * feat: colored console
1 parent a0314dc commit d562214

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

44 files changed

+1260
-11474
lines changed

.github/workflows/ci-dev.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ jobs:
1414
- uses: actions/checkout@v4
1515
- uses: actions/setup-node@v4
1616
with:
17-
node-version: '20.13.1'
17+
node-version: '20.16.0'
1818

1919
- run: npm ci
2020
- run: npm run format:check

.github/workflows/ci-full.yaml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ jobs:
1414
- uses: actions/checkout@v4
1515
- uses: actions/setup-node@v4
1616
with:
17-
node-version: '20.13.1'
17+
node-version: '20.16.0'
1818

1919
- run: npm ci
2020
- run: npm run format:check
@@ -47,7 +47,7 @@ jobs:
4747
- uses: actions/checkout@v4
4848
- uses: actions/setup-node@v4
4949
with:
50-
node-version: '20.13.1'
50+
node-version: '20.16.0'
5151
registry-url: 'https://registry.npmjs.org'
5252
- run: npm ci
5353
- run: npm run build

.gitignore

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,3 +12,5 @@ dist
1212

1313
# dotenv environment variables file
1414
.env
15+
16+
demo/esp32/include/**/*.h

CHANGELOG.md

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
# Change log
2+
3+
## 1.4
4+
5+
### 1.4.0
6+
7+
## 1.3
8+
9+
### 1.3.1
10+
11+
- feat: filename directives are uppercased
12+
- feat: filecount by extension
13+
- fix: readme
14+
15+
### 1.3.0
16+
17+
- C++ defines can be used
18+
- C++ defines group by filetypes: Should we be able to check if a given file name or number of files is correct? For this, we should find C++ defines in the generated code.
19+
20+
## 1.2
21+
22+
### 1.2.4
23+
24+
- Deps update
25+
26+
### 1.2.2
27+
28+
- Necessary C++ includes
29+
30+
### 1.2.1
31+
32+
- Windows paths
33+
34+
### 1.2.0
35+
36+
- ESP8266/ESP8285 is also supported
37+
38+
## 1.1
39+
40+
### 1.1.0
41+
42+
- ETag available

README.md

Lines changed: 40 additions & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
11
# `svelteesp32` ![image](https://badges.github.io/stability-badges/dist/stable.svg)
22

3+
[Changelog](CHANGELOG.md)
4+
35
# Convert Svelte (or React/Angular/Vue) JS application to serve it from ESP32/ESP8266 webserver
46

57
### Forget SPIFFS and LittleFS now
@@ -10,11 +12,13 @@ In order to be able to easily update OTA, it is important - from the users' poin
1012

1113
This npm package provides a solution for **inserting any JS client application into the ESP web server** (PsychicHttp and also ESPAsyncWebServer available, PsychicHttp is the default). For this, JS, html, css, font, assets, etc. files must be converted to binary byte array. Npm mode is easy to use and easy to **integrate into your CI/CD pipeline**.
1214

13-
> Starting with version v1.1.0, the ETag header is also supported.
15+
> Version v1.4.0 has a breaking change! --no-gzip changed to --gzip. Starting with this version c++ compiler directives are available to setup operation in project level.
16+
17+
> Starting with version v1.3.0, c++ defines can be used.
1418
1519
> Starting with version v1.2.0, ESP8266/ESP8285 is also supported.
1620
17-
> Starting with version v1.3.0, c++ defines can be used.
21+
> Starting with version v1.1.0, the ETag header is also supported.
1822
1923
### Usage
2024

@@ -28,29 +32,20 @@ After a successful Svelte build (rollup/webpack/vite) **create an includeable c+
2832

2933
```bash
3034
// for PsychicHttpServer
31-
npx svelteesp32 -e psychic -s ../svelteapp/dist -o ../esp32project/svelteesp32.h --etag
35+
npx svelteesp32 -e psychic -s ../svelteapp/dist -o ../esp32project/svelteesp32.h --etag=true
3236

3337
// for ESPAsyncWebServer
34-
npx svelteesp32 -e async -s ../svelteapp/dist -o ../esp32project/svelteesp32.h --etag
38+
npx svelteesp32 -e async -s ../svelteapp/dist -o ../esp32project/svelteesp32.h --etag=true
3539
```
3640

3741
During the **translation process**, the processed file details are visible, and at the end, the result shows the ESP's memory allocation (gzip size)
3842

3943
```
40-
[assets/index-KwubEIf-.js]
41-
✓ gzip used (38850 -> 12547)
42-
43-
[assets/index-Soe6cpLA.css]
44-
✓ gzip used (32494 -> 5368)
45-
46-
[favicon.png]
47-
x gzip unused (33249 -> 33282)
48-
49-
[index.html]
50-
✓ gzip used (472 -> 308)
51-
52-
[roboto_regular.json]
53-
✓ gzip used (363757 -> 93567)
44+
[assets/index-KwubEIf-.js] ✓ gzip used (38850 -> 12547)
45+
[assets/index-Soe6cpLA.css] ✓ gzip used (32494 -> 5368)
46+
[favicon.png] x gzip unused (33249 -> 33282)
47+
[index.html] x gzip unused (too small) (472 -> 308)
48+
[roboto_regular.json] ✓ gzip used (363757 -> 93567)
5449
5550
5 files, 458kB original size, 142kB gzip size
5651
../../../Arduino/EspSvelte/svelteesp32.h 842kB size
@@ -158,17 +153,21 @@ If you **only work on ESP32**, I recommend using PsychicHttpServer, which uses t
158153

159154
All modern browsers have been able to handle gzip-compressed content for years. For this reason, there is no question that the easily compressed JS and CSS files are stored compressed in the ESP32/ESP8266 and sent to the browser.
160155

161-
During the translation process, data in gzip format is generated and will be used if the **size is greater than 100 bytes** and we experience a **reduction of at least 15%**. In such a case, the compressed data is unconditionally sent to the browser with the appropriate **Content-Encoding** header information.
156+
During the translation process, data in gzip format is generated and will be used if the **size is greater than 1024 bytes** and we experience a **reduction of at least 15%**. In such a case, the compressed data is unconditionally sent to the browser with the appropriate **Content-Encoding** header information.
157+
158+
Automatic compression can be turned off with the `--gzip=false` option.
162159

163-
Automatic compression can be turned off with the `--no-gzip` option.
160+
> This setting has three states: yes, no, and compiler mode is available. In compiler mode, you can disable/enable Gzip by setting the `SVELTEESP32_ENABLE_GZIP` c++ compiler directive. For example, if using platformio, just write `-D SVELTEESP32_ENABLE_GZIP`.
164161
165162
### ETag
166163

167164
The ETag HTTP header can be used to significantly reduce network traffic. If the server sends ETag information, the client can check the integrity of the file by sending back this ETag (in `If-None-Match` header) without sending the data back again. All browsers use this function, the 304 HTTP response code is clearly visible in the network traffic.
168165

169166
Since microcontroller data traffic is moderately expensive, it is an individual decision whether to use the ETag or not. We **recommend using ETag**, which adds a bit more code (about 1-3%) but results in a much cleaner operation.
170167

171-
The use of ETag is **not enabled by default**, this can be achieved with the `--etag` switch.
168+
The use of ETag is **not enabled by default**, this can be achieved with the `--etag=true` switch.
169+
170+
> This setting has three states: yes, no, and compiler mode is available. In compiler mode, you can disable/enable ETag by setting the `SVELTEESP32_ENABLE_ETAG` c++ compiler directive. For example, if using platformio, just type `-D SVELTEESP32_ENABLE_ETAG`.
172171
173172
### Main entry point - index.html
174173

@@ -214,27 +213,35 @@ You can include a blocker error if a named file accidentally missing from the bu
214213
...
215214
```
216215

216+
You can use the following c++ directives at the project level if you want to configure the usage there: `SVELTEESP32_ENABLE_ETAG` and `SVELTEESP32_ENABLE_GZIP`. (Do not forget `--etag=compiler` or `--gzip=compiler` command line arg!)
217+
217218
### Command line options
218219

219-
| Option | Description | default |
220-
| ------------- | --------------------------------------------------------- | ----------------------- | ------- |
221-
| `-s` | **Source dist folder contains compiled web files** | |
222-
| `-e` | The engine for which the include file is created [psychic | async] | psychic |
223-
| `-o` | Generated output file with path | `svelteesp32.h` |
224-
| `--etag` | Use ETag header for cache | false |
225-
| `--no-gzip` | Do not compress content with gzip | false -> gzip used |
226-
| `--espmethod` | Name of generated method | `initSvelteStaticFiles` |
227-
| `--define` | Prefix of c++ defines | `SVELTEESP32` |
228-
| `-h` | Show help | |
220+
| Option | Description | default |
221+
| ------------- | ---------------------------------------------------------------- | ----------------------- |
222+
| `-s` | **Source dist folder contains compiled web files** | |
223+
| `-e` | The engine for which the include file is created (psychic/async) | psychic |
224+
| `-o` | Generated output file with path | `svelteesp32.h` |
225+
| `--etag` | Use ETag header for cache (true/false/compiler) | false |
226+
| `--gzip` | Compress content with gzip (true/false/compiler) | true |
227+
| `--created` | Include creation time | false |
228+
| `--version` | Include a version string, `--version=v$npm_package_version` | '' |
229+
| `--espmethod` | Name of generated method | `initSvelteStaticFiles` |
230+
| `--define` | Prefix of c++ defines | `SVELTEESP32` |
231+
| `-h` | Show help | |
229232

230233
### Q&A
231234

232235
- **How big a frontend application can be placed?** If you compress the content with gzip, even a 3-4Mb assets directory can be placed. This is a serious enough amount to serve a complete application.
233236

234-
- **How fast is cpp file compilation?** The cpp file can be large, but it can be compiled in a few seconds on any machine. If you don't modify your svelte/react app, it will use the already compiled cpp file (not recompile). This does not increase the speed of ESP32/ESP8266 development.
237+
- **How fast is cpp file compilation?** The cpp (.h) file can be large, but it can be compiled in a few seconds on any machine. If you don't modify your svelte/react app, it will use the already compiled cpp file (not recompile). This does not increase the speed of ESP32/ESP8266 development.
235238

236239
- **Does the solution use PROGMEM?** No and yes. ESP32 no longer has PROGMEM. (Exists, but does not affect the translation). Instead, if we use a const array in the global namespace, its content will be placed in the code area, i.e. it will not be used from the heap or the stack, so the content of the files to be served will be placed next to the code. When working on ESP8266, PROGMEM will actually be used.
237240

238-
- **Is this safe to use in production?** I suggest you give it a try! If you find it useful and safe in several different situations, feel free to use it, just like any other free library.
241+
- **Why is the .h file so big?** The source files are always larger than the binary compiled from them. Does the size information in the header (SVELTEESP32_SIZE, SVELTEESP32_SIZE_GZIP) show the actual memory allocation.
242+
243+
- **Is collaboration between groups supported?** Yes, the Frontend team produces the application, the use of svelteesp32 is part of the build process. Then, provided with a version number, the .h file is placed in git, which the ESP team translates into the platformio application.
239244

240245
- **Will you develop it further?** Since I use it myself, I will do my best to make the solution better and better.
246+
247+
- **Is this safe to use in production?** I suggest you give it a try! If you find it useful and safe in several different situations, feel free to use it, just like any other free library.

demo/esp32/include/placeholder.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
Generated header files

demo/esp32/include/svelteesp32async.h

Lines changed: 0 additions & 97 deletions
This file was deleted.

demo/esp32/include/svelteesp32psychic.h

Lines changed: 0 additions & 107 deletions
This file was deleted.

demo/esp32/platformio.ini

Lines changed: 118 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,10 +3,126 @@ platform = espressif32
33
board = wemos_d1_mini32
44
framework = arduino
55

6+
7+
68
[env:async]
7-
build_flags = -D ASYNC
9+
build_flags =
10+
-D ASYNC
11+
-I include/_
12+
lib_deps = https://github.com/me-no-dev/ESPAsyncWebServer
13+
14+
[env:async_E]
15+
build_flags =
16+
-D ASYNC
17+
-I include/e
18+
lib_deps = https://github.com/me-no-dev/ESPAsyncWebServer
19+
20+
[env:async_EC]
21+
build_flags =
22+
-D ASYNC
23+
-I include/ec
24+
-D SVELTEESP32_ENABLE_ETAG
25+
lib_deps = https://github.com/me-no-dev/ESPAsyncWebServer
26+
27+
[env:async_ECG]
28+
build_flags =
29+
-D ASYNC
30+
-I include/ecg
31+
-D SVELTEESP32_ENABLE_ETAG
32+
lib_deps = https://github.com/me-no-dev/ESPAsyncWebServer
33+
34+
[env:async_ECGC]
35+
build_flags =
36+
-D ASYNC
37+
-I include/ecgc
38+
-D SVELTEESP32_ENABLE_ETAG
39+
-D SVELTEESP32_ENABLE_GZIP
40+
lib_deps = https://github.com/me-no-dev/ESPAsyncWebServer
41+
42+
[env:async_EG]
43+
build_flags =
44+
-D ASYNC
45+
-I include/eg
46+
lib_deps = https://github.com/me-no-dev/ESPAsyncWebServer
47+
48+
[env:async_EGC]
49+
build_flags =
50+
-D ASYNC
51+
-I include/egc
52+
-D SVELTEESP32_ENABLE_GZIP
53+
lib_deps = https://github.com/me-no-dev/ESPAsyncWebServer
54+
55+
[env:async_G]
56+
build_flags =
57+
-D ASYNC
58+
-I include/g
59+
lib_deps = https://github.com/me-no-dev/ESPAsyncWebServer
60+
61+
[env:async_GC]
62+
build_flags =
63+
-D ASYNC
64+
-I include/gc
65+
-D SVELTEESP32_ENABLE_GZIP
866
lib_deps = https://github.com/me-no-dev/ESPAsyncWebServer
967

68+
69+
1070
[env:psychic]
11-
build_flags = -D PSYCHIC
71+
build_flags =
72+
-D PSYCHIC
73+
-I include/_
74+
lib_deps = hoeken/PsychicHttp
75+
76+
[env:psychic_E]
77+
build_flags =
78+
-D PSYCHIC
79+
-I include/e
80+
lib_deps = hoeken/PsychicHttp
81+
82+
[env:psychic_EC]
83+
build_flags =
84+
-D PSYCHIC
85+
-I include/ec
86+
-D SVELTEESP32_ENABLE_ETAG
87+
lib_deps = hoeken/PsychicHttp
88+
89+
[env:psychic_ECG]
90+
build_flags =
91+
-D PSYCHIC
92+
-I include/ecg
93+
-D SVELTEESP32_ENABLE_ETAG
94+
lib_deps = hoeken/PsychicHttp
95+
96+
[env:psychic_ECGC]
97+
build_flags =
98+
-D PSYCHIC
99+
-I include/ecgc
100+
-D SVELTEESP32_ENABLE_ETAG
101+
-D SVELTEESP32_ENABLE_GZIP
102+
lib_deps = hoeken/PsychicHttp
103+
104+
[env:psychic_EG]
105+
build_flags =
106+
-D PSYCHIC
107+
-I include/eg
108+
lib_deps = hoeken/PsychicHttp
109+
110+
[env:psychic_EGC]
111+
build_flags =
112+
-D PSYCHIC
113+
-I include/egc
114+
-D SVELTEESP32_ENABLE_GZIP
115+
lib_deps = hoeken/PsychicHttp
116+
117+
[env:psychic_G]
118+
build_flags =
119+
-D PSYCHIC
120+
-I include/g
121+
lib_deps = hoeken/PsychicHttp
122+
123+
[env:psychic_GC]
124+
build_flags =
125+
-D PSYCHIC
126+
-I include/gc
127+
-D SVELTEESP32_ENABLE_GZIP
12128
lib_deps = hoeken/PsychicHttp

demo/esp32/src/main.cpp

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44
#include <ESPAsyncWebServer.h>
55
#include "svelteesp32async.h"
66

7-
#if SVELTEESP32_COUNT != 5
7+
#if SVELTEESP32_COUNT != 10
88
#error Invalid file count
99
#endif
1010

@@ -30,7 +30,7 @@ void loop() {}
3030
#include <PsychicHttp.h>
3131
#include "svelteesp32psychic.h"
3232

33-
#if SVELTEESP32_COUNT != 5
33+
#if SVELTEESP32_COUNT != 10
3434
#error Invalid file count
3535
#endif
3636

demo/svelte/.eslintignore

Lines changed: 0 additions & 12 deletions
This file was deleted.

0 commit comments

Comments
 (0)