Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

chore: update Dockerfile to reduce the image size #327

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

chore: update Dockerfile to reduce the image size #327

wants to merge 1 commit into from

Conversation

tdurieux
Copy link

Hi there,

I've made a small improvement to the Dockerfile that I think could help optimize the image size.

Summary of the changes:

  • I added the --no-cache-dir to the pip command to disable the package cache.
  • I added the --no-install-recommends to with apt-get in order to not install unnecessary packages and reduce the image size.

Impact on the image size:

  • Image size before repair: 1.18 GB
  • Image size after repair: 887.29 MB
  • Difference: 323.78 MB (26.73%)

I hope that you will find these changes useful to you. Let me know if you have any questions or concerns.

Thanks,

@benzino77
Copy link
Owner

It was deleted by purpose:
ec1a007

@tdurieux
Copy link
Author

Sorry I did not see that.

@benzino77
Copy link
Owner

There was a problem with one package installation (can't remember which one - zeroconf?) as a dependency for platformio. That package was installed in wrong version when --no-cache-dir was in place.

@tdurieux
Copy link
Author

That is an interesting behavior, I was not aware that --no-cache-dir could impact the version selection.
I could only see it happening in a weird version conflict inside transitive dependencies.
I am not sure how to fix this problem correctly because relying on the cache to solve the problem seems a bit unreliable :)

@benzino77
Copy link
Owner

I'm not sure where the problem is (maybe with defined dependencies for platformio?), but I've made an observation that starting TasmoCompiler on GitPod, where there is no --no-cache-dir flag, works as expected.

@tdurieux
Copy link
Author

I reproduced the bug I dont have a better solution for the moment.
I will check the behavior of the image with only --no-install-recommends.

Processing tasmota4M (board: esp8266_4M2M; platform: https://github.com/tasmota/platform-espressif8266/releases/download/v2.7.4/platform-espressif8266-2.7.4.zip; framework: arduino)
--------------------------------------------------------------------------------
Platform Manager: Installing https://github.com/tasmota/platform-espressif8266/releases/download/v2.7.4/platform-espressif8266-2.7.4.zip
Downloading 0% 10% 20% 30% 40% 50% 60% 70% 80%
Unpacking
Platform Manager: [email protected]+20221113 has been installed!
Tool Manager: Installing platformio/toolchain-xtensa @ ~2.40802.0
Downloading 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Unpacking
Tool Manager: [email protected] has been installed!
Tool Manager: Installing tasmota/framework-arduinoespressif8266 @ ~2.7.4
Downloading 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Unpacking
Tool Manager: [email protected]+9 has been installed!
Tool Manager: Installing https://github.com/tasmota/esptool/releases/download/v4.3.1/esptool-4.3.1.zip
Downloading 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Unpacking
Tool Manager: [email protected] has been installed!
Tool Manager: Installing platformio/tool-scons @ ~4.40400.0
Downloading 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Unpacking
Tool Manager: [email protected] has been installed!
Verbose mode can be enabled via `-v, --verbose` option
CONFIGURATION: https://docs.platformio.org/page/boards/espressif8266/esp8266_4M2M.html
PLATFORM: Espressif 8266 (2.7.4+20221113) > Espressif Generic ESP8266 Tasmota 1M sketch 1M OTA 2M FS
HARDWARE: ESP8266 80MHz, 80KB RAM, 972.00KB Flash
PACKAGES: 
 - framework-arduinoespressif8266 @ 2.7.4+9 
 - tool-esptoolpy @ 1.40301.221017 (4.3.1) 
 - toolchain-xtensa @ 2.40802.200502 (4.8.2)
Converting tasmota.ino
Installing Python dependencies
Collecting zopfli>=0.2.1
  Downloading https://files.pythonhosted.org/packages/9a/ed/d004d5737f9546167eecf0ecd995ee1a796703e512deb2f2ea26cdbe4b3e/zopfli-0.2.2.zip (205kB)
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
Building wheels for collected packages: zopfli
  Running setup.py bdist_wheel for zopfli: started
  Running setup.py bdist_wheel for zopfli: finished with status 'error'
  Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-flclb6c9/zopfli/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/pip-wheel-ydt4aa7u --python-tag cp37:
  /tmp/pip-build-env-b8eo6qsm/lib/python3.7/site-packages/setuptools/config/setupcfg.py:515: SetuptoolsDeprecationWarning: The license_file parameter is deprecated, use license_files instead.
    warnings.warn(msg, warning_class)
  /tmp/pip-build-env-b8eo6qsm/lib/python3.7/site-packages/setuptools/installer.py:30: SetuptoolsDeprecationWarning: setuptools.installer is deprecated. Requirements should be satisfied by a PEP 517 installer.
    SetuptoolsDeprecationWarning,
  WARNING: The wheel package is not available.
  WARNING: The wheel package is not available.
  usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
     or: -c --help [cmd1 cmd2 ...]
     or: -c --help-commands
     or: -c cmd --help
  
  error: invalid command 'bdist_wheel'
  
  ----------------------------------------
  Running setup.py clean for zopfli
  Failed building wheel for zopfli
Failed to build zopfli
Installing collected packages: zopfli
  Running setup.py install for zopfli: started
    Running setup.py install for zopfli: finished with status 'error'
    Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-flclb6c9/zopfli/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-b4x6o0jj/install-record.txt --single-version-externally-managed --compile:
    /tmp/pip-build-env-b8eo6qsm/lib/python3.7/site-packages/setuptools/config/setupcfg.py:515: SetuptoolsDeprecationWarning: The license_file parameter is deprecated, use license_files instead.
      warnings.warn(msg, warning_class)
    /tmp/pip-build-env-b8eo6qsm/lib/python3.7/site-packages/setuptools/installer.py:30: SetuptoolsDeprecationWarning: setuptools.installer is deprecated. Requirements should be satisfied by a PEP 517 installer.
      SetuptoolsDeprecationWarning,
    WARNING: The wheel package is not available.
    WARNING: The wheel package is not available.
    running install
    /tmp/pip-build-env-b8eo6qsm/lib/python3.7/site-packages/setuptools/command/install.py:37: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
      setuptools.SetuptoolsDeprecationWarning,
    running build
    running build_py
    creating build
    creating build/lib.linux-aarch64-cpython-37
    creating build/lib.linux-aarch64-cpython-37/zopfli
    copying src/zopfli/_version.py -> build/lib.linux-aarch64-cpython-37/zopfli
    copying src/zopfli/gzip.py -> build/lib.linux-aarch64-cpython-37/zopfli
    copying src/zopfli/png.py -> build/lib.linux-aarch64-cpython-37/zopfli
    copying src/zopfli/__init__.py -> build/lib.linux-aarch64-cpython-37/zopfli
    copying src/zopfli/zlib.py -> build/lib.linux-aarch64-cpython-37/zopfli
    running egg_info
    writing src/zopfli.egg-info/PKG-INFO
    writing dependency_links to src/zopfli.egg-info/dependency_links.txt
    writing requirements to src/zopfli.egg-info/requires.txt
    writing top-level names to src/zopfli.egg-info/top_level.txt
    listing git files failed - pretending there aren't any
    reading manifest file 'src/zopfli.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    adding license file 'COPYING'
    writing manifest file 'src/zopfli.egg-info/SOURCES.txt'
    running build_ext
    building 'zopfli.zopfli' extension
    creating build/temp.linux-aarch64-cpython-37
    creating build/temp.linux-aarch64-cpython-37/src
    creating build/temp.linux-aarch64-cpython-37/zopfli
    creating build/temp.linux-aarch64-cpython-37/zopfli/src
    creating build/temp.linux-aarch64-cpython-37/zopfli/src/zopfli
    creating build/temp.linux-aarch64-cpython-37/zopfli/src/zopflipng
    creating build/temp.linux-aarch64-cpython-37/zopfli/src/zopflipng/lodepng
    aarch64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -I/usr/include/python3.7m -c src/zopflimodule.c -o build/temp.linux-aarch64-cpython-37/src/zopflimodule.o
    error: command 'aarch64-linux-gnu-gcc' failed: No such file or directory: 'aarch64-linux-gnu-gcc'
    
    ----------------------------------------
Command "/usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-flclb6c9/zopfli/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-b4x6o0jj/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-install-flclb6c9/zopfli/
*** Error 1
LDF: Library Dependency Finder -> https://bit.ly/configure-pio-ldf
LDF Modes: Finder ~ chain, Compatibility ~ strict
Found 94 compatible libraries
Scanning dependencies...
Dependency Graph
|-- Adafruit CCS811 Library @ 1.0.0
|   |-- Wire @ 1.0
|-- Adafruit MAX31865 library @ 1.0.1
|   |-- SPI @ 1.0
|-- Adafruit MCP9808 Library @ 1.1.2
|   |-- Wire @ 1.0
|-- Adafruit SGP30 Sensor @ 1.2.0
|   |-- Wire @ 1.0
|-- Adafruit SGP40 Sensor @ 1.1.0
|   |-- Adafruit BusIO @ 1.11.0
|   |   |-- Wire @ 1.0
|   |   |-- SPI @ 1.0
|-- Adafruit VEML7700 Library @ 1.0.0
|   |-- Adafruit BusIO @ 1.11.0
|   |   |-- Wire @ 1.0
|   |   |-- SPI @ 1.0
|   |-- Wire @ 1.0
|-- Adafruit seesaw Library @ 1.3.1
|   |-- Wire @ 1.0
|-- ESP8266Audio @ 1.9.5
|   |-- ESP8266HTTPClient @ 1.2
|   |   |-- ESP8266WiFi @ 1.0
|   |-- SD(esp8266) @ 2.0.0
|   |   |-- SDFS @ 0.1.0
|   |   |   |-- SPI @ 1.0
|   |   |   |-- ESP8266SdFat @ 1.1.0
|   |   |   |   |-- SPI @ 1.0
|   |-- SPI @ 1.0
|-- BME8563
|   |-- Wire @ 1.0
|-- EEPROM 24C128_256_521
|   |-- Wire @ 1.0
|-- I2Cdevlib-Core
|   |-- Wire @ 1.0
|-- LibTeleinfo @ 1.1.3
|-- MPU6050 I2C
|   |-- I2Cdevlib-Core
|   |   |-- Wire @ 1.0
|-- Mutichannel_Gas_Sensor @ 0.0.1
|   |-- Wire @ 1.0
|-- NTP Library
|   |-- ESP8266WiFi @ 1.0
|-- OpenTherm Library @ 0.9.0
|-- Process_control @ 1.0.0
|-- SevenSegmentTM1637 @ 1.1.0
|-- UdpListener @ 1.0
|-- Unishox Compressor Decompressor highly customized and optimized for ESP8266 and Tasmota @ 1.0
|-- VL53L0X @ 1.0.2
|   |-- Wire @ 1.0
|-- VL53L1X @ 1.0.1
|   |-- Wire @ 1.0
|-- TLS mini client derived from Arduino @ 1.0
|   |-- ESP8266WiFi @ 1.0
|   |-- BearSSL @ 0.6
|-- C2Programmer @ 1.0.0
|-- headers
|-- cc1101 @ 1.0
|   |-- SPI @ 1.0
|-- arduino-mcp2515-1.0.1
|   |-- SPI @ 1.0
|-- A4988_Stepper @ 0.0.1
|-- Adafruit Fingerprint Sensor Library @ 2.0.4
|   |-- TasmotaSerial @ 3.6.0
|-- Adafruit GFX Library @ 1.5.6
|   |-- SPI @ 1.0
|-- Adafruit LED Backpack Library @ 1.1.6
|   |-- Adafruit GFX Library @ 1.5.6
|   |   |-- SPI @ 1.0
|   |-- Wire @ 1.0
|-- Adafruit SH1106-gemu-1.0 @ 1.0
|   |-- Display renderer @ 1.0
|   |   |-- Adafruit GFX Library @ 1.5.6
|   |   |   |-- SPI @ 1.0
|   |   |-- SPI @ 1.0
|   |-- Wire @ 1.0
|-- Adafruit SSD1306 @ 1.3.0
|   |-- Adafruit GFX Library @ 1.5.6
|   |   |-- SPI @ 1.0
|   |-- SPI @ 1.0
|   |-- Wire @ 1.0
|   |-- Display renderer @ 1.0
|   |   |-- Adafruit GFX Library @ 1.5.6
|   |   |   |-- SPI @ 1.0
|   |   |-- SPI @ 1.0
|-- Adafruit SSD1331 OLED Driver Library for Arduino @ 1.2.0
|   |-- Adafruit GFX Library @ 1.5.6
|   |   |-- SPI @ 1.0
|   |-- SPI @ 1.0
|   |-- Display renderer @ 1.0
|   |   |-- Adafruit GFX Library @ 1.5.6
|   |   |   |-- SPI @ 1.0
|   |   |-- SPI @ 1.0
|-- Adafruit TSL2591
|   |-- Wire @ 1.0
|-- Arduino ST7789 Library @ 0.9.5
|   |-- Adafruit GFX Library @ 1.5.6
|   |   |-- SPI @ 1.0
|   |-- Display renderer @ 1.0
|   |   |-- Adafruit GFX Library @ 1.5.6
|   |   |   |-- SPI @ 1.0
|   |   |-- SPI @ 1.0
|   |-- SPI @ 1.0
|-- DNSServer @ 1.1.1
|   |-- ESP8266WiFi @ 1.0
|-- DnsClient @ 1.0
|   |-- ESP8266WiFi @ 1.0
|-- ESP8266HTTPClient @ 1.2
|   |-- ESP8266WiFi @ 1.0
|-- ESP8266SAM @ 1.0.1
|   |-- ESP8266Audio @ 1.9.5
|   |   |-- ESP8266HTTPClient @ 1.2
|   |   |   |-- ESP8266WiFi @ 1.0
|   |   |-- SD(esp8266) @ 2.0.0
|   |   |   |-- SDFS @ 0.1.0
|   |   |   |   |-- SPI @ 1.0
|   |   |   |   |-- ESP8266SdFat @ 1.1.0
|   |   |   |   |   |-- SPI @ 1.0
|   |   |-- SPI @ 1.0
|-- ESP8266WebServer @ 1.0
|   |-- ESP8266WiFi @ 1.0
|-- ESP8266WiFi @ 1.0
|-- ESP8266httpUpdate @ 1.3
|   |-- ESP8266HTTPClient @ 1.2
|   |   |-- ESP8266WiFi @ 1.0
|   |-- ESP8266WiFi @ 1.0
|-- ESP8266mDNS @ 1.2
|   |-- ESP8266WiFi @ 1.0
|-- FT5206_Library @ 1.0.0
|   |-- Wire @ 1.0
|-- FrogmoreScd30
|   |-- Wire @ 1.0
|-- FrogmoreScd40
|   |-- Wire @ 1.0
|-- ILI9341 @ 1.0.0
|   |-- SPI @ 1.0
|   |-- Display renderer @ 1.0
|   |   |-- Adafruit GFX Library @ 1.5.6
|   |   |   |-- SPI @ 1.0
|   |   |-- SPI @ 1.0
|-- IRremoteESP8266 @ 2.8.4
|-- JSMN JSON parser customized and optimized for ESP8266 and Tasmota @ 1.0
|-- KeeloqLib @ 1.1
|-- TasmotaLList @ 1.0
|-- LOLIN_HP303B @ 1.0.0
|   |-- SPI @ 1.0
|   |-- Wire @ 1.0
|-- LedControl @ 1.0.6
|-- LinkedList
|-- LiquidCrystal_I2C
|   |-- Wire @ 1.0
|-- LittleFS(esp8266) @ 0.1.0
|-- MFRC522 @ 1.4.7
|   |-- SPI @ 1.0
|-- MLX90640
|   |-- Wire @ 1.0
|-- MPU_accel
|   |-- Wire @ 1.0
|-- NeoPixelBus @ 2.6.7
|   |-- SPI @ 1.0
|-- NewPing @ 1.9.1
|-- OneWire @ 2.3.2
|-- PubSubClient @ 2.8
|-- RA8876 @ 1.0.2
|   |-- SPI @ 1.0
|   |-- Display renderer @ 1.0
|   |   |-- Adafruit GFX Library @ 1.5.6
|   |   |   |-- SPI @ 1.0
|   |   |-- SPI @ 1.0
|-- rc-switch @ 1.0.0
|-- RF24 @ 1.3.3
|   |-- SPI @ 1.0
|-- Ext-printf @ 1.0
|-- SD(esp8266) @ 2.0.0
|   |-- SDFS @ 0.1.0
|   |   |-- SPI @ 1.0
|   |   |-- ESP8266SdFat @ 1.1.0
|   |   |   |-- SPI @ 1.0
|-- SPI @ 1.0
|-- SSD3115 @ 1.0
|   |-- Display renderer @ 1.0
|   |   |-- Adafruit GFX Library @ 1.5.6
|   |   |   |-- SPI @ 1.0
|   |   |-- SPI @ 1.0
|   |-- SPI @ 1.0
|-- ESP8266SdFat @ 1.1.0
|   |-- SPI @ 1.0
|-- TM1638plus @ 1.7.0
|-- TasmotaModbus @ 3.6.0
|   |-- TasmotaSerial @ 3.6.0
|-- TasmotaSerial @ 3.6.0
|-- Ticker @ 1.0
|-- Joba_Tsl2561 @ 2.0.10
|   |-- Wire @ 1.0
|-- Wire @ 1.0
|-- XPT2046_Touchscreen
|   |-- SPI @ 1.0
|-- base64 @ 1.1.1
|-- BME68x
|-- Waveshare esp 2.9 inch e-paper display driver @ 1.0
|   |-- Display renderer @ 1.0
|   |   |-- Adafruit GFX Library @ 1.5.6
|   |   |   |-- SPI @ 1.0
|   |   |-- SPI @ 1.0
|-- Waveshare esp 4.2 inch e-paper display driver @ 1.0
|   |-- Display renderer @ 1.0
|   |   |-- Adafruit GFX Library @ 1.5.6
|   |   |   |-- SPI @ 1.0
|   |   |-- SPI @ 1.0
|-- Display renderer @ 1.0
|   |-- Adafruit GFX Library @ 1.5.6
|   |   |-- SPI @ 1.0
|   |-- SPI @ 1.0
|-- ESP KNX IP Library @ 0.5.2
|   |-- EEPROM @ 1.0
|   |-- ESP8266WebServer @ 1.0
|   |   |-- ESP8266WiFi @ 1.0
|   |-- ESP8266WiFi @ 1.0
|-- HPMA115S0 Arduino Library @ 1.0.0
|-- stm32_flash @ 1.0.0
|-- BearSSL @ 0.6
|-- universal display Library @ 0.1
|   |-- Adafruit GFX Library @ 1.5.6
|   |   |-- SPI @ 1.0
|   |-- SPI @ 1.0
|   |-- Wire @ 1.0
|   |-- Display renderer @ 1.0
|   |   |-- Adafruit GFX Library @ 1.5.6
|   |   |   |-- SPI @ 1.0
|   |   |-- SPI @ 1.0
|   |-- SSD3115 @ 1.0
|   |   |-- Display renderer @ 1.0
|   |   |   |-- Adafruit GFX Library @ 1.5.6
|   |   |   |   |-- SPI @ 1.0
|   |   |   |-- SPI @ 1.0
|   |   |-- SPI @ 1.0
Building in release mode
ModuleNotFoundError: No module named 'zopfli':
  File "/usr/local/lib/python3.7/dist-packages/platformio/builder/main.py", line 195:
    env.SConscript(item, exports="env")
  File "/root/.platformio/packages/tool-scons/scons-local-4.4.0/SCons/Script/SConscript.py", line 597:
    return _SConscript(self.fs, *files, **subst_kw)
  File "/root/.platformio/packages/tool-scons/scons-local-4.4.0/SCons/Script/SConscript.py", line 285:
    exec(compile(scriptdata, scriptname, 'exec'), call_stack[-1].globals)
  File "/tmp/Tasmota/pio-tools/gzip-firmware.py", line 33:
    from zopfli.gzip import compress
========================= [FAILED] Took 34.60 seconds =========================

Environment    Status    Duration
-------------  --------  ------------
tasmota4M      FAILED    00:00:34.601
==================== 1 failed, 0 succeeded in 00:00:34.601 ====================
Finished. Exit code: 1.

@tdurieux
Copy link
Author

That error was due to --no-install-recommends not due to --no-cache-dir.

With only --no-cache-dir, I succeed to compile with the config provided in #320 but the size reduction is much smaller (29mb).

image

image

image

@Jason2866
Copy link
Contributor

My experience, trying to use cache with Platformio is a lottery game. Been there.
After removing every cache ALL weird issues we had where solved permanently.
So for Tasmota we will never try again "cache"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@tdurieux @benzino77 @Jason2866