Update deps #24

auouymous · 2023-01-05T11:06:49Z

@elelay Do you know how to fix this issue on line 10 in bundle.sh?

Warning, library not available in any prefix: /Users/runner/work/gpodder-osx-bundle/gpodder-osx-bundle/_home/.new_local/share/pyenv/versions/3.10.2/lib/libpython3.10.dylib
Warning, library not available in any prefix: /usr/local/opt/gdbm/lib/libgdbm_compat.4.dylib
Warning, library not available in any prefix: /usr/local/opt/gdbm/lib/libgdbm.6.dylib
Traceback (most recent call last):
Warning, library not available in any prefix: /usr/local/opt/gdbm/lib/libgdbm_compat.4.dylib
Warning, library not available in any prefix: /usr/local/opt/gdbm/lib/libgdbm.6.dylib
Warning, library not available in any prefix: /Users/runner/work/gpodder-osx-bundle/gpodder-osx-bundle/_home/.new_local/share/pyenv/versions/3.10.2/lib/libpython3.10.dylib
Warning, library not available in any prefix: /usr/local/opt/gdbm/lib/libgdbm_compat.4.dylib
  File "/Users/runner/work/gpodder-osx-bundle/gpodder-osx-bundle/_home/.new_local/bin/gtk-mac-bundler", line 6, in <module>
    bundler.main.main(sys.argv[1:])
  File "/Users/runner/work/gpodder-osx-bundle/gpodder-osx-bundle/_bundler/bundler/main.py", line 18, in main
    bundler.run()
  File "/Users/runner/work/gpodder-osx-bundle/gpodder-osx-bundle/_bundler/bundler/bundler.py", line 494, in run
    path.copy_target(self.project)
  File "/Users/runner/work/gpodder-osx-bundle/gpodder-osx-bundle/_bundler/bundler/project.py", line 194, in copy_target
Warning, library not available in any prefix: /usr/local/opt/gdbm/lib/libgdbm.6.dylib
Warning, library not available in any prefix: /usr/local/opt/gdbm/lib/libgdbm_compat.4.dylib
Warning, library not available in any prefix: /usr/local/opt/gdbm/lib/libgdbm.6.dylib
    source = self.compute_source_path(project)
  File "/Users/runner/work/gpodder-osx-bundle/gpodder-osx-bundle/_bundler/bundler/project.py", line 187, in compute_source_path
    raise ValueError("Cannot find source to copy: " + source)
ValueError: Cannot find source to copy: /Users/runner/work/gpodder-osx-bundle/gpodder-osx-bundle/_home/gtk/inst/lib/python3.10/site-packages/pip

elelay · 2023-01-06T20:39:37Z

Seems like pip from pyenv gets in the way during the install of python3 in the destination path

2023-01-05T12:06:35.9641110Z Requirement already satisfied: pip in /Users/runner/work/gpodder-osx-bundle/gpodder-osx-bundle/_home/.new_local/lib/python3.10/site-packages (22.3.1)

Then it's not found when packaging.

elelay · 2023-01-06T20:40:13Z

Do you know why modulesets/upstream-ref is not updated?

auouymous · 2023-01-06T21:55:06Z

Do you know why modulesets/upstream-ref is not updated?

Because I manually updated each moduleset file, then I saw upstream-ref and found fetch_modules.sh. It wasn't a waste of my time because gtk-osx.modules has -DCMAKE_DISABLE_FIND_PACKAGE_BrotliDec=TRUE on two packages so they compile on github, and there are a couple stray xml closing tags in that upstream file.

--- a/modulesets-stable/gtk-osx.modules
+++ b/modulesets-stable/gtk-osx.modules
@@ -184,7 +184,7 @@
 
   <meson id="harfbuzz" mesonargs="-Dcoretext=enabled -Dfreetype=enabled -Ddocs=disabled -Dbenchmark=disabled -Dintrospection=enabled">
     <branch repo="github-tarball" version="4.1.0"
-            module="harfbuzz/harfbuzz/releases/download/4.1.0/harfbuzz-4.1.0.tar.xz">
+            module="harfbuzz/harfbuzz/releases/download/4.1.0/harfbuzz-4.1.0.tar.xz"
             hash="sha256:f7984ff4241d4d135f318a93aa902d910a170a8265b7eaf93b5d9a504eed40c8">
     </branch>
     <dependencies>
@@ -285,7 +285,7 @@
 
   <meson id="graphene" mesonargs="-Dtests=false">
     <branch repo="github-tarball" version="1.10.6"
-            module="ebassi/graphene/releases/download/1.10.6/graphene-1.10.6.tar.xz"/>
+            module="ebassi/graphene/releases/download/1.10.6/graphene-1.10.6.tar.xz"
             hash="sha256:80ae57723e4608e6875626a88aaa6f56dd25df75024bd16e9d77e718c3560b25"/>
     <dependencies>
       <dep package="glib"/>
@@ -338,7 +338,7 @@
       <dep package="freetype"/>
     </dependencies>
     <after>
-<!-- Circular dependency if we depend on meta-gtk-osx-gtk3 -->
+          <!-- Circular dependency if we depend on meta-gtk-osx-gtk3 -->
       <dep package="gtk+-3.0"/>
       <dep package="gtk-4"/>
     </after>

eunikolsky · 2023-02-23T17:35:00Z

@auouymous, re gpodder/gpodder#1348 (comment), I have the pip error when building this branch locally.

eunikolsky · 2023-02-24T06:11:09Z

I tried ./fetch_modules.sh and building from that. There is a build error:

…
*** Configuring harfbuzz-no-cairo *** [19/39]
meson --prefix /Users/u/proj/py/gpodder-osx-bundle/_home/gtk/inst --libdir lib -Dcoretext=enabled -Dfreetype=enabled -Ddocs=disabled -Dbenchmark=disabled -Dintrospection=disabled --wrap-mode=nofallback /Users/u/proj/py/gpodder-osx-bundle/_home/gtk/source/harfbuzz-4.1.0
The Meson build system
…
Run-time dependency freetype2 found: NO (tried pkgconfig, framework and cmake)
Not looking for a fallback subproject for the dependency freetype2 because:
Use of fallback dependencies is disabled.

../../../../gtk/source/harfbuzz-4.1.0/meson.build:87:0: ERROR: Dependency 'freetype2' is required but not found.
…

Even though freetype-no-harfbuzz was built before that and _home/gtk/inst/lib/libfreetype.dylib exists. There is some weirdness with circular dependencies: https://gitlab.gnome.org/GNOME/gtk-osx/-/blob/master/modulesets-stable/gtk-osx.modules#L124-129, https://github.com/harfbuzz/harfbuzz/blob/main/meson.build#L175-L178.

This is so complicated.

eunikolsky · 2023-02-25T06:36:26Z

In the meson build log, I found the more specific error message:

Package libbrotlidec was not found in the pkg-config search path.
Perhaps you should add the directory containing `libbrotlidec.pc'
to the PKG_CONFIG_PATH environment variable
Package 'libbrotlidec', required by 'freetype2', not found

Looking back in the logs, I stumbled upon this:

*** Configuring freetype-no-harfbuzz *** [17/39]
…
-- Found BrotliDec: /usr/local/include
…
*** Building freetype-no-harfbuzz *** [17/39]
ninja
[42/43] Linking C shared library libfreetype.6.18.1.dylib
ld: warning: dylib (/usr/local/lib/libbrotlidec.dylib) was built for newer macOS version (12.0) than being linked (10.9)
ld: warning: dylib (/usr/local/lib/libbrotlidec.dylib) was built for newer macOS version (12.0) than being linked (10.9)
[43/43] Creating library symlink libfreetype.6.dylib libfreetype.dylib

Yes, it found libbrotlidec.dylib installed by brew. /facepalm

@auouymous is correct in #24 about disabling the brotli dependency. Related links: https://gitlab.gnome.org/GNOME/gtk-osx/-/issues/62, https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=964185.

eunikolsky · 2023-02-25T07:42:34Z

So I ended up with the same pip error as in this PR. If I comment copying that directory and then rsync _build/pythonbase.app/Contents/Resources/{lib,share} into the released gPodder 3.11.1 package (to update the shared libraries), the window has the correct size, but the fonts are blurry. Basically, the same result as in gpodder/gpodder#1348 (comment). I have no idea how to fix that.
Can we release gPodder 3.11.1 using the application package from gPodder 3.10.21? Maybe that would fix the UI bugs?

auouymous · 2023-03-04T06:42:10Z

Thank you for those links, I will try removing my brotli patch and see if github builds. It probably won't fix the pip issue but will remove the need for the patch.

As for rolling back to the 3.10.21 bundle, changes were made for yt-dlp to work, and there is a change to fix a syncing issue. I don't know if the 3.10.21 bundle will build or not, but the last working commit used for 3.11.0 no longer builds. Even if the 3.10.21 bundle builds or can easily be made to build, it would require a gtk-osx update to make yt-dlp work.

auouymous · 2023-03-04T14:45:49Z

Seems that disabling brotli is still required. And latest gtk-osx fails to build libxml2, so no magic fix for the pip issue yet.

eunikolsky · 2023-03-04T18:11:18Z

As for rolling back to the 3.10.21 bundle, changes were made for yt-dlp to work, and there is a change to fix a syncing issue. I don't know if the 3.10.21 bundle will build or not, but the last working commit used for 3.11.0 no longer builds. Even if the 3.10.21 bundle builds or can easily be made to build, it would require a gtk-osx update to make yt-dlp work.

@auouymous, this makes sense in general, some updated packages may be required. However why would yt-dlp require gtk-osx?! In any case, in my opinion, the first priority is to make sure the basic functionality of gPodder works out of the box (correct window size and non-blurry text), and the second priority is to make sure the bundled extensions work (not everyone needs yt-dlp).

For reference, my update branch: https://github.com/eunikolsky/gpodder-osx-bundle/tree/latest_moduleset.

auouymous · 2023-03-05T03:35:28Z

I didn't add support for yt-dlp in the bundle, I just remember the bundle being updated to make yt-dlp work on Mac. Also keep in mind that not all Mac users have the hidpi issue, and they might not agree with removing the yt-dlp support they depend on.

As for https://github.com/eunikolsky/gpodder-osx-bundle/tree/latest_moduleset, I don't know why it would fail to download python with curl, but maybe it is because you used fetch_modules.py to download 2023-02-14 (https://github.com/eunikolsky/gpodder-osx-bundle/blob/latest_moduleset/modulesets/upstream-ref) but https://github.com/eunikolsky/gpodder-osx-bundle/blob/latest_moduleset/bootstrap.sh#L18 is still pinned to 2023-01-03. I'm rebuilding this PR with your patch to ignore pip...

One of my recent commits in this PR pinned the repo in fetch_modules.py and added a comment to sync with the pin in bootstrap.py. Once this is working it might be a good idea to add a file sourced by both that holds the hash for gtk-osx and bundler, and mention that file in the readme.

auouymous · 2023-07-07T02:01:31Z

It finally built!

@elelay Does this work? I had to remove the Tango theme, is that an issue?

If this works, I will try putting pip back in, clean up the patches and get a gpodder build for testing.

elelay · 2023-07-09T15:32:04Z

Hi,
sorry, I took too long to get the artifacts and they seem to be gone. I've triggered a re-run and will try to test it.

elelay · 2023-07-12T18:05:44Z

@auouymous the re module is not available!
In launcher.py import platform fails because of that
Even after working around this in launcher.py, still re is not available when running run-python

elelay · 2023-07-12T18:28:56Z

This is because re is now implemented as a folder with init.py, etc. vs re.py in previous version.
It should be added in https://github.com/auouymous/gpodder-osx-bundle/blob/update-deps/misc/bundle/pythonbase.bundle#L121

Here are build logs grepped using grep -P 'Creating .+gtk/inst/lib/python3.11/[^/]+$' for lib/python3.11 subdirectories:

- for directories explicitely removed in pythonbase.bundle
MISSING for directories not listed in pythonbase.bundle, re, tomllib, zoneinfo should be added, maybe also xmlrpc (412kB)
distutils, ensurepip, venv I'm not sure. They should certainly be explicitly marked ignored
__phello__ I don't know. It's to check that frozen modules work if I understand correctly

lib/python3.11/asyncio
lib/python3.11/collections
lib/python3.11/concurrent
lib/python3.11/config-3.11-darwin
lib/python3.11/ctypes
lib/python3.11/curses
- lib/python3.11/dbm
MISSING lib/python3.11/distutils
lib/python3.11/email
lib/python3.11/encodings
MISSING lib/python3.11/ensurepip
lib/python3.11/html
lib/python3.11/http
- lib/python3.11/idlelib
lib/python3.11/importlib
lib/python3.11/json
- lib/python3.11/lib2to3
lib/python3.11/logging
lib/python3.11/multiprocessing
- lib/python3.11/pydoc_data
MISSING lib/python3.11/re
lib/python3.11/sqlite3
lib/python3.11/test
-lib/python3.11/tkinter
MISSING lib/python3.11/tomllib
-lib/python3.11/turtledemo
lib/python3.11/unittest
lib/python3.11/urllib
- lib/python3.11/venv
lib/python3.11/wsgiref
lib/python3.11/xml
MISSING lib/python3.11/xmlrpc
MISSING lib/python3.11/zoneinfo
MISSING lib/python3.11/__phello__

auouymous · 2023-07-19T09:23:41Z

I added the missing modules, can you test again when it finishes building?

elelay · 2023-07-23T11:29:29Z

It works but pip3 is missing

auouymous · 2023-07-25T09:55:10Z

@elelay Do you know how to make it install pipenv 2022.11.4 or newer. I don't know if it would fix the failure, but the runtime instrospection warnings are caused by an older version that doesn't fully support python 3.11.

elelay · 2023-07-26T19:01:31Z

https://gitlab.gnome.org/GNOME/gtk-osx/-/blob/master/gtk-osx-setup.sh#L109 chooses the version.

elelay · 2023-07-26T20:41:18Z

nice try :-)
I'm rebuilding locally on Catalina to see if I can spot the issue.
Don't expect results until Tomorrow night

elelay · 2023-07-27T06:09:44Z

Yeah, so when python3 install bootstraps pip by calling python3 -E -m ensurepip --root xxxx it finds the one in _home/.new_local and says nothing is to be done.
the environment variable PYTHONUSERBASE defined by jhbuild is at fault.

elelay · 2023-07-27T06:23:23Z

Can you create runit.sh with following content and executable?

#!/bin/sh
unset PYTHONUSERBASE
python3 -E -m ensurepip --root /Users/jojolapin/gpodder-osx-bundle/_home/gtk/inst --default-pip

then call jhbuild run ./runit.sh at the end of build.sh? In my install it successfully installed pip in gtk/inst/bin

auouymous · 2023-07-27T07:45:36Z

build.sh is failing, so runit.sh is not called.

elelay · 2023-07-27T16:51:24Z

maybe rollback 59ebd8d?
It caused the failed build.sh because meson is not installed/found, right?

elelay · 2023-07-27T23:19:50Z

My mistake: the --root .... argument should not be there. See

WARNING: The scripts pip, pip3 and pip3.11 are installed in '/Users/runner/work/gpodder-osx-bundle/gpodder-osx-bundle/_home/gtk/inst/Users/runner/work/gpodder-osx-bundle/gpodder-osx-bundle/_home/gtk/inst/bin' which is not on PATH.

in the logs.

Please replace in runit.sh with:

python3 -E -m ensurepip --default-pip

auouymous · 2023-07-28T03:23:12Z

That worked!

auouymous · 2023-10-02T08:05:55Z

@elelay Does this build work any better?

elelay · 2023-10-02T16:54:41Z

I need to rebuild to access artifacts. If this build doesn't work, do you mind pushing to a branch in gpodder-osx-bundle ? Maybe then I access your artifacts?

auouymous · 2023-10-02T20:56:17Z

I added you to my fork, hopefully that gives you access to future artifacts there.

elelay · 2023-10-03T17:33:40Z

YES!
Finally it works 🥳

elelay · 2023-10-03T17:35:18Z

I was able to create a package from gpodder, I just had to s/3.9/3.11/ in release_on_macosx.sh .
So I'm ready to merge and tag a new version of gpodder-osx-bundle shortly thereafter.

auouymous · 2023-10-03T21:51:26Z

I'll clean up the commits and try to get a gpodder build out today for others to test! 🎆

The pip sed is no longer needed. Add comment to explain the other sed.

Remove CircleCI builds.

84fab41

auouymous mentioned this pull request Feb 23, 2023

Retina window scaling bug (double size/half content) on macOS gpodder/gpodder#1348

Open

auouymous force-pushed the update-deps branch from c69465a to 77a7956 Compare March 4, 2023 09:20

auouymous marked this pull request as draft March 4, 2023 10:02

auouymous force-pushed the update-deps branch from 77a7956 to 2772e6a Compare March 4, 2023 10:25

auouymous force-pushed the update-deps branch from 2ca3bab to f1b8b94 Compare July 6, 2023 13:00

elelay closed this Jul 26, 2023

elelay reopened this Jul 26, 2023

auouymous added 11 commits October 3, 2023 16:07

Disable brotli when building freetype.

8cb50a0

Pin build to macos 11.

b6262df

Update to actions/checkout@v3.

ec52c3c

Add pin.sh to manage pinned gtk-osx and gtk-mac-bundler commits.

63790d2

Clean up bootstrap.sh.

bf3cda8

The pip sed is no longer needed. Add comment to explain the other sed.

Install pip.

b3b1935

Remove Tango theme.

2806b46

Use python 3.11.

4afc23f

Add missing python modules.

acc2898

Update modulesets.

acf02eb

env.sh does not need to be executable.

b32c2ad

auouymous force-pushed the update-deps branch from cab6fb4 to b32c2ad Compare October 3, 2023 22:54

Format gpodder.modules to match style in other modulesets.

d961662

auouymous marked this pull request as ready for review October 4, 2023 00:53

auouymous merged commit 0aedaf1 into gpodder:master Oct 4, 2023
1 check failed

auouymous deleted the update-deps branch October 4, 2023 00:55

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update deps #24

Update deps #24

auouymous commented Jan 5, 2023

elelay commented Jan 6, 2023

elelay commented Jan 6, 2023

auouymous commented Jan 6, 2023

eunikolsky commented Feb 23, 2023

eunikolsky commented Feb 24, 2023

eunikolsky commented Feb 25, 2023

eunikolsky commented Feb 25, 2023

auouymous commented Mar 4, 2023

auouymous commented Mar 4, 2023

eunikolsky commented Mar 4, 2023

auouymous commented Mar 5, 2023

auouymous commented Jul 7, 2023

elelay commented Jul 9, 2023

elelay commented Jul 12, 2023

elelay commented Jul 12, 2023

auouymous commented Jul 19, 2023

elelay commented Jul 23, 2023

auouymous commented Jul 25, 2023

elelay commented Jul 26, 2023

elelay commented Jul 26, 2023

elelay commented Jul 27, 2023

elelay commented Jul 27, 2023

auouymous commented Jul 27, 2023

elelay commented Jul 27, 2023

elelay commented Jul 27, 2023 •

edited

Loading

auouymous commented Jul 28, 2023

auouymous commented Oct 2, 2023

elelay commented Oct 2, 2023

auouymous commented Oct 2, 2023

elelay commented Oct 3, 2023

elelay commented Oct 3, 2023

auouymous commented Oct 3, 2023

Update deps #24

Update deps #24

Conversation

auouymous commented Jan 5, 2023

elelay commented Jan 6, 2023

elelay commented Jan 6, 2023

auouymous commented Jan 6, 2023

eunikolsky commented Feb 23, 2023

eunikolsky commented Feb 24, 2023

eunikolsky commented Feb 25, 2023

eunikolsky commented Feb 25, 2023

auouymous commented Mar 4, 2023

auouymous commented Mar 4, 2023

eunikolsky commented Mar 4, 2023

auouymous commented Mar 5, 2023

auouymous commented Jul 7, 2023

elelay commented Jul 9, 2023

elelay commented Jul 12, 2023

elelay commented Jul 12, 2023

auouymous commented Jul 19, 2023

elelay commented Jul 23, 2023

auouymous commented Jul 25, 2023

elelay commented Jul 26, 2023

elelay commented Jul 26, 2023

elelay commented Jul 27, 2023

elelay commented Jul 27, 2023

auouymous commented Jul 27, 2023

elelay commented Jul 27, 2023

elelay commented Jul 27, 2023 • edited Loading

auouymous commented Jul 28, 2023

auouymous commented Oct 2, 2023

elelay commented Oct 2, 2023

auouymous commented Oct 2, 2023

elelay commented Oct 3, 2023

elelay commented Oct 3, 2023

auouymous commented Oct 3, 2023

elelay commented Jul 27, 2023 •

edited

Loading