Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

2025-04 dev log #13

Closed
michielbdejong opened this issue Jan 6, 2025 · 78 comments
Closed

2025-04 dev log #13

michielbdejong opened this issue Jan 6, 2025 · 78 comments

Comments

@michielbdejong
Copy link
Owner

michielbdejong commented Jan 6, 2025

(edited to remove TUBS work that I previously/naively thought I would get to this week)

@michielbdejong
Copy link
Owner Author

-> before that, continue from #12 (comment)

@michielbdejong
Copy link
Owner Author

I should set up the redirects for sites like prejournal.org so I can let them expire

@michielbdejong
Copy link
Owner Author

node ./bin/server.js -c config/file-acp.json -f ./.data

@michielbdejong
Copy link
Owner Author

Hm, I can use https://empty-solid-app.pondersource.com/ to access http://localhost:3000/asdf/ and hand-edit ./data/asdf/.acr in VSCode in the meantime and this works well for experimentation.

But if I refresh the page, the client ID rotates:

2025-01-20T09:38:37.343Z [DPoPWebIdExtractor] {Primary} info: Verified WebID via DPoP-bound access token. WebID: https://michielbdejong.solidcommunity.net/profile/card#me, client ID: d240994e277b77a712814f9c5f3d3686, issuer: https://solidcommunity.net
2025-01-20T09:40:14.364Z [DPoPWebIdExtractor] {Primary} info: Verified WebID via DPoP-bound access token. WebID: https://michielbdejong.solidcommunity.net/profile/card#me, client ID: 418834c6b40afe525dc0dba3e43d9c82, issuer: https://solidcommunity.net

I'm also not sure how to specify this client ID in the ACR. Presumably these client ID's are given out by solidcommunity.net over which I have no control.

I'll try using http://localhost:3000 as the IDP so I can see the client ID's getting minted, hopefully.

@michielbdejong
Copy link
Owner Author

2025-01-20T09:44:26.781Z [HandlerServerConfigurator] {Primary} info: Received GET request for /.oidc/auth?
client_id=y5WsXIkZOBsjXQd-3bxpz&
redirect_uri=https%3A%2F%2Fempty-solid-app.pondersource.com&
response_type=code&
scope=openid%20offline_access%20webid&
state=8155c34a37af429a84b50949a8fdaa35&
code_challenge=-xJqFHTjoz-tmhjs7QwEvubvsaf...&
code_challenge_method=S256&
prompt=consent&
response_mode=query

@michielbdejong
Copy link
Owner Author

$ cat .data/.internal/idp/adapter/Client/y5WsXIkZOBsjXQd-3bxpz\$.json 
{"key":"idp/adapter/Client/y5WsXIkZOBsjXQd-3bxpz",
"payload":{"payload":{"application_type":"web",
"grant_types":["authorization_code","refresh_token"],
"id_token_signed_response_alg":"ES256",
"require_auth_time":false,"response_types":["code"],
"subject_type":"public",
"token_endpoint_auth_method":"client_secret_basic",
"post_logout_redirect_uris":[],
"require_pushed_authorization_requests":false,
"dpop_bound_access_tokens":false,
"client_id_issued_at":1737366266,
"client_id":"y5WsXIkZOBsjXQd-3bxpz",
"client_secret_expires_at":0,
"client_secret":"XsFXyOZah0yOe80gzk95Qex0PrR5Cf9ZEimimtIubejxSXScu...",
"redirect_uris":["https://empty-solid-app.pondersource.com"]}}}

@michielbdejong
Copy link
Owner Author

2025-01-20T09:49:04.487Z [DPoPWebIdExtractor] {Primary} warn: Error verifying WebID via DPoP-bound access token:
The access token issuer doesn't match its associated WebID's trusted OIDC issuers.
Actual: 
Expected: http://localhost:3000/

@michielbdejong
Copy link
Owner Author

I wonder whether this is something I messed up or whether this is an incompatibility between empty-solid-app and CSS-ACP.
I'll reset the data and try again, with CSS-WAC as well as with CSS-ACP.

@michielbdejong
Copy link
Owner Author

michielbdejong commented Jan 20, 2025

OK, testing with Penny now, that's much more convenient.

This works:

        acp:allOf [
            a acp:Matcher;
            acp:agent <http://localhost:3000/asdf/profile/card#me>;
            acp:client "nqNPJK2GIxlZwedoKoeuJ"
        ]

@michielbdejong
Copy link
Owner Author

After log-out-and-back-in at Penny, I need to edit it to acp:client "f9fOi9PGx3Bcvoxiewmt3" though. I wonder whether there is another way of doing this.

@michielbdejong
Copy link
Owner Author

OK, so let's work with the assumption that these rotating client ID's are a good thing, and that the launcher app should edit ACRs each time a Solid app is opened in a new browser window.

@michielbdejong
Copy link
Owner Author

How would this work? The user could:

  • first visit the launcher
  • click an app to log into (maybe it receives a nonce)
  • go to the IDP, a client ID is assigned
  • the app could now somehow tell the launcher its client ID (maybe using the nonce) and the folders it needs

Maybe the launcher is not a launcher but an Authorization Server in this case.

Downside would be that the user first have to go app -> IDP -> app to give their identity, and then separately app -> AS -> app to give access to folders.

The other option would be static client ID's which Penny supports (see Matrix), and then the launcher would work as originally intended. maybe that's better.

@michielbdejong
Copy link
Owner Author

michielbdejong commented Jan 20, 2025

OK, now testing the following:

  • Run CSS-ACP on port 3000: node ./bin/server.js -c config/file-acp.json -f ./.data --loggingLevel=debug
  • edit ~/gh/CommunitySolidServer/CommunitySolidServer/.data/asdf/clientid.jsonld
{"@context":"https://www.w3.org/ns/solid/oidc-context.jsonld","client_id":"http://localhost:3000/asdf/clientid.jsonld","redirect_uris":["http://localhost:3001"],"client_name":"Penny Local","token_endpoint_auth_method":"none","grant_types":["refresh_token","authorization_code"],"response_types":["code"],"scope":"openid webid offline_access"}
  • copy README.acr to clientid.jsonld.acr (edit the acp:resource triple)
  • Edit Penny src/functions/connect.ts idpsWithClientIdEnabled to add http://localhost:3000
  • Run Penny on port 3001: export NEXT_PUBLIC_CLIENT_ID=http://localhost:3000/asdf/clientid.jsonld ; npm run dev
  • Now you should be able to add acp:client <http://localhost:3000/asdf/clientid.jsonld> in the root ACR without interrupting Penny access

@michielbdejong
Copy link
Owner Author

You can remove <#publicReadAccess> to make this easier to test. OK, so this works! Great. Next step: create three apps that use a public client ID in the same way:

  • launcher
  • bookmarks app 1
  • bookmarks app 2

@michielbdejong
Copy link
Owner Author

Ah, it looks pretty simple, seems I can just add a parameter clientId here

@michielbdejong
Copy link
Owner Author

Let's see if I can use https://github.com/pdsinterop/launcher-exploration

@michielbdejong
Copy link
Owner Author

Transferred this devlog repo from https://github.com/tubsproject/devlog to https://github.com/michielbdejong/devlog because I realised maybe even TUBS can not be an umbrella for all my work. I did initially put it under TUBS on purpose, to emphasise that Unhosted, Prejournal, Ponder Source, Solid, data portability, Federated Bookkeeping and even post-capitalism should all be glued together into TUBS, and maybe I'll move it back some day, but I did realise two things:

  1. Even though LedgerLoops and TUBS both fall under post-capitalist network software, the data portability aspect of LedgerLoops is not so clear. Some people will want to buy into LedgerLoops without buying into TUBS, and vice versa. It therefore needs to be independent from TUBS.

2)I might want to work on topics that are not post-capitalist data portability, and still weave that activity into my (personal) devlog. The relation between the arrow of time, information theory, and computational complexity is one. Music production is another. There may be others in years to come. ToS;DR is also unrelated.

@michielbdejong
Copy link
Owner Author

michielbdejong commented Jan 20, 2025

@michielbdejong
Copy link
Owner Author

Trying to npm install https://github.com/michielbdejong/poddit with pyenv local 2 but no luck. Trying on GitPod now.

@michielbdejong
Copy link
Owner Author

Fixed the yarn install issue with michielbdejong/poddit@b5497eb

@michielbdejong
Copy link
Owner Author

npm start on poddit now results in:

Starting the development server...

/Users/michiel/gl/vincenttunru/poddit/node_modules/chokidar/lib/fsevents-handler.js:28
  return (new fsevents(path)).on('fsevent', callback).start();
          ^

TypeError: fsevents is not a constructor
    at createFSEventsInstance (/Users/michiel/gl/vincenttunru/poddit/node_modules/chokidar/lib/fsevents-handler.js:28:11)
    at setFSEventsListener (/Users/michiel/gl/vincenttunru/poddit/node_modules/chokidar/lib/fsevents-handler.js:82:16)
    at FsEventsHandler._watchWithFsEvents (/Users/michiel/gl/vincenttunru/poddit/node_modules/chokidar/lib/fsevents-handler.js:252:16)
    at FSWatcher.<anonymous> (/Users/michiel/gl/vincenttunru/poddit/node_modules/chokidar/lib/fsevents-handler.js:386:25)
    at LOOP (node:fs:2752:14)
    at process.processTicksAndRejections (node:internal/process/task_queues:77:11)

Node.js v20.9.0

Maybe i can fix this with a fresh yarn install.

@michielbdejong
Copy link
Owner Author

Hm, build error again:

node-pre-gyp http GET https://fsevents-binaries.s3-us-west-2.amazonaws.com/v1.2.4/fse-v1.2.4-node-v115-darwin-x64.tar.gz
node-pre-gyp http 404 https://fsevents-binaries.s3-us-west-2.amazonaws.com/v1.2.4/fse-v1.2.4-node-v115-darwin-x64.tar.gz
node-pre-gyp WARN Tried to download(404): https://fsevents-binaries.s3-us-west-2.amazonaws.com/v1.2.4/fse-v1.2.4-node-v115-darwin-x64.tar.gz 
node-pre-gyp WARN Pre-built binaries not found for [email protected] and [email protected] (node-v115 ABI, unknown) (falling back to source compile with node-gyp) 
node-pre-gyp http 404 status code downloading tarball https://fsevents-binaries.s3-us-west-2.amazonaws.com/v1.2.4/fse-v1.2.4-node-v115-darwin-x64.tar.gz 
gyp info it worked if it ends with ok
gyp info using [email protected]
gyp info using [email protected] | darwin | x64
gyp info ok 
gyp info it worked if it ends with ok
gyp info using [email protected]
gyp info using [email protected] | darwin | x64
gyp info spawn /Users/michiel/.pyenv/shims/python2
gyp info spawn args [
gyp info spawn args   '/Users/michiel/gl/vincenttunru/poddit/node_modules/node-gyp/gyp/gyp_main.py',
gyp info spawn args   'binding.gyp',
gyp info spawn args   '-f',
gyp info spawn args   'make',
gyp info spawn args   '-I',
gyp info spawn args   '/Users/michiel/gl/vincenttunru/poddit/node_modules/react-scripts/node_modules/fsevents/build/config.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/Users/michiel/gl/vincenttunru/poddit/node_modules/node-gyp/addon.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/Users/michiel/.node-gyp/20.9.0/include/node/common.gypi',
gyp info spawn args   '-Dlibrary=shared_library',
gyp info spawn args   '-Dvisibility=default',
gyp info spawn args   '-Dnode_root_dir=/Users/michiel/.node-gyp/20.9.0',
gyp info spawn args   '-Dnode_gyp_dir=/Users/michiel/gl/vincenttunru/poddit/node_modules/node-gyp',
gyp info spawn args   '-Dnode_lib_file=/Users/michiel/.node-gyp/20.9.0/<(target_arch)/node.lib',
gyp info spawn args   '-Dmodule_root_dir=/Users/michiel/gl/vincenttunru/poddit/node_modules/react-scripts/node_modules/fsevents',
gyp info spawn args   '-Dnode_engine=v8',
gyp info spawn args   '--depth=.',
gyp info spawn args   '--no-parallel',
gyp info spawn args   '--generator-output',
gyp info spawn args   'build',
gyp info spawn args   '-Goutput_dir=.'
gyp info spawn args ]
ERROR:root:code for hash md5 was not found.

According to https://stackoverflow.com/questions/59269208/errorrootcode-for-hash-md5-was-not-found-when-using-any-hg-mercurial-command it may be a problem with the link between python@2 and openssl. Sigh. I'll try on GitPod.

@michielbdejong
Copy link
Owner Author

On GitPod the build fails with:

gyp: Undefined variable standalone_static_library in binding.gyp while trying to load binding.gyp

@michielbdejong
Copy link
Owner Author

Maybe I should npm upgrade the dependencies or use an older node version

@michielbdejong
Copy link
Owner Author

OK that helps a lot, but still getting:

Failed to compile
./node_modules/asynciterator/dist/asynciterator.mjs
Can't import the named export 'EventEmitter' from non EcmaScript module (only default export is available)
This error occurred during the build time and cannot be dismissed.

@michielbdejong
Copy link
Owner Author

The .gitpod-ci.yml file proposes node-10, trying that

@michielbdejong
Copy link
Owner Author

@michielbdejong
Copy link
Owner Author

Hm, not sure why I was seeing a CSS index.html on localhost:3000 but now it works, using GitPod with forwarded port:

Image

@michielbdejong
Copy link
Owner Author

I need to make poddit compatible with pivot

@michielbdejong
Copy link
Owner Author

michielbdejong commented Jan 21, 2025

I'll remove the solid-auth-client dependency and see what that breaks. Maybe also remove the popup.html but that does mean I have to introduce some new UI elements to replace it.

@michielbdejong
Copy link
Owner Author

OK, michielbdejong/poddit@9fe4cd0
Now I need to add the session management from https://github.com/michielbdejong/empty-solid-app into that file.

@michielbdejong
Copy link
Owner Author

I'm using this but not sure if it's idiomatic:

const [webId, setWebId] = useWebId();

@michielbdejong
Copy link
Owner Author

https://react.dev/learn#updating-the-screen suggests yes so I'll use it for the other hooks too then, for consistency.

@michielbdejong
Copy link
Owner Author

I should define useSession instead of useWebId

@michielbdejong
Copy link
Owner Author

ah no, this is not necessary because @inrupt/solid-client-authn-js stores the session and exposes it

@michielbdejong
Copy link
Owner Author

OK, this is annoying, I moved the session state up into the app so that I could pass it down to elements.
But now I find that useBookmarks depends on useWebId, so I still need it there.
Should I keep useWebId as a hook, and then update it from the app? But then how can I have it as a global state?

@michielbdejong
Copy link
Owner Author

In https://dev.to/chaituknag/renderprops-vs-custom-hooks-which-one-to-use-5a0 I'm seeing the pattern of returning just the value from the custom hook, makes sense.

@michielbdejong
Copy link
Owner Author

I wonder about reuse of function useData(url) though, will useEffect know to fetch the data only once? Maybe I can try that out in a small code example.

@michielbdejong
Copy link
Owner Author

@michielbdejong
Copy link
Owner Author

michielbdejong commented Jan 23, 2025

Added a warning to pivot since we want Pivot to run on solidcommunity.net and can't easily switch that server to ACP.

One of my hopes with the migration is that we could demo that we finally have an MVP of Solid working, but turns out it's not enough.
I'll run an ACP-based Pivot config in my demo.

@michielbdejong
Copy link
Owner Author

I think I understand custom hooks now, although handleIncomingRedirect is getting called like 20 times.

I wonder how other Solid - React apps solve that.

and then on to:

Error: Fetcher: TypeError: undefined Failed to execute 'fetch' on 'Window': Illegal invocation

@michielbdejong
Copy link
Owner Author

solidBookmarker has the useEffect in the app component, uses onSessionRestore and also uses React routing - which may make it different?

@michielbdejong
Copy link
Owner Author

@michielbdejong
Copy link
Owner Author

I think I should pass session.fetch to rdflib like this

@michielbdejong
Copy link
Owner Author

Ah, that helps! Next issue: I should add https://github.com/CommunitySolidServer/CommunitySolidServer/blob/main/templates/pod/acp/profile/card.acr to my pod on pivot

@michielbdejong
Copy link
Owner Author

I don't see a pod getting created in /root/pivot/data/michielbdejong on pivot.pondersource.com.
Trying now with updated default config, if that works, switching back from WAC to ACP.

@michielbdejong
Copy link
Owner Author

OK, so the default config works properly. Making this config change and restarting:

diff --git a/config/prod.json b/config/prod.json
index 87be263..87a262e 100644
--- a/config/prod.json
+++ b/config/prod.json
@@ -20,7 +20,7 @@
     "css:config/identity/ownership/token.json",
     "css:config/identity/pod/static.json",
     "css:config/ldp/authentication/dpop-bearer.json",
-    "css:config/ldp/authorization/webacl.json",
+    "css:config/ldp/authorization/acp.json",
     "css:config/ldp/handler/default.json",
     "css:config/ldp/metadata-parser/default.json",
     "css:config/ldp/metadata-writer/default.json",
@@ -29,7 +29,7 @@
     "css:config/storage/key-value/resource-store.json",
     "css:config/storage/location/pod.json",
     "pivot:config/storage/middleware/default.json",
-    "css:config/util/auxiliary/acl.json",
+    "css:config/util/auxiliary/acr.json",
     "css:config/util/identifiers/subdomain.json",
     "css:config/util/logging/winston.json",
     "css:config/util/representation-conversion/default.json",
rm -rf data/
mkdir -p data
cp -r www data/
vim config/prod.json 
npx community-solid-server -c ./config/prod.json ./custom-config.json -f ./data --httpsKey /etc/letsencrypt/live/pivot.pondersource.com/privkey.pem --httpsCert /etc/letsencrypt/live/pivot.pondersource.com/fullchain.pem -p 443 -b https://pivot.pondersource.com -m .

@michielbdejong
Copy link
Owner Author

First thing I notice is that data/www/index.html is not showing. Instead I'm seeing mashlib on https://pivot.pondersource.com.

I got into an infinite redirect because I was still logged in, but I found a trick for removing my cookie: I visit https://pivot.pondersource.com/.well-known/openid-configuration, and then I can go to the icon in my browser addressbar that lets me remove the cookie. After that, the infinite redirect no longer happens.

@michielbdejong
Copy link
Owner Author

Ah, I see the error:

2025-01-23T13:10:35.597Z [BaseResourcesGenerator] {Primary} warn: Ignoring non-existing template folder /root/pivot/node_modules/css-mashlib/templates/pod/acp
2025-01-23T13:10:35.598Z [BaseResourcesGenerator] {Primary} warn: Ignoring non-existing template folder /root/pivot/node_modules/css-mashlib/templates/pod/base

That should be fixable with npm install.

@michielbdejong
Copy link
Owner Author

Ah no, of course, it's because css-mashlib has no acp templates. I have to update the config to use node_modules/@solid/community-server/templates/ instead!

@michielbdejong
Copy link
Owner Author

trying with SolidOS/css-mashlib#12 now

@michielbdejong
Copy link
Owner Author

Adding;

:manualAddition
    a solid:TypeRegistration;
    solid:forClass bookmark:Bookmark;
    solid:instanceContainer </public/bookmarks/>.

copied from https://michielbdejong.solidcommunity.net/settings/publicTypeIndex.ttl

@michielbdejong
Copy link
Owner Author

It's now creating https://asdf.pivot.pondersource.com/public/bookmarks.ttl but not putting any bookmarks into it yet

@michielbdejong
Copy link
Owner Author

michielbdejong commented Jan 23, 2025

It feels very close to working now

Bigger picture for basically all of January: #15

Moving TUBS work to February.

@michielbdejong
Copy link
Owner Author

4 bookmarks in store reported - continue debugging from here tomorrow:
michielbdejong/poddit@b383c80

@michielbdejong
Copy link
Owner Author

I'm noting that Poddit is now getting full access to my pod, but that's something I'll fix later. I will need to add a client ID doc for the launcher on https://pivot.pondersource.com/ and whitelist that, alongside with mashlib, penny for instance, see SolidOS/css-mashlib#12 (comment)
I have to stop somewhere though. making mashlib-on-the-pod work might be a stretch goal or left as an exercise for the reader.

@michielbdejong
Copy link
Owner Author

-> https://gitlab.com/vincenttunru/poddit/-/merge_requests/2
Next: deploy this. To 5apps maybe?

@michielbdejong
Copy link
Owner Author

Continuing in #14

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant