Crawl Images issue

[altomator]

Hi all,

I'm trying out Vikus after seeing it demonstrated at the IIIF Napoli conference last June.

When I run the "Generate instance" on the default collection:
https://iiif.wellcomecollection.org/presentation/collections/genres/Stickers

or any of mine, like this one:
https://mirador.bnf.fr/?iiif-content=https://raw.githubusercontent.com/altomator/IIIF/main/collection/collection-bd.json

it failed at the CrawlImages step:
[2023-11-18 17:24:00 +0000] [685] [INFO] connection open
vikus-iiif-generator-main-vikusdocker-1 | this function is a coroutine: crawlCollection
vikus-iiif-generator-main-redis-1 | 1:M 18 Nov 2023 17:24:33.956 * 1 changes in 3600 seconds. Saving...
vikus-iiif-generator-main-redis-1 | 1:M 18 Nov 2023 17:24:33.960 * Background saving started by pid 20
vikus-iiif-generator-main-redis-1 | 20:C 18 Nov 2023 17:24:33.980 * DB saved on disk
vikus-iiif-generator-main-redis-1 | 20:C 18 Nov 2023 17:24:33.981 * Fork CoW for RDB: current 1 MB, peak 1 MB, average 0 MB
vikus-iiif-generator-main-redis-1 | 1:M 18 Nov 2023 17:24:34.067 * Background saving terminated with success
vikus-iiif-generator-main-vikusdocker-1 | crawlCollection took 2.0 seconds
vikus-iiif-generator-main-vikusdocker-1 | this function is a coroutine: crawlImages
vikus-iiif-generator-main-vikusdocker-1 | crawlImages took 0.0023 seconds
vikus-iiif-generator-main-vikusdocker-1 | this function is a coroutine: makeSpritesheets
vikus-iiif-generator-main-vikusdocker-1 | ['/modules/sharpsheet/bin/sharpsheet', '/data/juicy-omelette/images/thumbs/.jpg', '--outputPath', '/data/juicy-omelette/images/sprites', '--outputFormat', 'jpg', '--outputQuality', '60', '--sheetDimension', '2048', '--spriteSize', '224']
vikus-iiif-generator-main-vikusdocker-1 | 17:24:36 INFO INFO:rich:stdout: sharpsheet.py:42
vikus-iiif-generator-main-vikusdocker-1 | looking for images at
vikus-iiif-generator-main-vikusdocker-1 | /data/juicy-omelette/images/thumbs/.jpg
vikus-iiif-generator-main-vikusdocker-1 | done
vikus-iiif-generator-main-vikusdocker-1 |
vikus-iiif-generator-main-vikusdocker-1 | ERROR ERROR:rich:error generating spritesheet: no sharpsheet.py:45
vikus-iiif-generator-main-vikusdocker-1 | images found
vikus-iiif-generator-main-vikusdocker-1 |
vikus-iiif-generator-main-vikusdocker-1 | makeSpritesheets took 0.18 seconds
vikus-iiif-generator-main-vikusdocker-1 | this function is a coroutine: makeFeatures
vikus-iiif-generator-main-vikusdocker-1 | 0/0 images found in cache
vikus-iiif-generator-main-vikusdocker-1 | makeFeatures took 1.2 seconds
vikus-iiif-generator-main-vikusdocker-1 | this function is a coroutine: makeUmap
vikus-iiif-generator-main-vikusdocker-1 | 17:24:39 INFO INFO:umaper:Fit UMAP dimensionReduction.py:30
vikus-iiif-generator-main-vikusdocker-1 | INFO INFO:umaper:X shape: (0,) dimensionReduction.py:32
vikus-iiif-generator-main-vikusdocker-1 | [2023-11-18 17:24:39 +0000] [572] [ERROR] Exception in ASGI application
vikus-iiif-generator-main-vikusdocker-1 | Traceback (most recent call last):
vikus-iiif-generator-main-vikusdocker-1 | File "/usr/local/lib/python3.8/site-packages/uvicorn/protocols/http/httptools_impl.py", line 426, in run_asgi
vikus-iiif-generator-main-vikusdocker-1 | result = await app( # type: ignore[func-returns-value]
vikus-iiif-generator-main-vikusdocker-1 | File "/usr/local/lib/python3.8/site-packages/uvicorn/middleware/proxy_headers.py", line 84, in call
vikus-iiif-generator-main-vikusdocker-1 | return await self.app(scope, receive, send)
vikus-iiif-generator-main-vikusdocker-1 | File "/usr/local/lib/python3.8/site-packages/fastapi/applications.py", line 1106, in call
vikus-iiif-generator-main-vikusdocker-1 | await super().call(scope, receive, send)
vikus-iiif-generator-main-vikusdocker-1 | File "/usr/local/lib/python3.8/site-packages/starlette/applications.py", line 122, in call
vikus-iiif-generator-main-vikusdocker-1 | await self.middleware_stack(scope, receive, send)
vikus-iiif-generator-main-vikusdocker-1 | File "/usr/local/lib/python3.8/site-packages/starlette/middleware/errors.py", line 184, in call
vikus-iiif-generator-main-vikusdocker-1 | raise exc
vikus-iiif-generator-main-vikusdocker-1 | File "/usr/local/lib/python3.8/site-packages/starlette/middleware/errors.py", line 162, in call
vikus-iiif-generator-main-vikusdocker-1 | await self.app(scope, receive, send)
vikus-iiif-generator-main-vikusdocker-1 | File "/usr/local/lib/python3.8/site-packages/starlette/middleware/cors.py", line 91, in call
vikus-iiif-generator-main-vikusdocker-1 | await self.simple_response(scope, receive, send, request_headers=headers)
vikus-iiif-generator-main-vikusdocker-1 | File "/usr/local/lib/python3.8/site-packages/starlette/middleware/cors.py", line 146, in simple_response
vikus-iiif-generator-main-vikusdocker-1 | await self.app(scope, receive, send)
vikus-iiif-generator-main-vikusdocker-1 | File "/usr/local/lib/python3.8/site-packages/starlette/middleware/exceptions.py", line 79, in call
vikus-iiif-generator-main-vikusdocker-1 | raise exc
vikus-iiif-generator-main-vikusdocker-1 | File "/usr/local/lib/python3.8/site-packages/starlette/middleware/exceptions.py", line 68, in call
vikus-iiif-generator-main-vikusdocker-1 | await self.app(scope, receive, sender)
vikus-iiif-generator-main-vikusdocker-1 | File "/usr/local/lib/python3.8/site-packages/fastapi/middleware/asyncexitstack.py", line 20, in call
vikus-iiif-generator-main-vikusdocker-1 | raise e
vikus-iiif-generator-main-vikusdocker-1 | File "/usr/local/lib/python3.8/site-packages/fastapi/middleware/asyncexitstack.py", line 17, in call
vikus-iiif-generator-main-vikusdocker-1 | await self.app(scope, receive, send)
vikus-iiif-generator-main-vikusdocker-1 | File "/usr/local/lib/python3.8/site-packages/starlette/routing.py", line 718, in call
vikus-iiif-generator-main-vikusdocker-1 | await route.handle(scope, receive, send)
vikus-iiif-generator-main-vikusdocker-1 | File "/usr/local/lib/python3.8/site-packages/starlette/routing.py", line 276, in handle
vikus-iiif-generator-main-vikusdocker-1 | await self.app(scope, receive, send)
vikus-iiif-generator-main-vikusdocker-1 | File "/usr/local/lib/python3.8/site-packages/starlette/routing.py", line 66, in app
vikus-iiif-generator-main-vikusdocker-1 | response = await func(request)
vikus-iiif-generator-main-vikusdocker-1 | File "/usr/local/lib/python3.8/site-packages/fastapi/routing.py", line 274, in app
vikus-iiif-generator-main-vikusdocker-1 | raw_response = await run_endpoint_function(
vikus-iiif-generator-main-vikusdocker-1 | File "/usr/local/lib/python3.8/site-packages/fastapi/routing.py", line 191, in run_endpoint_function
vikus-iiif-generator-main-vikusdocker-1 | return await dependant.call(**values)
vikus-iiif-generator-main-vikusdocker-1 | File "/scripts/main.py", line 411, in run
vikus-iiif-generator-main-vikusdocker-1 | await make_umap(
vikus-iiif-generator-main-vikusdocker-1 | File "/scripts/main.py", line 351, in make_umap
vikus-iiif-generator-main-vikusdocker-1 | await makeUmap(
vikus-iiif-generator-main-vikusdocker-1 | File "/scripts/helpers.py", line 37, in wrapper
vikus-iiif-generator-main-vikusdocker-1 | result = await helper(func, *args, **kwargs)
vikus-iiif-generator-main-vikusdocker-1 | File "/scripts/helpers.py", line 29, in helper
vikus-iiif-generator-main-vikusdocker-1 | return await func(*args, **kwargs)
vikus-iiif-generator-main-vikusdocker-1 | File "/scripts/vikus.py", line 208, in makeUmap
vikus-iiif-generator-main-vikusdocker-1 | embedding = umaper.fit_transform(features)
vikus-iiif-generator-main-vikusdocker-1 | File "/scripts/dimensionReduction.py", line 33, in fit_transform
vikus-iiif-generator-main-vikusdocker-1 | embeddings = self.umap.fit_transform(X)
vikus-iiif-generator-main-vikusdocker-1 | File "/usr/local/lib/python3.8/site-packages/umap/umap.py", line 2887, in fit_transform
vikus-iiif-generator-main-vikusdocker-1 | self.fit(X, y, force_all_finite)
vikus-iiif-generator-main-vikusdocker-1 | File "/usr/local/lib/python3.8/site-packages/umap/umap_.py", line 2354, in fit
vikus-iiif-generator-main-vikusdocker-1 | X = check_array(X, dtype=np.float32, accept_sparse="csr", order="C", force_all_finite=force_all_finite)
vikus-iiif-generator-main-vikusdocker-1 | File "/usr/local/lib/python3.8/site-packages/sklearn/utils/validation.py", line 938, in check_array
vikus-iiif-generator-main-vikusdocker-1 | raise ValueError(
vikus-iiif-generator-main-vikusdocker-1 | ValueError: Expected 2D array, got 1D array instead:
vikus-iiif-generator-main-vikusdocker-1 | array=[].
vikus-iiif-generator-main-vikusdocker-1 | Reshape your data either using array.reshape(-1, 1) if your data has a single feature or array.reshape(1, -1) if it contains a single sample.

[cpietsch]

Hi @altomator,
I will have a look at this issue. It looks like the api at wellcomecollection differs from the time the code was created.

[altomator]

Thanks a lot!But I failed too to use it on my IIIF collections.All the best,JpEnvoyé de mon iPhoneLe 24 nov. 2023 à 11:55, Christopher Pietsch ***@***.***> a écrit : Hi @altomator, I will have a look at this issue. It looks like the api at wellcomecollection differs from the time the code was created. —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>