Problems upgrading from a custom 1.11.1 to 1.14.12 #688

GauteR · 2023-06-29T11:24:29Z

GauteR
Jun 29, 2023

Hi, at NLB in Norway we are now trying to upgrade our version of Daisy Pipeline, but we're having some issues with the newest version, in terms of it not working at all. We are using a modified version of 1.11.1 which works fine, but the newest version (1.14.12) has some features we would like to implement into our solution.

I am running it locally and getting the following errors on the attached file:

<?xml version='1.0' encoding='UTF-8'?>
<jobRequest xmlns="http://www.daisy.org/ns/pipeline/data">
  <priority>medium</priority>
  <script href="http://localhost:8282/ws/scripts/html-to-pef"/>
  <option name="html">185259.html</option>
  <option name="stylesheet">braille.scss</option>
  <option name="stylesheet-parameters">(skip-margin-top-of-page:true)</option>
  <option name="transform">(formatter:dotify)(translator:liblouis)(dots:6)(grade:0)</option>
  <option name="page-width">38</option>
  <option name="page-height">29</option>
  <option name="duplex">true</option>
  <option name="hyphenation">false</option>
  <option name="line-spacing">single</option>
  <option name="include-production-notes">true</option>
  <option name="toc-depth">2</option>
  <option name="maximum-number-of-sheets">50</option>
  <option name="allow-volume-break-inside-leaf-section-factor">10</option>
  <option name="prefer-volume-break-before-higher-level-factor">1</option>
</jobRequest>

2023-06-29 10:58:50,818 [INFO ] com.xmlcalabash.runtime.XViewport - jar:file:/opt/daisy-pipeline2/system/common/org.daisy.pipeline.modules.braille.html-to-pef-6.0.1_backup.jar!/xml/xproc/html-to-pef.convert.xpl:153:86:Transforming MathML
2023-06-29 10:58:50,849 [INFO ] com.xmlcalabash.runtime.XOtherwise - jar:file:/opt/daisy-pipeline2/system/common/org.daisy.pipeline.modules.braille.html-to-pef-6.0.1_backup.jar!/xml/xproc/html-to-pef.convert.xpl:248:80:Transforming from XML with inline CSS to PEF
2023-06-29 10:58:50,850 [DEBUG] com.xmlcalabash.runtime.XTry - jar:file:/opt/daisy-pipeline2/system/common/org.daisy.pipeline.modules.braille.html-to-pef-6.0.1_backup.jar!/xml/xproc/html-to-pef.convert.xpl:257:130:px:transform query=(input:css)(output:pef)(formatter:dotify)(translator:liblouis)(dots:6)(grade:0)(document-locale:no)
2023-06-29 10:58:50,854 [DEBUG] o.d.p.b.c.AbstractTransformProvider - No match for query (translator:liblouis)(dots:6)(grade:0)(document-locale:no)(input:text-css)(output:braille)
2023-06-29 10:58:50,860 [DEBUG] o.d.p.common.calabash.impl.LogError - Could not find a Transform for query: (input:css)(output:pef)(formatter:dotify)(translator:liblouis)(dots:6)(grade:0)(document-locale:no)
        at html-to-pef.convert.xpl:265
        at html-to-pef.convert.xpl:258
        at html-to-pef.convert.xpl:257
        at html-to-pef.convert.xpl:248
        at html-to-pef.convert.xpl:169
        at __processed__html-to-pef.xpl:665
2023-06-29 10:58:50,860 [ERROR] o.d.p.common.calabash.impl.LogError - Could not find a Transform for query: (input:css)(output:pef)(formatter:dotify)(translator:liblouis)(dots:6)(grade:0)(document-locale:no) (Please see detailed log for more info.)
2023-06-29 10:58:50,860 [ERROR] com.xmlcalabash.runtime.XAtomicStep - jar:file:/opt/daisy-pipeline2/system/common/org.daisy.pipeline.modules.braille.html-to-pef-6.0.1_backup.jar!/xml/xproc/html-to-pef.convert.xpl:306:128:Failed to convert XML with inline CSS to PEF

It's responding no match for query, but I cannot seem to find what querys are allowed anywhere. Removing parts of it, does not work either.

This is the file that we're trying to convert (it has already been validated and passed through our older version, but just won't pass through on this. If we can determine what's wrong with it, we can change our production system accordingly, but right now the errors we get does not provide enough information.
Example file

Braille.scss

Any help with solving this would be greatly appreciated.

Answered by bertfrees

Sep 14, 2023

The issue was resolved by using a Docker image built from my latest local development version (1.14.15-SNAPSHOT). It probably indeed had to do with the encoding related bug that I fixed locally.

We still don't know what is the difference between the ZIP file that NLB's production system creates and the one that I created in my test. When the former is used, the conversion fails, but when the latter is used, the conversion succeeds (both with the 1.14.14 Docker image).

But the most important is that the issue was fixed :-)

View full answer

bertfrees · 2023-07-03T19:45:03Z

bertfrees
Jul 3, 2023
Maintainer

Hi Gaute, feel free to send your example file by email, that way we can use it for development purposes, but it is not available for the rest of the world.

I will do some testing and get back to you.

0 replies

bertfrees · 2023-07-12T08:26:54Z

bertfrees
Jul 12, 2023
Maintainer

I was able to reproduce your issue with version 1.14.11. But not with the latest version, 1.14.14. This was probably a bug that I fixed, although I don't remember when I did that. Would it be possible to try 1.14.14? Thanks.

0 replies

GauteR · 2023-07-31T09:31:34Z

GauteR
Jul 31, 2023
Author

It does get past the error in 1.14.14, but another error occurs. Not sure where the error originates from.

<?xml version="1.0" encoding="UTF-8"?>
<error xmlns="http://www.daisy.org/ns/pipeline/data" query="http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.4.1">
    <description>Value 'false' not accepted for option 'hyphenation' of script 'html-to-pef' 'false' is not in the list of allowed values.</description>
    <trace>java.lang.Throwable: Value 'false' not accepted for option 'hyphenation' of script 'html-to-pef' 'false' is not in the list of allowed values.
        at org.daisy.pipeline.webservice.impl.GenericResource.getErrorRepresentation(GenericResource.java:40)
        at org.daisy.pipeline.webservice.impl.JobsResource.badRequest(JobsResource.java:220)
        at org.daisy.pipeline.webservice.impl.JobsResource.createResource(JobsResource.java:184)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at org.restlet.resource.ServerResource.doHandle(ServerResource.java:503)
        at org.restlet.resource.ServerResource.post(ServerResource.java:1215)
        at org.restlet.resource.ServerResource.doHandle(ServerResource.java:592)
        at org.restlet.resource.ServerResource.doNegotiatedHandle(ServerResource.java:649)
        at org.restlet.resource.ServerResource.doConditionalHandle(ServerResource.java:348)
        at org.restlet.resource.ServerResource.handle(ServerResource.java:951)
        at org.restlet.resource.Finder.handle(Finder.java:246)
        at org.restlet.routing.Filter.doHandle(Filter.java:159)
        at org.restlet.routing.Filter.handle(Filter.java:206)
        at org.restlet.routing.Router.doHandle(Router.java:431)
        at org.restlet.routing.Router.handle(Router.java:648)
        at org.restlet.routing.Filter.doHandle(Filter.java:159)
        at org.restlet.routing.Filter.handle(Filter.java:206)
        at org.restlet.routing.Filter.doHandle(Filter.java:159)
        at org.restlet.routing.Filter.handle(Filter.java:206)
        at org.restlet.routing.Filter.doHandle(Filter.java:159)
        at org.restlet.engine.application.StatusFilter.doHandle(StatusFilter.java:155)
        at org.restlet.routing.Filter.handle(Filter.java:206)
        at org.restlet.routing.Filter.doHandle(Filter.java:159)
        at org.restlet.routing.Filter.handle(Filter.java:206)
        at org.restlet.engine.CompositeHelper.handle(CompositeHelper.java:211)
        at org.restlet.engine.application.ApplicationHelper.handle(ApplicationHelper.java:84)
        at org.restlet.Application.handle(Application.java:381)
        at org.restlet.routing.Filter.doHandle(Filter.java:159)
        at org.restlet.routing.Filter.handle(Filter.java:206)
        at org.restlet.routing.Router.doHandle(Router.java:431)
        at org.restlet.routing.Router.handle(Router.java:648)
        at org.restlet.routing.Filter.doHandle(Filter.java:159)
        at org.restlet.routing.Filter.handle(Filter.java:206)
        at org.restlet.routing.Router.doHandle(Router.java:431)
        at org.restlet.routing.Router.handle(Router.java:648)
        at org.restlet.routing.Filter.doHandle(Filter.java:159)
        at org.restlet.engine.application.StatusFilter.doHandle(StatusFilter.java:155)
        at org.restlet.routing.Filter.handle(Filter.java:206)
        at org.restlet.routing.Filter.doHandle(Filter.java:159)
        at org.restlet.routing.Filter.handle(Filter.java:206)
        at org.restlet.engine.CompositeHelper.handle(CompositeHelper.java:211)
        at org.restlet.Component.handle(Component.java:392)
        at org.restlet.Server.handle(Server.java:516)
        at org.restlet.engine.ServerHelper.handle(ServerHelper.java:72)
        at org.restlet.engine.adapter.HttpServerHelper.handle(HttpServerHelper.java:152)
        at org.restlet.ext.jetty.JettyServerHelper$WrappedServer.handle(JettyServerHelper.java:170)
        at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:851)
        at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
        at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:622)
        at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:46)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:603)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:538)
        at java.base/java.lang.Thread.run(Thread.java:829)
    </trace>
</error>

0 replies

bertfrees · 2023-08-01T09:06:20Z

bertfrees
Aug 1, 2023
Maintainer

Hi Gaute. Every now and then some script options change. That is what happened here. You can use the CLI or check the documentation to know the current list of options and allowed values.

0 replies

GauteR · 2023-08-01T12:09:10Z

GauteR
Aug 1, 2023
Author

Changing it to hyphenation: none gave us the next error:

2023-08-01 12:04:21,437 [DEBUG] com.xmlcalabash.library.DefaultStep - Running {http://www.daisy.org/ns/pipeline/xproc}log-error !1.19.2.1.2.1
2023-08-01 12:04:21,439 [DEBUG] o.d.p.common.calabash.impl.LogError - An element with display: -obfl-toc must have no inline content with not at least one target-counter(), target-string() or target-text() value (that references an element that does not participate in a named flow) within the same inline formatting context.
        at library.xsl:11
        at css-to-obfl.xsl:1320
        at css-to-obfl.xsl:1275
        at css-to-obfl.xsl:1275
        at css-to-obfl.xsl:1049
        at css-to-obfl.xsl:528
        at css-to-obfl.xsl:332
        at css-to-obfl.xsl:285
        at css-to-obfl.xpl:875
        at css-to-obfl.xpl:437
        at dotify-transform.xpl:76
2023-08-01 12:04:21,439 [ERROR] o.d.p.common.calabash.impl.LogError - An element with display: -obfl-toc must have no inline content with not at least one target-counter(), target-string() or target-text() value (that references an element that does not participate in a named flow) within the same inline formatting context. (Please see detailed log for more info.)
2023-08-01 12:04:21,439 [DEBUG] com.xmlcalabash.library.Identity - Running identity result
2023-08-01 12:04:21,439 [ERROR] com.xmlcalabash.runtime.XAtomicStep - jar:file:/opt/daisy-pipeline2/system/common/org.daisy.pipeline.modules.braille.html-to-pef-6.1.0.jar!/xml/xproc/html-to-pef.convert.xpl:308:128:Failed to convert XML with inline CSS to PEF

0 replies

bertfrees · 2023-08-04T17:24:13Z

bertfrees
Aug 4, 2023
Maintainer

I've seen this error but can't remember exactly what caused it. It might be due to an encoding issue (which has been fixed in the upcoming version). How are you running Pipeline? Still with Docker?

1 reply

GauteR Aug 5, 2023
Author

We're using Docker, yes.

bertfrees · 2023-08-05T18:06:54Z

bertfrees
Aug 5, 2023
Maintainer

I can not reproduce the issue with the book that you sent and the options you provided. I run these commands:

docker run -it -e PIPELINE2_WS_HOST=0.0.0.0 -e PIPELINE2_WS_AUTHENTICATION=false -p 8181:8181 daisyorg/pipeline:v1.14.14

dp2 html-to-pef -o ~/Desktop/out --data ~/Desktop/185259.zip \
                --html 185259.html \
                --stylesheet braille-nlb.scss \
                --transform "(formatter:dotify)(translator:liblouis)(dots:6)(grade:0)" \
                --page-width 38 \
                --page-height 29 \
                --toc-depth 2 \
                --maximum-number-of-sheets 50 \
                --include-production-notes true \
                --hyphenation none \
                --allow-volume-break-inside-leaf-section-factor 10 \
                --prefer-volume-break-before-higher-level-factor 1 \
                --stylesheet-parameters "(skip-margin-top-of-page:true)" \
                --line-spacing single \
                --duplex true

1 reply

GauteR Aug 22, 2023
Author

I am using your parameters and am still getting the same error:

2023-08-22 14:02:39,743 [DEBUG] o.d.p.common.calabash.impl.LogError - An element with display: -obfl-toc must have no inline content with not at least one target-counter(), target-string() or target-text() value (that references an element that does not participate in a named flow) within the same inline formatting context.
        at library.xsl:11
        at css-to-obfl.xsl:1320
        at css-to-obfl.xsl:1275
        at css-to-obfl.xsl:1275
        at css-to-obfl.xsl:1049
        at css-to-obfl.xsl:528
        at css-to-obfl.xsl:332
        at css-to-obfl.xsl:285
        at css-to-obfl.xpl:875
        at css-to-obfl.xpl:437
        at dotify-transform.xpl:76
2023-08-22 14:02:39,743 [ERROR] o.d.p.common.calabash.impl.LogError - An element with display: -obfl-toc must have no inline content with not at least one target-counter(), target-string() or target-text() value (that references an element that does not participate in a named flow) within the same inline formatting context. (Please see detailed log for more info.)

I am uncertain what the error message means, can you specify?

As for reproducing the error, maybe something different in the environment?
I am using: Docker Engine v24.0.2 (Windows) if that matters.
With the following config:

{
  "builder": {
    "features": {
      "buildkit": true
    },
    "gc": {
      "defaultKeepStorage": "20GB",
      "enabled": true
    }
  },
  "experimental": false
}

bertfrees · 2023-08-24T17:42:40Z

bertfrees
Aug 24, 2023
Maintainer

I am uncertain what the error message means, can you specify?

In order for the CSS renderer to correctly render the table of contents at the beginning of the volumes, the CSS needs to specified where the TOC entries refer to. This happens through the target-counter() function for instance (this is the case in NLB's CSS). Your CSS file seems correct, but for some reason some or all of this information is lost by the time the CSS renderer needs to do its thing. I don't know why this is happening, this is why it is so important I can reproduce the error.

I can't really imagine that the version of Docker would make a difference.

Could it be the input? Perhaps you could email me the files again, but this time the ZIP file with both the 185259.html and braille-nlb.scss files inside. (For my own tests, I created a ZIP myself from the HTML file that you emailed me and the CSS file that you attached to the issue. Perhaps we're not using exactly the same input.)

0 replies

bertfrees · 2023-09-14T10:11:46Z

bertfrees
Sep 14, 2023
Maintainer

The issue was resolved by using a Docker image built from my latest local development version (1.14.15-SNAPSHOT). It probably indeed had to do with the encoding related bug that I fixed locally.

We still don't know what is the difference between the ZIP file that NLB's production system creates and the one that I created in my test. When the former is used, the conversion fails, but when the latter is used, the conversion succeeds (both with the 1.14.14 Docker image).

But the most important is that the issue was fixed :-)

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Problems upgrading from a custom 1.11.1 to 1.14.12 #688

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 9 comments 2 replies

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

Select a reply

Problems upgrading from a custom 1.11.1 to 1.14.12 #688

GauteR Jun 29, 2023

Replies: 9 comments · 2 replies

bertfrees Jul 3, 2023 Maintainer

bertfrees Jul 12, 2023 Maintainer

GauteR Jul 31, 2023 Author

bertfrees Aug 1, 2023 Maintainer

GauteR Aug 1, 2023 Author

bertfrees Aug 4, 2023 Maintainer

GauteR Aug 5, 2023 Author

bertfrees Aug 5, 2023 Maintainer

GauteR Aug 22, 2023 Author

bertfrees Aug 24, 2023 Maintainer

bertfrees Sep 14, 2023 Maintainer

GauteR
Jun 29, 2023

Replies: 9 comments 2 replies

bertfrees
Jul 3, 2023
Maintainer

bertfrees
Jul 12, 2023
Maintainer

GauteR
Jul 31, 2023
Author

bertfrees
Aug 1, 2023
Maintainer

GauteR
Aug 1, 2023
Author

bertfrees
Aug 4, 2023
Maintainer

GauteR Aug 5, 2023
Author

bertfrees
Aug 5, 2023
Maintainer

GauteR Aug 22, 2023
Author

bertfrees
Aug 24, 2023
Maintainer

bertfrees
Sep 14, 2023
Maintainer