Docker testing environment #183

sstenchlak · 2022-04-06T00:02:30Z

sstenchlak
Apr 6, 2022
Maintainer

Building the image

Currently, you need to build the cli and the image by yourself. GitHub Actions may automate this step in the future.

npm install
echo "BACKEND=https://backend.dataspecer.com" > applications/cli/.env.local
npx lerna bootstrap --scope cli --include-dependencies
npx lerna run build --scope cli --include-dependencies
cd utils/testing-environment
sh ./pre-build.sh
sudo docker build -t dataspecer .

Example use of the environment

Create an empty directory with:

input.xml

<?xml version="1.0" encoding="UTF-8"?>
<tourist_destination>
  <has_contact>
    <e-mail>[email protected]</e-mail>
    <phone>123456789</phone>
  </has_contact>
</tourist_destination>

rdf-to-sparql.xsl

<?xml version="1.0" encoding="UTF-8" ?>
<xsl:transform xmlns="http://www.w3.org/2005/sparql-results#" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">
    <xsl:output method="xml" encoding="utf-8" indent="yes"/>
    <xsl:strip-space elements="*"/>

    <xsl:template match="/rdf:RDF">
        <sparql>
            <head>
                <variable name="s"/>
                <variable name="p"/>
                <variable name="o"/>
            </head>
            <results>
                <xsl:for-each select="rdf:Description">
                    <xsl:call-template name="desc"/>
                </xsl:for-each>
            </results>
        </sparql>
    </xsl:template>

    <xsl:template name="desc">
        <xsl:variable name="id">
            <xsl:apply-templates select="@*"/>
        </xsl:variable>
        <xsl:for-each select="*">
            <xsl:call-template name="prop">
                <xsl:with-param name="subj">
                    <xsl:copy-of select="$id"/>
                </xsl:with-param>
            </xsl:call-template>
        </xsl:for-each>
    </xsl:template>

    <xsl:template name="prop">
        <xsl:param name="subj"/>
        <result>
            <binding name="s">
                <xsl:copy-of select="$subj"/>
            </binding>
            <binding name="p">
                <uri>
                    <xsl:value-of select="concat(namespace-uri(),local-name())"/>
                </uri>
            </binding>
            <binding name="o">
                <xsl:apply-templates select="@rdf:resource|@rdf:nodeID|text()"/>
            </binding>
        </result>
    </xsl:template>

    <xsl:template match="@xml:lang">
        <xsl:copy-of select="."/>
    </xsl:template>
    <xsl:template match="@rdf:datatype">
        <xsl:attribute name="datatype">
            <xsl:value-of select="."/>
        </xsl:attribute>
    </xsl:template>
    <xsl:template match="@rdf:about|@rdf:resource">
        <uri>
            <xsl:value-of select="."/>
        </uri>
    </xsl:template>
    <xsl:template match="@rdf:nodeID">
        <bnode>
            <xsl:value-of select="."/>
        </bnode>
    </xsl:template>
    <xsl:template match="text()">
        <literal>
            <xsl:apply-templates select="parent::*/@xml:lang|parent::*/@rdf:datatype"/>
            <xsl:value-of select="."/>
        </literal>
    </xsl:template>

    <xsl:template match="@*|*"/>
</xsl:transform>

script.sh

dataspecer generate https://ofn.gov.cz/data-specification/c885c417-23f6-4f06-86dd-52e066c310de
pygmentize -g input.xml
xmllint --schema ukazka-cli/tourist-destination/schema.xsd input.xml --noout
xslt3 -xsl:ukazka-cli/tourist-destination/lifting.xslt -s:input.xml -o:rdf_query.xml -t
pygmentize -g rdf_query.xml
xslt3 -xsl:rdf-to-sparql.xsl -s:rdf_query.xml -o:sparql-result.xml -t
pygmentize -g sparql-result.xml
xslt3 -xsl:ukazka-cli/tourist-destination/lowering.xslt -s:sparql-result.xml -o:output.xml -t
pygmentize -g output.xml

Execute the script.sh file in the testing environment and see the results in the current directory.

sudo docker run -i --rm -v "$PWD":/usr/src/app dataspecer < script.sh

jakubklimek · 2022-04-29T06:34:37Z

jakubklimek
Apr 29, 2022
Maintainer

@sstenchlak Currently, when script.sh contains only:

dataspecer generate https://ofn.gov.cz/data-specification/09c09f8f-a2df-4abe-bd2a-9228fb22a474

I get:

klimek@KUBA-1:/mnt/c/Tools/dataspecer-test$ docker run -i --rm -v "$PWD":/usr/src/app dataspecer < script.sh
loading data specifications
dataspecer generate <data-specification-iri>

Generate artifacts from the given data specification.

Options:
  --help         Show help                                             [boolean]
  --backend-url  The URL of the backend to use.           [string] [default: ""]

TypeError [ERR_INVALID_URL]: Invalid URL
    at new NodeError (node:internal/errors:372:5)
    at URL.onParseError (node:internal/url:553:9)
    at new URL (node:internal/url:629:5)
    at BackendConnector.<anonymous> (/bin/dataspecer:277:31)
    at step (/bin/dataspecer:242:23)
    at Object.next (/bin/dataspecer:223:53)
    at /bin/dataspecer:217:71
    at new Promise (<anonymous>)
    at __webpack_modules__.7770.__awaiter (/bin/dataspecer:213:12)
    at BackendConnector.__webpack_modules__.7770.BackendConnector.readDataSpecification (/bin/dataspecer:272:16) {
  input: '/data-specification',
  code: 'ERR_INVALID_URL'
}

1 reply

sstenchlak Apr 29, 2022
Maintainer Author

Hi, have you used the instructions from above? You need to set a backend URL during the build

echo "BACKEND=https://backend.dataspecer.com" > applications/cli/.env.local

In any case, executing dataspecer with --backend-url=https://backend.dataspecer.com should override the empty default value. Sorry for the general error, I will enforce the option if no backend is specified during the build.

jakubklimek · 2022-04-29T07:15:03Z

jakubklimek
Apr 29, 2022
Maintainer

I have...

But yes, with the explicit parameter it works.

1 reply

sstenchlak Apr 29, 2022
Maintainer Author

Sorry for the issue. It should be fixed in e1aab5a.

jakubklimek · 2022-04-29T08:31:26Z

jakubklimek
Apr 29, 2022
Maintainer

Still does not work for me. I tried rebuilding the image with --no-cache and the behavior still seems to be the same.

2 replies

sstenchlak Apr 29, 2022
Maintainer Author

The image does not build the CLI itself. You need to build it manually again.

It should be enough to run npm run build in applications/cli/, execute the pre-build.sh script, and then rebuild the image.

jakubklimek Apr 29, 2022
Maintainer

yes, that was it. It works now.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Docker testing environment #183

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 3 comments 4 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

Select a reply

Docker testing environment #183

sstenchlak Apr 6, 2022 Maintainer

Building the image

Example use of the environment

Replies: 3 comments · 4 replies

jakubklimek Apr 29, 2022 Maintainer

sstenchlak Apr 29, 2022 Maintainer Author

jakubklimek Apr 29, 2022 Maintainer

sstenchlak Apr 29, 2022 Maintainer Author

jakubklimek Apr 29, 2022 Maintainer

sstenchlak Apr 29, 2022 Maintainer Author

jakubklimek Apr 29, 2022 Maintainer

sstenchlak
Apr 6, 2022
Maintainer

Replies: 3 comments 4 replies

jakubklimek
Apr 29, 2022
Maintainer

sstenchlak Apr 29, 2022
Maintainer Author

jakubklimek
Apr 29, 2022
Maintainer

sstenchlak Apr 29, 2022
Maintainer Author

jakubklimek
Apr 29, 2022
Maintainer

sstenchlak Apr 29, 2022
Maintainer Author

jakubklimek Apr 29, 2022
Maintainer