Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[absolete]Connectors' markers implementation -- 010, 060, 069 #157

Closed
wants to merge 12 commits into from
Closed

[absolete]Connectors' markers implementation -- 010, 060, 069 #157

Changes from 1 commit
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
e5d36f3
DF/10: Add EOM/EOP options
anastasiakaida Aug 10, 2018
ffe85f7
DF/10: Add EOM/EOP in usage()
anastasiakaida Aug 10, 2018
0d943b7
DF/10: $EOM/$EOP usage
anastasiakaida Aug 10, 2018
a033b60
DF/10: EOM/EOP mode dependencies
anastasiakaida Aug 10, 2018
d73cc39
DF/10: 'ERROR' level formalization
anastasiakaida Aug 10, 2018
087d379
DF/60: usage() changes
anastasiakaida Aug 10, 2018
be8c7f6
DF/60: read line fix
anastasiakaida Aug 10, 2018
af8e53b
DF/60: custom EOP implementation
anastasiakaida Aug 10, 2018
317d1c9
DF/60: custom EOB implementation
anastasiakaida Aug 10, 2018
aa7f30a
DF/69: custom EOP/EOB implementation
anastasiakaida Aug 10, 2018
99d39c1
DF/60: bug fixes
anastasiakaida Aug 14, 2018
8cfbb65
DF/60: fix reading EOB marker
anastasiakaida Aug 14, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 12 additions & 3 deletions Utils/Dataflow/060_upload2virtuoso/uploadTTL.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,8 @@ OPTIONS:
}

upload_files () {

EOProcess=""

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

EOProcess is explicitly defined in both upload_files and upload_stream. How user value is supposed to be used?

if [ -z "$1" ] ; then
echo "(ERROR) Input file is not specified." >&2
usage
Expand Down Expand Up @@ -119,7 +120,8 @@ upload_files () {
;;
esac

eval "$cmd" || { echo "(ERROR) An error occured while uploading file: $INPUTFILE" >&2; continue; }
eval "$cmd" || { echo "(ERROR) An error occured while uploading file: $INPUTFILE" >&2; } &
echo -ne "$EOProcess"
Copy link
Collaborator

@mgolosova mgolosova Aug 13, 2018 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happened here?
"custom EOP implementation" makes me think that the answer should be "Custom EOP marker output added". But the diff says:

  • continue directive removed;
  • eval "$cmd" || echo is sent to background;
  • custom EOP output added.

What are the first two changes are made for?

Update: by the way, I see I made a comment to this change in #130; now I see addressed only the part concerning continue (and it was done quite silently, as if anyone reading this commit must know about our conversation).
If you think the continue directive should be deleted -- explain it in commit log; if you just follow my faint comment "I`m not sure if it is needed here..." -- then do not. Ask why it is not needed. Then think, does this change belong to this commit or not. Please, do not try to guess what do I expect you to do... if you know -- do it and show that you understand what and why was done. If not -- ask, and ask again if needed, and do it till you understand the idea and can explain even better than me.
What about everything else I wrote there?


done

Expand All @@ -128,6 +130,8 @@ upload_files () {
}

upload_stream () {
EOProcess="\0"

local delimiter=$'\n'

[ -z "$TYPE" ] && { echo "(ERROR) input data format is not specified. Exiting." >&2; return 2;}
Expand Down Expand Up @@ -170,7 +174,7 @@ upload_stream () {
n=`ps axf | grep 'curl' | grep "$HOST:$PORT" | grep -v 'grep' | wc -l`
done
echo "$line" | $cmd &>/dev/null || { echo "(ERROR) An error occured while uploading stream data." >&2; continue; } &
echo -n $'\6'
echo -ne "$EOProcess"
done
done
}
Expand Down Expand Up @@ -209,6 +213,10 @@ do
DELIMITER=`echo -e $2`
shift
;;
-E|--eop)
EOP="$2"
shift
;;
-u|--user)
USER="$2"
shift
Expand Down Expand Up @@ -238,6 +246,7 @@ done
[ -z "$GRAPH" ] && GRAPH=http://$HOST:$PORT/$GRAPH_PATH
[ -z "$DELIMITER" ] && DELIMITER=$'\0'
[ "x$DELIMITER" = "xNOT SPECIFIED" ] && DELIMITER=$'\n'
[ -n "$EOP" ] && EOProcess="$EOP"

cmdTTL="curl --retry 3 -s -f -X POST --digest -u $USER:$PASSWD -H Content-Type:text/turtle -G http://$HOST:$PORT/sparql-graph-crud-auth --data-urlencode graph=$GRAPH"
cmdSPARQL="curl --retry 3 -s -f -H 'Accept: text/csv' -G http://$HOST:$PORT/sparql --data-urlencode query"
Expand Down