Elias Steyaert changes for google cloud #28

EliasSteyaert · 2024-04-30T13:06:43Z

No description provided.

Adding the ETN file into my github

… the making of the DATA folder)

… the server

… from

…thon_interpreter variable

…ment to python3

MaybeJustJames · 2024-05-31T12:17:14Z

internships/ETN-Elias-howest/test

delete this

MaybeJustJames · 2024-05-31T12:17:44Z

usr/bin/argument-per-line

@@ -1,5 +1,4 @@
 #!/bin/bash
-


Add this line back in

MaybeJustJames · 2024-05-31T12:18:57Z

usr/bin/irbuild

@@ -66,70 +66,70 @@ if [ "$BUILD" ]; then

    # Import any manifest information.

-    if [ -e "$DATA/irefindex_manifest" ] && ! "$TOOLS/irdata_database_action.py" "$DATABASE" "$SQL/import_irefindex_manifest.sql" "$DATA" $PSQL_OPTIONS ; then
+    if [ -e "$DATA/irefindex_manifest" ] && ! "${USE_PYTHON_INTERPRETER}" "$TOOLS/irdata_database_action.py" "$DATABASE" "$SQL/import_irefindex_manifest.sql" "$DATA" $PSQL_OPTIONS ; then


Suggested change

if [ -e "$DATA/irefindex_manifest" ] && ! "${USE_PYTHON_INTERPRETER}" "$TOOLS/irdata_database_action.py" "$DATABASE" "$SQL/import_irefindex_manifest.sql" "$DATA" $PSQL_OPTIONS ; then

if [ -e "$DATA/irefindex_manifest" ] && ! "$TOOLS/irdata_database_action.py" "$DATABASE" "$SQL/import_irefindex_manifest.sql" "$DATA" $PSQL_OPTIONS ; then

MaybeJustJames · 2024-05-31T12:20:07Z

usr/bin/irdata-config

@@ -138,7 +145,7 @@ BHF_UCL_DOWNLOAD_FILES="bhfucl_psicquic.txt"
 BHF_UCL_DIR_NAME="BHF-UCL"
 BHF_UCL_OTHER_FILE_PATTERN='*.txt'
 BHF_UCL_PARSER='irdata_parse_bhfucl.sh'
-BHF_UCL_IMPORT_TEMPLATE='import_mitab.sql'
+BHF_UCL_IMPORT_TEMPLATE='import_mitab.sql'  


Suggested change

BHF_UCL_IMPORT_TEMPLATE='import_mitab.sql'

BHF_UCL_IMPORT_TEMPLATE='import_mitab.sql'

MaybeJustJames · 2024-05-31T12:20:25Z

usr/bin/irdata-config

@@ -761,7 +768,7 @@ fi
 # Data directory.

 SYSDATA="$SYSPREFIX/var/lib/irdata/data"
-
+echo "marker1 $DATA"


Suggested change

echo "marker1 $DATA"

MaybeJustJames · 2024-05-31T12:24:08Z

usr/bin/irdata_parse_refseq.sh

+# Log the value of $1
+echo "main_script: \$1 is $1" > /home/irefindex/logfile.log
+


Suggested change

# Log the value of $1

echo "main_script: \$1 is $1" > /home/irefindex/logfile.log

MaybeJustJames · 2024-05-31T12:24:47Z

usr/bin/irdata_parse_uniprot.sh

@@ -65,8 +68,21 @@ for FILENAME in $FILENAMES; do

        # Split the data file into $PROCESS pieces.
        # Pass each process its rank (0-based) and the total number of processes.
+        echo "can I see this echo, which is situated in my uniprotscript" >> /home/irefindex/logfile2.log


Suggested change

echo "can I see this echo, which is situated in my uniprotscript" >> /home/irefindex/logfile2.log

MaybeJustJames · 2024-05-31T12:25:20Z

usr/bin/irdata_split_irefscape_data.sh

Delete this file

MaybeJustJames · 2024-05-31T12:26:05Z

usr/bin/irparallel

Go back to how it was

MaybeJustJames · 2024-05-31T12:26:27Z

usr/bin/irparse

Go back to how it was

Adding week 4 to the ETN

adding week 5 and 6

Getting the code blocks cleaner when knitted

…iable usage

deleting an empty file

…sks/data instead of the home directory (through the postgresql.conf)

adding new info to the ETN up to wednesday 12/06/2024

minor changes

making a README file

minor changes

…rrors

Updated the ETN for the evaluation

updating the planning

EliasSteyaert and others added 30 commits April 24, 2024 16:55

test

9b1080b

ETN-EliasSteyaert-stage.Rmd

3c156d4

Adding the ETN file into my github

adding a file with the google cloud IP adress in

df9ecab

changing some directories

6b708cd

correcting the usage of a variable

70a1a10

adding a new variable DATAPREFIX

aaca91b

Merge branch 'master' of github.com:EliasSteyaert/stage-VIB-irefindex

f644d90

correcting a directory

cecf7e1

adding 2 markers, adding and updating (still needs automatisation for…

b5f8e4f

… the making of the DATA folder)

updating the data_dir variable and creating an empty existing_dirs array

c7fa1b4

a new variable data2

22fa4ad

commenting the jobs that can't be run

85aa31f

adding code to create a venv

26fbb9e

adding the USE_PYTHON_INTERPRETER variable

f100291

adjusting the variable USE_PYTHON_INTERPRETER

794f69c

Update ETN-EliasSteyaert-stage.Rmd

5fa8ac1

changed the cloning the github part to copying my own files to use on…

8971164

… the server

adding an extra backslash (since it doesn't matter)

c6a2983

added an echo message to see where the USE_PYTHON_INTERPRETER sourced…

648b8b3

… from

changing a backslash (minor unimportant detail)

ff3321d

Merge branch 'master' of github.com:EliasSteyaert/stage-VIB-irefindex

c925c35

added the python_interpreter as an environment

ad00f23

trying to find a solution for the error msgs

d84ca62

Added code to put the pyhton interpreter before in the scripts

7080461

adding the python interpreter before the

a6c7c99

adding a sudo_command and markers

f94320f

putting things in commands because they don't work anymore and use_py…

4e04620

…thon_interpreter variable

New block of code to give the scripts the execute privileges

5557e0e

making it more user friendly

fe7eb38

adding a block of code to install pip and changing the python environ…

31514ce

…ment to python3

EliasSteyaert added 2 commits May 29, 2024 09:46

making changes to gete irparallel working with the script

d143e16

drop

f6adfd1

MaybeJustJames changed the title ~~Initial steps to run on google cloud~~ Elias Steyaert changes for google cloud May 31, 2024

MaybeJustJames requested changes May 31, 2024

View reviewed changes

EliasSteyaert and others added 26 commits May 31, 2024 16:58

removing the interpreter

86040ee

Update ETN-EliasSteyaert-stage.Rmd

03a44d6

Adding week 4 to the ETN

Update ETN-EliasSteyaert-stage.Rmd

03951d0

adding week 5 and 6

Merge branch 'master' of github.com:EliasSteyaert/stage-VIB-irefindex

9cdd8f4

Update ETN-EliasSteyaert-stage.Rmd

27ddbc4

Getting the code blocks cleaner when knitted

Merge branch 'master' of github.com:EliasSteyaert/stage-VIB-irefindex

140171f

minor adjustments to find the ongoing probleme, which was a wrong var…

682bbb7

…iable usage

the .ansible logs were going onto the / disk again

b51fd24

minor adjustment to figure out if the parsing worked

d75b2ae

Delete internships/ETN-Elias-howest/test

2e62588

deleting an empty file

Changing code so the data from postgresql gets written to the /mnt/di…

335da85

…sks/data instead of the home directory (through the postgresql.conf)

Merge branch 'master' of github.com:EliasSteyaert/stage-VIB-irefindex

de192ad

removing echos that were added to understand a problem better

918a47e

adjusting so the .async_dir doesn't write on the / disk again

e73343e

adding a download_data variable

84e9489

minor or no changes when trying to find out how irimport really works

a6a3e43

Update ETN-EliasSteyaert-stage.Rmd

60a73bf

adding new info to the ETN up to wednesday 12/06/2024

Update ETN-EliasSteyaert-stage.Rmd

d14366d

minor changes

Create README.md

0410b79

making a README file

Update README.md

0b384b9

minor changes

extra adjustments so that the path of the data gets changed without e…

df8f727

…rrors

Update ETN-EliasSteyaert-stage.Rmd

50d0581

Updated the ETN for the evaluation

Update ETN-EliasSteyaert-stage.Rmd

894d7bc

updating the planning

pushing the final versions of the playbooks

668f7f1

Merge branch 'master' of github.com:EliasSteyaert/stage-VIB-irefindex

ddc0790

deleting commented lines

ed76b1f

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Elias Steyaert changes for google cloud #28

Elias Steyaert changes for google cloud #28

EliasSteyaert commented Apr 30, 2024

MaybeJustJames May 31, 2024

MaybeJustJames May 31, 2024

MaybeJustJames May 31, 2024

MaybeJustJames May 31, 2024

MaybeJustJames May 31, 2024

MaybeJustJames May 31, 2024

MaybeJustJames May 31, 2024

MaybeJustJames May 31, 2024

MaybeJustJames May 31, 2024

MaybeJustJames May 31, 2024

	if [ -e "$DATA/irefindex_manifest" ] && ! "${USE_PYTHON_INTERPRETER}" "$TOOLS/irdata_database_action.py" "$DATABASE" "$SQL/import_irefindex_manifest.sql" "$DATA" $PSQL_OPTIONS ; then
	if [ -e "$DATA/irefindex_manifest" ] && ! "$TOOLS/irdata_database_action.py" "$DATABASE" "$SQL/import_irefindex_manifest.sql" "$DATA" $PSQL_OPTIONS ; then

	BHF_UCL_IMPORT_TEMPLATE='import_mitab.sql'
	BHF_UCL_IMPORT_TEMPLATE='import_mitab.sql'

		# Log the value of $1
		echo "main_script: \$1 is $1" > /home/irefindex/logfile.log

Elias Steyaert changes for google cloud #28

Are you sure you want to change the base?

Elias Steyaert changes for google cloud #28

Conversation

EliasSteyaert commented Apr 30, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment