driverless h2oai python pipeline tries to connect to H2O server at port 12345

[cyrilthank]

Hi

I am currently using the h2oai trial for scoping the features.

I am finding that after completing the training process and downloading the scoring pipeline, when i execute the run_example.py it fails while connecting to localhost:12345

Question: does the h2oai server need to be up and running while the run_example.py is executed

Is there a way to workaround this and have run_example.py execute (in isolation) without connecting to the h2o server?

Why is the run_example.py trying to connect to h2o server

[goldentom42]

Hi @cyrilthank,

Thank you for trying our free trial of DriverlessAI.
Could you please tell us or give us a link to the run_example.py script you are trying to execute?

Best regards.

[cyrilthank]

Entire error message is as below

• pm=pip
  ++ pwd
• current_dir=/h2o/scoring-pipeline
  +++ dirname ./run_example.sh
  ++ cd .
  ++ pwd
• script_dir=/h2o/scoring-pipeline
• cd /h2o/scoring-pipeline
• source ./common-functions.sh
• main
• parse_args_and_exec
• '[' '' '!=' '' ']'
• [[ pip == \p\i\p ]]
• run_using_pip
• '[' '!' -d env ']'
• source env/bin/activate
  ++ '[' env/bin/activate = ./run_example.sh ']'
  ++ deactivate nondestructive
  ++ unset -f pydoc
  ++ '[' -z '' ']'
  ++ '[' -z '' ']'
  ++ '[' -n /bin/bash ']'
  ++ hash -r
  ++ '[' -z '' ']'
  ++ unset VIRTUAL_ENV
  ++ '[' '!' nondestructive = nondestructive ']'
  ++ VIRTUAL_ENV=/h2o/scoring-pipeline/env
  ++ export VIRTUAL_ENV
  ++ _OLD_VIRTUAL_PATH=/home/cyril/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:.
  ++ PATH=/h2o/scoring-pipeline/env/bin:/home/cyril/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:.
  ++ export PATH
  ++ '[' -z '' ']'
  ++ '[' -z '' ']'
  ++ _OLD_VIRTUAL_PS1=
  ++ '[' x '!=' x ']'
  +++ basename /h2o/scoring-pipeline/env
  ++ PS1='(env) '
  ++ export PS1
  ++ alias pydoc
  ++ true
  ++ '[' -n /bin/bash ']'
  ++ hash -r
• python example.py
  2019-08-29 04:50:07,534 C: D: M: 15605 DEBUG : Could not locate cudart (None) or other error: #_python example.pyget_cuda_versions_subprocess: undefined symbol: cudaRuntimeGetVersion
  2019-08-29 04:50:07,654 C: D: M: 15605 DEBUG : Could not locate cudnn (None) or other error #_python example.pyget_cuda_versions_subprocess: undefined symbol: cudnnGetVersion
  2019-08-29 04:50:21,894 C:▇ D:80.1GB M:1.9GB 15462 INFO : Starting H2O server for recipes
  Traceback (most recent call last):
  File "/h2o/scoring-pipeline/env/lib/python3.6/site-packages/h2o/h2o.py", line 265, in init
  "connected.", "not found."))
  File "/h2o/scoring-pipeline/env/lib/python3.6/site-packages/h2o/backend/connection.py", line 323, in open
  conn._cluster = conn._test_connection(retries, messages=_msgs)
  File "/h2o/scoring-pipeline/env/lib/python3.6/site-packages/h2o/backend/connection.py", line 603, in _test_connection
  % (self._base_url, max_retries, "\n".join(errors)))
  h2o.exceptions.H2OConnectionError: Could not establish link to the H2O cloud http://localhost:50331 after 5 retries
  [50:22.18] H2OConnectionError: Unexpected HTTP error: HTTPConnectionPool(host='localhost', port=50331): Max retries exceeded with url: /3/Cloud (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f1f3494b5f8>: Failed to establish a new connection: [Errno 111] Connection refused',))
  [50:22.39] H2OConnectionError: Unexpected HTTP error: HTTPConnectionPool(host='localhost', port=50331): Max retries exceeded with url: /3/Cloud (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f1f3494bd30>: Failed to establish a new connection: [Errno 111] Connection refused',))
  [50:22.59] H2OConnectionError: Unexpected HTTP error: HTTPConnectionPool(host='localhost', port=50331): Max retries exceeded with url: /3/Cloud (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f1f3494b6d8>: Failed to establish a new connection: [Errno 111] Connection refused',))
  [50:22.79] H2OConnectionError: Unexpected HTTP error: HTTPConnectionPool(host='localhost', port=50331): Max retries exceeded with url: /3/Cloud (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f1f34963438>: Failed to establish a new connection: [Errno 111] Connection refused',))
  [50:23.00] H2OConnectionError: Unexpected HTTP error: HTTPConnectionPool(host='localhost', port=50331): Max retries exceeded with url: /3/Cloud (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f1f34963b00>: Failed to establish a new connection: [Errno 111] Connection refused',))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "example.py", line 7, in
from scoring_h2oai_experiment_49205d0e_c403_11e9_9fbc_0800273a584e import Scorer
File "/h2o/scoring-pipeline/env/lib/python3.6/site-packages/scoring_h2oai_experiment_49205d0e_c403_11e9_9fbc_0800273a584e/init.py", line 1, in
from .scorer import Scorer
File "/h2o/scoring-pipeline/env/lib/python3.6/site-packages/scoring_h2oai_experiment_49205d0e_c403_11e9_9fbc_0800273a584e/scorer.py", line 18, in
load_all_custom_recipes()
File "h2oaicore/recipe_server_support.py", line 43, in h2oaicore.recipe_server_support.load_all_custom_recipes
File "h2oaicore/recipe_server_support.py", line 23, in h2oaicore.recipe_server_support.start_servers_for_recipes
File "h2oaicore/recipe_server_support.py", line 117, in h2oaicore.recipe_server_support.start_h2o_recipes
File "h2oaicore/recipe_server_support.py", line 118, in h2oaicore.recipe_server_support.start_h2o_recipes
File "/h2o/scoring-pipeline/env/lib/python3.6/site-packages/h2o/h2o.py", line 277, in init
bind_to_localhost=bind_to_localhost)
File "/h2o/scoring-pipeline/env/lib/python3.6/site-packages/h2o/backend/server.py", line 138, in start
bind_to_localhost=bind_to_localhost, log_dir=log_dir, log_level=log_level)
File "/h2o/scoring-pipeline/env/lib/python3.6/site-packages/h2o/backend/server.py", line 271, in _launch_server
java = self._find_java()
File "/h2o/scoring-pipeline/env/lib/python3.6/site-packages/h2o/backend/server.py", line 410, in _find_java
raise H2OStartupError("Cannot find Java. Please install the latest JRE from\n"
h2o.exceptions.H2OStartupError: Cannot find Java. Please install the latest JRE from
http://www.oracle.com/technetwork/java/javase/downloads/index.html
cyril@h2oclient:/h2o/scoring-pipeline$

[cyrilthank]

==============calling code

cyril@h2oclient:/h2o/scoring-pipeline$ cat run.sh
#/h2o/scoring-pipeline/h2oai.txt
export DRIVERLESS_AI_LICENSE_FILE="/h2o/scoring-pipeline/h2oai.txt"
run_example.sh 
cyril@h2oclient:/h2o/scoring-pipeline$ 
=============================

[cyrilthank]

==============run_example.sh

#!/usr/bin/env bash

set -o pipefail
set -ex

            deactivate
        fi
    fi

    source env/bin/activate
    python example.py
    deactivate
}

run_using_conda(){
    env_name="$(head -n 1 environment.yml | cut -d ' ' -f 2)"
    create_conda_env ${env_name}
    source activate ${env_name}
    python example.py
    source deactivate
}

cd "${script_dir}"
source "./common-functions.sh"

main $@
cd "${current_dir}"

[cyrilthank]

==========example.py

# Copyright 2019 H2O.ai; Proprietary License;  -*- encoding: utf-8 -*-
print(train_munged)  # for model fitting (use entire frame, no cross-validation)
print(valid_munged)  # for model validation (parameter tuning)
print(test_munged)   # for final pipeline testing (one time)

#
# To retrieve the original feature column names, use the Scorer.get_column_names() method:
# This method retrieves the input column names
#

print('---------- Retrieve column names ----------')
print(scorer.get_column_names())

#
# To retrieve the transformed column names, use the Scorer.get_transformed_column_names() method:
# This method retrieves the transformed column names
#

print('---------- Retrieve transformed column names ----------')
print(scorer.get_transformed_column_names())

cyril@h2oclient:/h2o/scoring-pipeline$
cyril@h2oclient:/h2o/scoring-pipeline$ vi run_example.sh
print(train_munged)  # for model fitting (use entire frame, no cross-validation)
print(valid_munged)  # for model validation (parameter tuning)
print(test_munged)   # for final pipeline testing (one time)

#
# To retrieve the original feature column names, use the Scorer.get_column_names() method:
# This method retrieves the input column names 
#

print('---------- Retrieve column names ----------')
print(scorer.get_column_names())

#
# To retrieve the transformed column names, use the Scorer.get_transformed_column_names() method:
# This method retrieves the transformed column names
#

print('---------- Retrieve transformed column names ----------')
print(scorer.get_transformed_column_names())

[cyrilthank]

apologies for the bold letters caused by 'dumb copy & paste'
files.zip

[goldentom42]

I suspect java is not installed on the server and therefore it cannot start H2O open source clusters:

raise H2OStartupError("Cannot find Java. Please install the latest JRE from\n")

Could you try installing and check if that solves the problem ?

[cyrilthank]

Thanks @goldentom42 you are right

However after installing i get the below

---------- Score Row ----------
Stalling, checking threads for check_period=28 stalled_period=2 [75.5, 0.0, 0.0, 0.0, 3.3, 0.0, 0.4, 0.0, 0.5, 0.6, 0.4] [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0] : proctitle: |python example.py2/3 (2 1) (True) [0, 0, 0, 3, 0, 0.4, 0, 0.5, 0.6, 0.4] [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
Stalling, checking threads for check_period=28 stalled_period=2 [75.5, 0.0, 0.0, 0.0, 3.3, 0.0, 0.4, 0.0, 0.5, 0.6, 0.4] [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0] : proctitle: |python example.py2/3 (2 1) (True) [0, 0, 0, 3, 0, 0.4, 0, 0.5, 0.6, 0.4] [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
Stalling, checking threads for check_period=30 stalled_period=3 [75.5, 0.0, 0.0, 0.0, 3.3, 0.0, 0.4, 0.0, 0.5, 0.6, 0.4, 0.5, 0.3] [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0] : proctitle: |python example.py2/3 (2 1) (True) [0, 3, 0, 0.4, 0, 0.5, 0.6, 0.4, 0.5, 0.3] [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
Stalling, checking threads for check_period=30 stalled_period=3 [75.5, 0.0, 0.0, 0.0, 3.3, 0.0, 0.4, 0.0, 0.5, 0.6, 0.4, 0.5, 0.3] [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0] : proctitle: |python example.py2/3 (2 1) (True) [0, 3, 0, 0.4, 0, 0.5, 0.6, 0.4, 0.5, 0.3] [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
Stalling for too long > 440.02 and CPU: [75.5, 0.0, 0.0, 0.0, 3.3, 0.0, 0.4, 0.0, 0.5, 0.6, 0.4, 0.5, 0.3, 0.3, 1.4, 0.8, 1.8, 0.6, 0.0, 1.2] % and GPU: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0] % for n_gpus=0 : proctitle: |python example.py2/3 (2 1) (True) [0.6, 0.4, 0.5, 0.3, 0.3, 1, 0.8, 1, 0.6, 0] [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
Stalling for too long > 440.02 and CPU: [75.5, 0.0, 0.0, 0.0, 3.3, 0.0, 0.4, 0.0, 0.5, 0.6, 0.4, 0.5, 0.3, 0.3, 1.4, 0.8, 1.8, 0.6, 0.0, 1.2] % and GPU: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0] % for n_gpus=0 : proctitle: |python example.py2/3 (2 1) (True) [0.6, 0.4, 0.5, 0.3, 0.3, 1, 0.8, 1, 0.6, 0] [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
Traceback (most recent call last):
  File "example.py", line 33, in <module>
    'At 2.7 MM, my bid is .70/sf higher than his unitsunder construction.',  # text
  File "/h2o/scoring-pipeline/scoring_h2oai_experiment_b90fbaca_c984_11e9_8285_0800273a584e/scorer.py", line 198, in score
    fast_approx=fast_approx)
  File "/h2o/scoring-pipeline/scoring_h2oai_experiment_b90fbaca_c984_11e9_8285_0800273a584e/scorer.py", line 98, in _predict
    preds = predict(input_frame, output_margin=output_margin, pred_contribs=pred_contribs, fast_approx=fast_approx)
  File "/h2o/scoring-pipeline/scoring_h2oai_experiment_b90fbaca_c984_11e9_8285_0800273a584e/scorer.py", line 69, in _predict
    fast_approx=fast_approx, **kwargs)
  File "h2oaicore/models.py", line 7473, in h2oaicore.models.StackedEnsemble.predict_base
  File "h2oaicore/models.py", line 7502, in h2oaicore.models.StackedEnsemble._predict_with_transform
  File "h2oaicore/models.py", line 7048, in h2oaicore.models.StackedBaseModels.transform
  File "h2oaicore/systemutils.py", line 2046, in h2oaicore.systemutils.mppool.submit_tryget
  File "h2oaicore/systemutils.py", line 1675, in h2oaicore.systemutils.mppool.try_get_internal
RuntimeError: Stalling for too long > 440.02 and CPU: [75.5, 0.0, 0.0, 0.0, 3.3, 0.0, 0.4, 0.0, 0.5, 0.6, 0.4, 0.5, 0.3, 0.3, 1.4, 0.8, 1.8, 0.6, 0.0, 1.2] % and GPU: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0] % for n_gpus=0 : proctitle: |python example.py2/3 (2 1) (True) [0.6, 0.4, 0.5, 0.3, 0.3, 1, 0.8, 1, 0.6, 0] [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
cyril@h2oclient:/h2o/scoring-pipeline$ 

can you please help

[cyrilthank]

can you please help with this error which i get after i run (post java install)