Running mip algorithms on exareme

• Usage

  EXAREME as back-end service is accessible from the Web portal throw the corresponding REST API. An example usage is the following:

  1. GET /mining/algorithms : Get the list of available algorithms from the web portal and present them to the user appropriately.  
  2. POST /mining/query/<algorithm_name : User selects an algorithm and provides the required algorithm parameters.
  3. POST /mining/query/<query_key>/status : Web portal submit the selected algorithm with the fulfilled parameters for execution.    
  4. POST /mining/query/<query_key>/result  : Web portal checks the execution status of the algorithm and upon completion request the results in order to visualize them appropriately. On each hospital Exareme worker is using the RAW query engine in order to access the hospital data. For this purpose we have developed a RAW query engine driver, based on a REST API, that integrates Exareme with RAW.

• Input

  1. No input is required for getting the list of available algorithms.
  2. Algorithm parameters in a key-value format are required as input when submitting an algorithm for execution.
  3. The algorithm execution key (which is returned upon submission) is required when requesting the algorithm execution status or results.

• Output

  1. Get list of available algorithms/ Submit an algorithm/ Get an algorithm execution status: returns a simple JSON.
  2. Get an algorithm execution results: returns JSON or AVRO format.

• Distributed dataflow

  1. A user poses a global request (in the form of template's parameters) through the web portal as follows: He selects an algorithm and provides the required algorithm parameters. Then, the web portal submits to Exareme the selected algorithm with the fulfilled parameters for execution.
  2. Exareme executes the global request as follows: The master transforms the global request to local and global operators. Then, it schedules and dispatches the local operators to the appropriate workers. Each worker executes them (via RAW query engine) and the aggregated results are forwarded back to the master. The master executes the global operators on the aggregated results of all hospitals and produces the overall results. This process is repeated till the federation system computes the response of the initial global request. When the final results have been computed, they are returned to the web portal and presented to the user.

• Configuration

  • Clone https://github.com/madgik/mip-algorithms repository.
  • By default, Exareme is configured to use rawdb as external local engine.
  • Set up rawdb and fill the host, port parameters in mip-algorithms/properties.json.
  • Clone https://github.com/madgik/exareme.git repository and checkout 'mip' branch.
  • Congigure the gateway.properties with your local path of mip-algorithms.
  • Run Exareme in local mode for testing.

• Sample Client :

  • in JS