Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Refactor get_provider_config() #1

Merged
merged 6 commits into from
May 29, 2022
Merged

Refactor get_provider_config() #1

merged 6 commits into from
May 29, 2022

Conversation

aornugent
Copy link

@aornugent aornugent commented May 29, 2022 •
edited
Loading

Description

Hey @padmajabhol - I tested out your code and got it mostly working. This now compiles compliant configuration for the GCBM_New_Demo_Run.

Major changes

  • I moved the templates into the rest_gcbm_api folder so that they're persistent (a user never needs to know they're there) and updated curl.md.
  • I reformatted the paths of input files to follow the expected structure of your code.
  • I changed some variable names to make it more readable

How Has This Been Tested?

Steps to reproduce:

cd local/rest_api_gcbm
docker-compose up

# in another tab
curl -F disturbances='@disturbances/disturbances_2011_moja.tiff' \
            -F disturbances='@disturbances/disturbances_2012_moja.tiff' \
            -F disturbances='@disturbances/disturbances_2013_moja.tiff' \
            -F disturbances='@disturbances/disturbances_2014_moja.tiff' \
            -F disturbances='@disturbances/disturbances_2015_moja.tiff' \
            -F disturbances='@disturbances/disturbances_2016_moja.tiff' \
            -F disturbances='@disturbances/disturbances_2018_moja.tiff' \
            -F classifiers='@classifiers/Classifier1_moja.tiff' \
            -F classifiers='@classifiers/Classifier2_moja.tiff' \
            -F db='@db/gcbm_input.db' \
            -F miscellaneous='@miscellaneous/initial_age_moja.tiff' \
            -F miscellaneous='@miscellaneous/mean_annual_temperature_moja.tiff' \
            -F title="run4" \
            http://localhost:8080/gcbm/upload
            
curl -d "title=run4" -X POST http://localhost:8080/gcbm/dynamic

# view run status inside the container
# inspect files in local/rest_api_gcbm/input/run4/outputs

Additional Context (Please include any Screenshots/gifs if relevant)

# input/run4/output/simulation.log
# the only block with valid data seems to be Record 49
flint.gcbm | <2022-05-29 03:58:18.244308> (info) - 103: Stats level [Block]: Record [   49 of   100]: Index [( 12674, 51)]: Time(ms) [Total:   244310, Processed:   103966]: Units [Total:   160000, Processed:    11347, NotProcessed:   148653, Errors:        0, LU/second:      654]
flint.gcbm | <2022-05-29 03:58:19.816830> (info) - 103:Thread Finished 
flint.gcbm | <2022-05-29 03:58:19.818766> (info) -   0:Summary of processing by Tile
flint.gcbm | <2022-05-29 03:58:19.818882> (info) -   0: Stats level [Tile]: Index [( 12674)]: Time(sec) [Total:   420000, Processed:   103000]: Units [Total: 16000000, Processed:    11347, NotProcessed: 15988653, Errors:        0, LU/second:    38019]
flint.gcbm | <2022-05-29 03:58:19.818916> (info) -   0:Summary of processing for full run
flint.gcbm | <2022-05-29 03:58:19.818937> (info) -   0: Stats level [Global]: Time(sec) [Total:   420000, Processed:   103000]: Units [Total: 16000000, Processed:    11347, NotProcessed: 15988653, Errors:        0, LU/second:    38019]
flint.gcbm | <2022-05-29 03:58:19.818963> (info) -   0:Start Time           : Sun May 29 03:53:52 2022 UTC
flint.gcbm | <2022-05-29 03:58:19.818987> (info) -   0:Finish Time          : Sun May 29 03:58:19 2022 UTC
flint.gcbm | <2022-05-29 03:58:19.819008> (info) -   0:Total Time (seconds) : 267
flint.gcbm | <2022-05-29 03:58:19.819036> (info) -   0:Effective All LU/millisecond:             59
flint.gcbm | <2022-05-29 03:58:19.819070> (info) -   0:Effective All LU/second:               59925
flint.gcbm | <2022-05-29 03:58:19.819094> (info) -   0:Effective Processed LU/millisecond:        0
flint.gcbm | <2022-05-29 03:58:19.819120> (info) -   0:Effective Processed LU/second:            42
flint.gcbm | <2022-05-29 03:58:19.819195> (info) - Loading results into output/gcbm_output.db
flint.gcbm | <2022-05-29 03:58:19.949697> (info) - Loading DateDimension
flint.gcbm | <2022-05-29 03:58:19.955525> (info) - Loading PoolDimension
flint.gcbm | <2022-05-29 03:58:19.963247> (info) - Loading LandClassDimension
flint.gcbm | <2022-05-29 03:58:19.967984> (info) - Loading ModuleInfoDimension
flint.gcbm | <2022-05-29 03:58:19.973034> (info) - Loading AgeClassDimension
flint.gcbm | <2022-05-29 03:58:19.980568> (info) - Loading LocationDimension
flint.gcbm | <2022-05-29 03:58:19.988416> (info) - Loading DisturbanceTypeDimension
flint.gcbm | <2022-05-29 03:58:19.995872> (info) - Loading DisturbanceDimension
flint.gcbm | <2022-05-29 03:58:20.003567> (info) - Loading Pools
flint.gcbm | <2022-05-29 03:58:20.021039> (info) - Loading Fluxes
flint.gcbm | <2022-05-29 03:58:20.036909> (info) - Loading ErrorDimension
flint.gcbm | <2022-05-29 03:58:20.037060> (info) - Loading LocationErrorDimension
flint.gcbm | <2022-05-29 03:58:20.037138> (info) - Loading AgeArea
flint.gcbm | <2022-05-29 03:58:20.044566> (info) - SQLite insert complete.

Checklist:

There's a bit more to do, but this is starting to look really good! See how much easier our curl call is 💯 🏆

We really need an integration test that checks the results are the same as GCBM_Demo_Run. Maybe we can ask Namya for help?

@Namyalg
Copy link

Namyalg commented May 29, 2022

Yes, this restructuring is more logical.
As for the testing, moja-global#117 is the best way I am able to think of it
Comparing the logs at the end of the run should give us an indication of the simulation status a comparison between the 2 runs

@padmajabhol padmajabhol merged commit 37bf6cf into padmajabhol:gcbm_api May 29, 2022
@aornugent
Copy link
Author

aornugent commented May 29, 2022 •
edited
Loading

Here is the result for the GCBM_New_Demo_Run.zip:

# git merge namyalg/simulation status
# cd local/tests
# python simulation-status.py >> gcbm_simulation.log
{
   "container_running":true,
   "logs":[
      "[2022-05-29 08:05:51 +0000] [1] [INFO] Starting gunicorn 20.1.0\n",
      "[2022-05-29 08:05:51 +0000] [1] [INFO] Listening at: http://0.0.0.0:8080 (1)\n",
      "[2022-05-29 08:05:51 +0000] [1] [INFO] Using worker: gthread\n",
      "[2022-05-29 08:05:51 +0000] [8] [INFO] Booting worker with pid: 8\n",
      "<2022-05-29 08:13:13.156353> (info) - Config has files: 7\n",
      "<2022-05-29 08:13:13.156530> (info) - Using Logging Configuration: /app/input/run4/logging.conf\n",
      "<2022-05-29 08:13:13.156544> (info) - Using configurations: \n",
      "<2022-05-29 08:13:13.156553> (info) - localdomain.json\n",
      "<2022-05-29 08:13:13.156564> (info) - pools_cbm.json\n",
      "<2022-05-29 08:13:13.156571> (info) - modules_cbm.json\n",
      "<2022-05-29 08:13:13.156580> (info) - modules_output.json\n",
      "<2022-05-29 08:13:13.156588> (info) - spinup.json\n",
      "<2022-05-29 08:13:13.156596> (info) - variables.json\n",
      "<2022-05-29 08:13:13.156606> (info) - internal_variables.json\n",
      "<2022-05-29 08:13:13.156614> (info) - Using provider configurations: \n",
      "<2022-05-29 08:13:13.156623> (info) - provider_config.json\n",
      "<2022-05-29 08:13:13.178697> (info) - Using operation manager: \n",
      "<2022-05-29 08:13:13.178720> (info) - spinup ON, sequencer - CBMSpinupSequencer\n",
      "<2022-05-29 08:13:13.189449> (info) - GCBM version: 1.0.0\n",
      "<2022-05-29 08:13:13.203525> (info) -   0:Queuing tile: ( 12674) 1 of 1\n",
      "<2022-05-29 08:13:13.204060> (info) -   0:Total Blocks queued for simulation: 100\n",
      "<2022-05-29 08:13:13.204076> (info) -   0:Run ID = '2f4e7fe0-df27-11ec-acea-0242ac120002'\n",
      "<2022-05-29 08:13:13.204087> (info) -   0:Threads created for processing: 4\n",
      "<2022-05-29 08:13:13.205221> (info) - GCBM version: 1.0.0\n",
      "<2022-05-29 08:13:13.208889> (info) - 100:Run ID = '2f4e7fe0-df27-11ec-acea-0242ac120002'\n",
      "<2022-05-29 08:13:13.208920> (info) - 100:Thread Created\n",
      "<2022-05-29 08:13:13.209810> (info) - GCBM version: 1.0.0\n",
      "<2022-05-29 08:13:13.213562> (info) - 101:Run ID = '2f4e7fe0-df27-11ec-acea-0242ac120002'\n",
      "<2022-05-29 08:13:13.213589> (info) - 101:Thread Created\n",
      "<2022-05-29 08:13:13.214491> (info) - GCBM version: 1.0.0\n",
      "<2022-05-29 08:13:13.218244> (info) - 102:Run ID = '2f4e7fe0-df27-11ec-acea-0242ac120002'\n",
      "<2022-05-29 08:13:13.218270> (info) - 102:Thread Created\n",
      "<2022-05-29 08:13:13.219271> (info) - GCBM version: 1.0.0\n",
      "<2022-05-29 08:13:13.223042> (info) - 103:Run ID = '2f4e7fe0-df27-11ec-acea-0242ac120002'\n",
      "<2022-05-29 08:13:13.223069> (info) - 103:Thread Created\n",
      "<2022-05-29 08:13:13.223082> (info) - modules loaded: \n",
      "<2022-05-29 08:13:13.223133> (info) - \tlibrary: internal.flint, module name: TransactionManagerAfterSubmitModule\n",
      "<2022-05-29 08:13:13.223145> (info) - \tlibrary: moja.modules.cbm, module name: CBMAgeIndicators\n",
      "<2022-05-29 08:13:13.223154> (info) - \tlibrary: moja.modules.cbm, module name: CBMAggregatorLandUnitData\n",
      "<2022-05-29 08:13:13.223164> (info) - \tlibrary: moja.modules.cbm, module name: CBMAggregatorSQLiteWriter\n",
      "<2022-05-29 08:13:13.223173> (info) - \tlibrary: moja.modules.cbm, module name: CBMBuildLandUnitModule\n",
      "<2022-05-29 08:13:13.223181> (info) - \tlibrary: moja.modules.cbm, module name: CBMDecayModule\n",
      "<2022-05-29 08:13:13.223190> (info) - \tlibrary: moja.modules.cbm, module name: CBMDisturbanceEventModule\n",
      "<2022-05-29 08:13:13.223199> (info) - \tlibrary: moja.modules.cbm, module name: CBMDisturbanceListener\n",
      "<2022-05-29 08:13:13.223208> (info) - \tlibrary: moja.modules.cbm, module name: CBMGrowthModule\n",
      "<2022-05-29 08:13:13.223217> (info) - \tlibrary: moja.modules.cbm, module name: CBMLandClassTransitionModule\n",
      "<2022-05-29 08:13:13.223226> (info) - \tlibrary: moja.modules.cbm, module name: CBMSequencer\n",
      "<2022-05-29 08:13:13.223235> (info) - \tlibrary: moja.modules.cbm, module name: CBMTransitionRulesModule\n",
      "<2022-05-29 08:13:13.223243> (info) - \tlibrary: moja.modules.gdal, module name: WriteVariableGeotiff\n",
      "<2022-05-29 08:13:13.223253> (info) - Operation manager: Simple, Version: 1.0, Config: (Kahan[OFF], ZeroTransfers[OFF], AllowNegativeTransfers[ON], WarnNegativeTransfers[OFF])\n",
      "<2022-05-29 08:13:13.223811> (info) - 100:Thread Started \n",
      "<2022-05-29 08:13:13.223859> (info) - 101:Thread Started \n",
      "<2022-05-29 08:13:13.223906> (info) - 102:Thread Started \n",
      "<2022-05-29 08:13:13.223950> (info) - 103:Thread Started \n",
      "<2022-05-29 08:13:15.248426> (info) - 103: Stats level [Block]: Record [   99 of   100]: Index [( 12674,  1)]: Time(ms) [Total:     1742, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    91842]\n",
      "<2022-05-29 08:13:15.248894> (info) - 101: Stats level [Block]: Record [   98 of   100]: Index [( 12674,  2)]: Time(ms) [Total:     1742, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    91813]\n",
      "<2022-05-29 08:13:15.255216> (info) - 100: Stats level [Block]: Record [  100 of   100]: Index [( 12674,  0)]: Time(ms) [Total:     1750, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    91387]\n",
      "<2022-05-29 08:13:15.275430> (info) - 102: Stats level [Block]: Record [   97 of   100]: Index [( 12674,  3)]: Time(ms) [Total:     1767, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    90548]\n",
      "<2022-05-29 08:13:16.979002> (info) - 101: Stats level [Block]: Record [   95 of   100]: Index [( 12674,  5)]: Time(ms) [Total:     1729, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92496]\n",
      "<2022-05-29 08:13:16.979955> (info) - 103: Stats level [Block]: Record [   96 of   100]: Index [( 12674,  4)]: Time(ms) [Total:     1731, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92423]\n",
      "<2022-05-29 08:13:16.989499> (info) - 102: Stats level [Block]: Record [   93 of   100]: Index [( 12674,  7)]: Time(ms) [Total:     1713, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    93360]\n",
      "<2022-05-29 08:13:17.010895> (info) - 100: Stats level [Block]: Record [   94 of   100]: Index [( 12674,  6)]: Time(ms) [Total:     1755, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    91148]\n",
      "<2022-05-29 08:13:18.714531> (info) - 101: Stats level [Block]: Record [   92 of   100]: Index [( 12674,  8)]: Time(ms) [Total:     1735, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92207]\n",
      "<2022-05-29 08:13:18.718386> (info) - 103: Stats level [Block]: Record [   91 of   100]: Index [( 12674,  9)]: Time(ms) [Total:     1738, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92050]\n",
      "<2022-05-29 08:13:18.721349> (info) - 102: Stats level [Block]: Record [   90 of   100]: Index [( 12674, 10)]: Time(ms) [Total:     1731, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92404]\n",
      "<2022-05-29 08:13:18.745880> (info) - 100: Stats level [Block]: Record [   89 of   100]: Index [( 12674, 11)]: Time(ms) [Total:     1734, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92234]\n",
      "<2022-05-29 08:13:20.449163> (info) - 102: Stats level [Block]: Record [   86 of   100]: Index [( 12674, 14)]: Time(ms) [Total:     1727, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92620]\n",
      "<2022-05-29 08:13:20.449964> (info) - 101: Stats level [Block]: Record [   88 of   100]: Index [( 12674, 12)]: Time(ms) [Total:     1735, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92211]\n",
      "<2022-05-29 08:13:20.456509> (info) - 103: Stats level [Block]: Record [   87 of   100]: Index [( 12674, 13)]: Time(ms) [Total:     1737, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92066]\n",
      "<2022-05-29 08:13:20.481015> (info) - 100: Stats level [Block]: Record [   85 of   100]: Index [( 12674, 15)]: Time(ms) [Total:     1734, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92225]\n",
      "<2022-05-29 08:13:22.185342> (info) - 102: Stats level [Block]: Record [   84 of   100]: Index [( 12674, 16)]: Time(ms) [Total:     1735, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92174]\n",
      "<2022-05-29 08:13:22.191275> (info) - 101: Stats level [Block]: Record [   83 of   100]: Index [( 12674, 17)]: Time(ms) [Total:     1741, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    91898]\n",
      "<2022-05-29 08:13:22.199979> (info) - 103: Stats level [Block]: Record [   82 of   100]: Index [( 12674, 18)]: Time(ms) [Total:     1743, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    91784]\n",
      "<2022-05-29 08:13:22.220259> (info) - 100: Stats level [Block]: Record [   81 of   100]: Index [( 12674, 19)]: Time(ms) [Total:     1738, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92007]\n",
      "<2022-05-29 08:13:23.912890> (info) - 102: Stats level [Block]: Record [   80 of   100]: Index [( 12674, 20)]: Time(ms) [Total:     1727, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92635]\n",
      "<2022-05-29 08:13:23.926122> (info) - 101: Stats level [Block]: Record [   79 of   100]: Index [( 12674, 21)]: Time(ms) [Total:     1734, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92240]\n",
      "<2022-05-29 08:13:23.935480> (info) - 103: Stats level [Block]: Record [   78 of   100]: Index [( 12674, 22)]: Time(ms) [Total:     1735, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92205]\n",
      "<2022-05-29 08:13:23.954459> (info) - 100: Stats level [Block]: Record [   77 of   100]: Index [( 12674, 23)]: Time(ms) [Total:     1733, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92275]\n",
      "<2022-05-29 08:13:25.643560> (info) - 102: Stats level [Block]: Record [   76 of   100]: Index [( 12674, 24)]: Time(ms) [Total:     1730, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92467]\n",
      "<2022-05-29 08:13:25.662235> (info) - 101: Stats level [Block]: Record [   75 of   100]: Index [( 12674, 25)]: Time(ms) [Total:     1735, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92176]\n",
      "<2022-05-29 08:13:25.671304> (info) - 103: Stats level [Block]: Record [   74 of   100]: Index [( 12674, 26)]: Time(ms) [Total:     1735, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92188]\n",
      "<2022-05-29 08:13:25.690263> (info) - 100: Stats level [Block]: Record [   73 of   100]: Index [( 12674, 27)]: Time(ms) [Total:     1735, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92190]\n",
      "<2022-05-29 08:13:27.378917> (info) - 102: Stats level [Block]: Record [   72 of   100]: Index [( 12674, 28)]: Time(ms) [Total:     1735, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92217]\n",
      "<2022-05-29 08:13:27.401808> (info) - 101: Stats level [Block]: Record [   71 of   100]: Index [( 12674, 29)]: Time(ms) [Total:     1739, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    91994]\n",
      "<2022-05-29 08:13:27.410693> (info) - 103: Stats level [Block]: Record [   70 of   100]: Index [( 12674, 30)]: Time(ms) [Total:     1739, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    91999]\n",
      "<2022-05-29 08:13:27.430800> (info) - 100: Stats level [Block]: Record [   69 of   100]: Index [( 12674, 31)]: Time(ms) [Total:     1740, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    91939]\n",
      "<2022-05-29 08:13:29.106702> (info) - 102: Stats level [Block]: Record [   68 of   100]: Index [( 12674, 32)]: Time(ms) [Total:     1727, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92622]\n",
      "<2022-05-29 08:13:29.138991> (info) - 101: Stats level [Block]: Record [   67 of   100]: Index [( 12674, 33)]: Time(ms) [Total:     1736, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92121]\n",
      "<2022-05-29 08:13:29.146291> (info) - 103: Stats level [Block]: Record [   66 of   100]: Index [( 12674, 34)]: Time(ms) [Total:     1735, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92201]\n",
      "<2022-05-29 08:13:29.163739> (info) - 100: Stats level [Block]: Record [   65 of   100]: Index [( 12674, 35)]: Time(ms) [Total:     1732, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92342]\n",
      "<2022-05-29 08:13:30.837302> (info) - 102: Stats level [Block]: Record [   64 of   100]: Index [( 12674, 36)]: Time(ms) [Total:     1730, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92470]\n",
      "<2022-05-29 08:13:30.878752> (info) - 101: Stats level [Block]: Record [   63 of   100]: Index [( 12674, 37)]: Time(ms) [Total:     1739, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    91985]\n",
      "<2022-05-29 08:13:30.883496> (info) - 103: Stats level [Block]: Record [   62 of   100]: Index [( 12674, 38)]: Time(ms) [Total:     1736, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92116]\n",
      "<2022-05-29 08:13:30.899022> (info) - 100: Stats level [Block]: Record [   61 of   100]: Index [( 12674, 39)]: Time(ms) [Total:     1735, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92217]\n",
      "<2022-05-29 08:13:32.565315> (info) - 102: Stats level [Block]: Record [   60 of   100]: Index [( 12674, 40)]: Time(ms) [Total:     1727, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92607]\n",
      "<2022-05-29 08:13:32.613329> (info) - 101: Stats level [Block]: Record [   59 of   100]: Index [( 12674, 41)]: Time(ms) [Total:     1734, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92259]\n",
      "<2022-05-29 08:13:32.620932> (info) - 103: Stats level [Block]: Record [   58 of   100]: Index [( 12674, 42)]: Time(ms) [Total:     1737, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92103]\n",
      "<2022-05-29 08:13:32.633805> (info) - 100: Stats level [Block]: Record [   57 of   100]: Index [( 12674, 43)]: Time(ms) [Total:     1734, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92244]\n",
      "<2022-05-29 08:13:34.298307> (info) - 102: Stats level [Block]: Record [   56 of   100]: Index [( 12674, 44)]: Time(ms) [Total:     1732, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92341]\n",
      "<2022-05-29 08:13:34.351478> (info) - 101: Stats level [Block]: Record [   55 of   100]: Index [( 12674, 45)]: Time(ms) [Total:     1737, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92069]\n",
      "<2022-05-29 08:13:34.359478> (info) - 103: Stats level [Block]: Record [   54 of   100]: Index [( 12674, 46)]: Time(ms) [Total:     1738, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92044]\n",
      "<2022-05-29 08:13:34.370850> (info) - 100: Stats level [Block]: Record [   53 of   100]: Index [( 12674, 47)]: Time(ms) [Total:     1736, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92124]\n",
      "<2022-05-29 08:13:36.017315> (info) - 102: Stats level [Block]: Record [   52 of   100]: Index [( 12674, 48)]: Time(ms) [Total:     1718, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    93092]\n",
      "<2022-05-29 08:13:36.078471> (info) - 101: Stats level [Block]: Record [   51 of   100]: Index [( 12674, 49)]: Time(ms) [Total:     1726, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92663]\n",
      "<2022-05-29 08:13:36.085319> (info) - 103: Stats level [Block]: Record [   50 of   100]: Index [( 12674, 50)]: Time(ms) [Total:     1725, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92722]\n",
      "<2022-05-29 08:13:37.736466> (info) - 102: Stats level [Block]: Record [   48 of   100]: Index [( 12674, 52)]: Time(ms) [Total:     1718, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    93084]\n",
      "<2022-05-29 08:13:37.812729> (info) - 101: Stats level [Block]: Record [   47 of   100]: Index [( 12674, 53)]: Time(ms) [Total:     1733, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92276]\n",
      "<2022-05-29 08:13:37.819697> (info) - 103: Stats level [Block]: Record [   46 of   100]: Index [( 12674, 54)]: Time(ms) [Total:     1734, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92266]\n",
      "<2022-05-29 08:13:39.456875> (info) - 102: Stats level [Block]: Record [   45 of   100]: Index [( 12674, 55)]: Time(ms) [Total:     1720, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    93016]\n",
      "<2022-05-29 08:13:39.544599> (info) - 101: Stats level [Block]: Record [   44 of   100]: Index [( 12674, 56)]: Time(ms) [Total:     1731, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92403]\n",
      "<2022-05-29 08:13:39.552221> (info) - 103: Stats level [Block]: Record [   43 of   100]: Index [( 12674, 57)]: Time(ms) [Total:     1732, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92364]\n",
      "<2022-05-29 08:13:41.180357> (info) - 102: Stats level [Block]: Record [   42 of   100]: Index [( 12674, 58)]: Time(ms) [Total:     1723, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92850]\n",
      "<2022-05-29 08:13:41.273464> (info) - 101: Stats level [Block]: Record [   41 of   100]: Index [( 12674, 59)]: Time(ms) [Total:     1728, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92562]\n",
      "<2022-05-29 08:13:41.280317> (info) - 103: Stats level [Block]: Record [   40 of   100]: Index [( 12674, 60)]: Time(ms) [Total:     1727, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92601]\n",
      "<2022-05-29 08:13:42.904292> (info) - 102: Stats level [Block]: Record [   39 of   100]: Index [( 12674, 61)]: Time(ms) [Total:     1723, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92826]\n",
      "<2022-05-29 08:13:43.007875> (info) - 101: Stats level [Block]: Record [   38 of   100]: Index [( 12674, 62)]: Time(ms) [Total:     1734, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92264]\n",
      "<2022-05-29 08:13:43.016580> (info) - 103: Stats level [Block]: Record [   37 of   100]: Index [( 12674, 63)]: Time(ms) [Total:     1736, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92165]\n",
      "<2022-05-29 08:13:44.625121> (info) - 102: Stats level [Block]: Record [   36 of   100]: Index [( 12674, 64)]: Time(ms) [Total:     1720, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92994]\n",
      "<2022-05-29 08:13:44.737349> (info) - 101: Stats level [Block]: Record [   35 of   100]: Index [( 12674, 65)]: Time(ms) [Total:     1729, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92528]\n",
      "<2022-05-29 08:13:44.747984> (info) - 103: Stats level [Block]: Record [   34 of   100]: Index [( 12674, 66)]: Time(ms) [Total:     1731, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92424]\n",
      "<2022-05-29 08:13:46.345843> (info) - 102: Stats level [Block]: Record [   33 of   100]: Index [( 12674, 67)]: Time(ms) [Total:     1720, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    93000]\n",
      "<2022-05-29 08:13:46.470862> (info) - 101: Stats level [Block]: Record [   32 of   100]: Index [( 12674, 68)]: Time(ms) [Total:     1733, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92311]\n",
      "<2022-05-29 08:13:46.482965> (info) - 103: Stats level [Block]: Record [   31 of   100]: Index [( 12674, 69)]: Time(ms) [Total:     1734, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92233]\n",
      "<2022-05-29 08:13:48.069263> (info) - 102: Stats level [Block]: Record [   30 of   100]: Index [( 12674, 70)]: Time(ms) [Total:     1723, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92854]\n",
      "<2022-05-29 08:13:48.203607> (info) - 101: Stats level [Block]: Record [   29 of   100]: Index [( 12674, 71)]: Time(ms) [Total:     1732, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92352]\n",
      "<2022-05-29 08:13:48.216675> (info) - 103: Stats level [Block]: Record [   28 of   100]: Index [( 12674, 72)]: Time(ms) [Total:     1733, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92301]\n",
      "<2022-05-29 08:13:49.788094> (info) - 102: Stats level [Block]: Record [   27 of   100]: Index [( 12674, 73)]: Time(ms) [Total:     1718, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    93103]\n",
      "<2022-05-29 08:13:49.928621> (info) - 101: Stats level [Block]: Record [   26 of   100]: Index [( 12674, 74)]: Time(ms) [Total:     1724, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92766]\n",
      "<2022-05-29 08:13:49.946477> (info) - 103: Stats level [Block]: Record [   25 of   100]: Index [( 12674, 75)]: Time(ms) [Total:     1729, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92509]\n",
      "<2022-05-29 08:13:51.504307> (info) - 102: Stats level [Block]: Record [   24 of   100]: Index [( 12674, 76)]: Time(ms) [Total:     1715, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    93244]\n",
      "<2022-05-29 08:13:51.663115> (info) - 101: Stats level [Block]: Record [   23 of   100]: Index [( 12674, 77)]: Time(ms) [Total:     1734, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92261]\n",
      "<2022-05-29 08:13:51.682726> (info) - 103: Stats level [Block]: Record [   22 of   100]: Index [( 12674, 78)]: Time(ms) [Total:     1735, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92165]\n",
      "<2022-05-29 08:13:53.224032> (info) - 102: Stats level [Block]: Record [   21 of   100]: Index [( 12674, 79)]: Time(ms) [Total:     1719, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    93054]\n",
      "<2022-05-29 08:13:53.393977> (info) - 101: Stats level [Block]: Record [   20 of   100]: Index [( 12674, 80)]: Time(ms) [Total:     1730, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92454]\n",
      "<2022-05-29 08:13:53.415640> (info) - 103: Stats level [Block]: Record [   19 of   100]: Index [( 12674, 81)]: Time(ms) [Total:     1732, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92343]\n",
      "<2022-05-29 08:13:54.945299> (info) - 102: Stats level [Block]: Record [   18 of   100]: Index [( 12674, 82)]: Time(ms) [Total:     1720, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92970]\n",
      "<2022-05-29 08:13:55.122790> (info) - 101: Stats level [Block]: Record [   17 of   100]: Index [( 12674, 83)]: Time(ms) [Total:     1728, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92563]\n",
      "<2022-05-29 08:13:55.145098> (info) - 103: Stats level [Block]: Record [   16 of   100]: Index [( 12674, 84)]: Time(ms) [Total:     1729, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92527]\n",
      "<2022-05-29 08:13:56.671007> (info) - 102: Stats level [Block]: Record [   15 of   100]: Index [( 12674, 85)]: Time(ms) [Total:     1725, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92731]\n",
      "<2022-05-29 08:13:56.846474> (info) - 101: Stats level [Block]: Record [   14 of   100]: Index [( 12674, 86)]: Time(ms) [Total:     1723, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92838]\n",
      "<2022-05-29 08:13:56.877779> (info) - 103: Stats level [Block]: Record [   13 of   100]: Index [( 12674, 87)]: Time(ms) [Total:     1732, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92355]\n",
      "<2022-05-29 08:13:58.393650> (info) - 102: Stats level [Block]: Record [   12 of   100]: Index [( 12674, 88)]: Time(ms) [Total:     1722, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92897]\n",
      "<2022-05-29 08:13:58.575310> (info) - 101: Stats level [Block]: Record [   11 of   100]: Index [( 12674, 89)]: Time(ms) [Total:     1728, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92561]\n",
      "<2022-05-29 08:13:58.610507> (info) - 103: Stats level [Block]: Record [   10 of   100]: Index [( 12674, 90)]: Time(ms) [Total:     1732, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92353]\n",
      "<2022-05-29 08:14:00.114107> (info) - 102: Stats level [Block]: Record [    9 of   100]: Index [( 12674, 91)]: Time(ms) [Total:     1720, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    93014]\n",
      "<2022-05-29 08:14:00.304095> (info) - 101: Stats level [Block]: Record [    8 of   100]: Index [( 12674, 92)]: Time(ms) [Total:     1728, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92564]\n",
      "<2022-05-29 08:14:00.338902> (info) - 103: Stats level [Block]: Record [    7 of   100]: Index [( 12674, 93)]: Time(ms) [Total:     1728, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92584]\n",
      "<2022-05-29 08:14:01.836248> (info) - 102: Stats level [Block]: Record [    6 of   100]: Index [( 12674, 94)]: Time(ms) [Total:     1721, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92923]\n",
      "<2022-05-29 08:14:02.038738> (info) - 101: Stats level [Block]: Record [    5 of   100]: Index [( 12674, 95)]: Time(ms) [Total:     1734, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92251]\n",
      "<2022-05-29 08:14:02.076936> (info) - 103: Stats level [Block]: Record [    4 of   100]: Index [( 12674, 96)]: Time(ms) [Total:     1737, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92071]\n",
      "<2022-05-29 08:14:03.561614> (info) - 102: Stats level [Block]: Record [    3 of   100]: Index [( 12674, 97)]: Time(ms) [Total:     1725, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92750]\n",
      "<2022-05-29 08:14:03.561873> (info) - 102:Thread Finished \n",
      "<2022-05-29 08:14:03.767425> (info) - 101: Stats level [Block]: Record [    2 of   100]: Index [( 12674, 98)]: Time(ms) [Total:     1728, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92569]\n",
      "<2022-05-29 08:14:03.767638> (info) - 101:Thread Finished \n",
      "<2022-05-29 08:14:03.799602> (info) - 103: Stats level [Block]: Record [    1 of   100]: Index [( 12674, 99)]: Time(ms) [Total:     1722, Processed:        0]: Units [Total:   160000, Processed:        0, NotProcessed:   160000, Errors:        0, LU/second:    92892]\n",
      "<2022-05-29 08:14:03.799807> (info) - 103:Thread Finished \n"
   ]
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@aornugent @Namyalg @padmajabhol