Skip to content

v0.7.0
Compare
Choose a tag to compare
@netj netj released this 13 Jul 23:06
· 260 commits to v0.7.x since this release
v0.7.0
f4e6dfe
  • Provides a new command-line interface deepdive with a new standard DeepDive application layout.
    • No more installation/configuration complication: Users run everything through the only deepdive command, and everything just works in any environment. The only possible failure mode is not being able to run deepdive command, e.g., by not setting up the PATH environment correctly.
    • No more pathname/environment clutter in apps: repeated settings for DEEPDIVE_HOME, APP_HOME, PYTHONPATH, LD_LIBRARY_PATH, PGHOST, PGPORT, ... in run.sh or env.sh or env_local.sh or env_db.sh or etc. are gone. Path names (e.g., extractor udf) in application.conf are all relative to the application root, and brittle relative paths are no longer used in any of the examples.
    • Clear separation of app code from infrastructure code, as well as source code from object code: No more confusing of deepdive source tree with binary/executable/shared-library distribution or temporary/log/output directories.
    • Binary releases can be built with make package.
  • Here are a summary of changes visible to users:
    • Application settings is now kept in deepdive.conf file instead of application.conf.
    • Database settings is now done by putting everything (host, port, user, password, database name) into a single URL in file db.url.
    • Path names (e.g., extractor udf) in deepdive.conf are all relative to the application root unless they are absolute paths.
    • SQL queries against the database can be run easily with deepdive sql command when run under an application.
    • Database schema is now put in file schema.sql and optional initial data loading can be done by a script input/init.sh. Input data is recommended to be kept under input/.
    • By passing the pipeline name as an extra argument to the deepdive run command, different pipelines can be run very easily: No more application.conf editing.
    • Logs and outputs are placed under application root, under snapshot/.
  • Adds piggy extractor that replaces the now deprecated plpy extractor.
  • Includes the latest DDlog compiler with extended syntax support for writing more real world applications.
  • Includes the latest Mindbender with Dashboard GUI for producing summary reports after each DeepDive run and interactively analyzing data products.
Assets 4
Loading