Skip to content

v0.3.0

Latest
Latest
Compare
Choose a tag to compare
@ramfox ramfox released this 04 May 19:52
· 38 commits to master since this release
v0.3.0
3f4b696
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.

v0.3.0 (2021-05-04)

This release of the dataset package includes one major change, adding a Stats component, as well as a few minor changes and a bunch of bug fixes that are listed below.

Stats component

Introducing the stats component, a top-level component that provides the mechanics to quickly generate the stats using probabilistic structures. Unlike previous iterations of calculating stats, it is not bound to size/time limitations. We calculate and store different kinds of stats based on the content of the given column or fields. The different types as of this release are numeric, boolean, and string. We've moved the qri/stats package into dataset under the name dsstats.

Take a look at our spec for details on how stats are calculated.

Bug Fixes

  • dataset.BodyFile: if no dataset exists, return nil (84c88eb)
  • dataset: DropTransients drops peername field (656948d)
  • dsgen: fix flag for number of rows in dsgen CLI (8a042f1)
  • meta: marshalling to json object should not modify private meta field (5a55038)
  • meta,structure: serializing to JSON includes path value (cfd5aca)
  • preview: rename CreatePreview -> Preview, don't consume input dataset files (b7a9395)
  • stats: avoid nil ptr panic (#242) (556268c)
  • stats: limiting top-k frequencies to 200 (#239) (74e6f19)
  • transform: Assign() overwrites Steps field (ce73c09)

Features

  • commit: add RunID field to Commit struct (ecaf655)
  • preview: CreatePreview takes a dataset.Dataset and returns a truncated version (1fae175)
  • dataset: add ID field to dataset.Dataset (ceb9ee1)
  • detect.Structure: move struture detection function down from qri (2330b0f)
  • dsio.ReadAll: add ReadAll, ReadAllObject, ReadAllArray functions (80263b4)
  • dsstats: move stats package from qri core, rename to dsstats (e5257e0)
  • dstest: Add Readme support (d480331)
  • dstest: add CompareGoldenDatasetAndUpdate convenience function (577ff3f)
  • dstest: add Template function (f588dde)
  • dstest: configuraable CompareDatasets, Golden File Functions (1019334)
  • ShallowCompare,PathMap: add utility methods for comparing components (80c9f61)
  • SigningBytes: new SigningBytes includes all components (1b5ddf1)
  • stats: add Assign method, stats component tests (23fb3fd)
  • stats: add stats component (4e9ca61)
  • stats: use 'sa' as kind prefix, marshal stats to/from JSON (5235164)
  • transform: add Syntaxes field to Transform struct (8a30d20)
  • type: utility to check type presence for columns in tabular (#244) (9b4fc79)

BREAKING CHANGES

  • dataset: older versions of qri that attempt to verify the signature of datasets with a
    non-empty ID string field will error.
  • removed Compare* functions, use dstest.Compare instead
Assets 2
Loading