Code

DRAFT Setup documentation#215

Code

Draft

wkdewey wants to merge 128 commits intodevCDRH/datura:devfrom setup_documentationCDRH/datura:setup_documentationCopy head branch name to clipboard

+3,556-571

Commits

Commits on Mar 1, 2023

• moves code out of bin elasticsearch files and into module

  Show description for 4e34171
  

  jduss4

  authored and

  wkdewey

  committed
  4e34171

• removes unnecessary dtd for french 17

  

  jduss4

  authored and

  wkdewey

  committed
  172c4aa

• combines some parameter gathering files

  

  jduss4

  authored and

  wkdewey

  committed
  4483c44

• updates gems and fixes test suite

  Show description for c70169b
  

  jduss4

  authored and

  wkdewey

  committed
  c70169b

• in progress working on validator for es fields)

  

  jduss4

  authored and

  wkdewey

  committed
  2aa5f38

• creates validator for elasticsearch postings

  Show description for a5d0047
  

  jduss4

  authored and

  wkdewey

  committed
  a5d0047

• whoops missed one

  

  jduss4

  authored and

  wkdewey

  committed
  8615211

• refactored validator to handle nested field specific mapping

  Show description for 9459f9d
  

  jduss4

  authored and

  wkdewey

  committed
  9459f9d

• get rid of unnecessary variable definitions

  

  jduss4

  authored and

  wkdewey

  committed
  4235116

• put data methods in index class

  wkdewey

  committed
  be55c60

• require_relative so that tests can be run from base directory

  wkdewey

  committed
  6bda669

• move puts from bin methods into es classes

  wkdewey

  committed
  9535604

• change get_schema to return rather than puts

  Show description for 5cd5b4d
  wkdewey

  committed
  5cd5b4d

• simplify regex

  wkdewey

  committed
  b3da61e

• drop unnecessary conditional

  wkdewey

  committed
  da6721f

• return early if invalid nested field found

  wkdewey

  committed
  d7ec579

• change coverage-spatial to spatial

  wkdewey

  committed
  66396ab

• Update CHANGELOG.md

  Show description for fefe254
  wkdewey

  committed
  fefe254

• moves code out of bin elasticsearch files and into module

  Show description for 621193d
  

  jduss4

  authored and

  wkdewey

  committed
  621193d

• in progress working on validator for es fields)

  

  jduss4

  authored and

  wkdewey

  committed
  59c9dd5

• creates validator for elasticsearch postings

  Show description for 162f980
  

  jduss4

  authored and

  wkdewey

  committed
  162f980

• add byebug and update gems

  wkdewey

  committed
  ef20fe8

• specify proper api_version, add xsl file for ead

  wkdewey

  committed
  1470e12

• add ead to format_to_class

  wkdewey

  committed
  17f3ac6

• add date helpers from newer Datura version

  wkdewey

  committed
  e39f423

• add byebug to gemspec

  wkdewey

  committed
  95ea454

• add gem

  wkdewey

  committed
  ae49d18

• add file_ead class

  wkdewey

  committed
  5d56df7

• add EadToES class

  wkdewey

  committed
  5acd4f1

• add files for EadToEs

  wkdewey

  committed
  5b67707

• add EadToEsItems class and associated files

  wkdewey

  committed
  023fd82

• add xsl file for ead (not functional yet)

  wkdewey

  committed
  fc6e4f0

• remove gem doc that is messing things up

  wkdewey

  committed
  ba17a76

• print full error message, not just something went wrong

  wkdewey

  committed
  3523b83

• fix xpath

  wkdewey

  committed
  a6e2a51

• add all require fields, including unfilled ones

  wkdewey

  committed
  0839528

• fix xpaths hash

  wkdewey

  committed
  3b10607

• make EadToEsItems a separate class

  wkdewey

  committed
  90739d5

• add abstract field and fix bad xpaths

  wkdewey

  committed
  10543b7

• add a backtrace to error handling

  wkdewey

  committed
  0d3c07a

• grab 'items' at any nesting of the EAD

  wkdewey

  committed
  49ba94b

• add xpaths and fields, and make sure eadtoesitems inherits from eadtoes

  wkdewey

  committed
  8604663

• change order of get id to fix bug

  wkdewey

  committed
  dbb5659

• add documentation for adding new format

  wkdewey

  committed
  f3628b6

• adjust and add fields for items

  wkdewey

  committed
  6cbe577

• add items to repository xpaths

  wkdewey

  committed
  352efd1

• fix image_url xpath

  wkdewey

  committed
  ef6024e

• add puts statements for debugging

  wkdewey

  committed
  a7c0c9e

• try another way to debug

  wkdewey

  committed
  9bd0e38

• test for nil specifically

  wkdewey

  committed
  7f1fa42

• add debugging statements to get_schema

  wkdewey

  committed
  30294ff

• try debugging with byebug

  wkdewey

  committed
  f13dcae

• remove debugging info

  wkdewey

  committed
  a61363a

• add alternative field

  wkdewey

  committed
  5b87459

• add relation field

  wkdewey

  committed
  fcf01b3

• add spatial field

  wkdewey

  committed
  d2d5939

• fix a get_text method

  wkdewey

  committed
  102a5a8

• change post_es to match jessica's changes

  wkdewey

  committed
  cf85953

• change CommonXML to Datura helpers

  wkdewey

  committed
  b35aaeb

• change xpaths to be less specific to Walt Whitman

  wkdewey

  committed
  436f85d

• refactor title fields and xpaths

  wkdewey

  committed
  c85b31f

• add creator override for items so it is an array

  wkdewey

  committed
  349ff66

• change creators to creator

  wkdewey

  committed
  e640971

• add rdf schema

  wkdewey

  committed
  7286114

• update schemas to include rdf fields

  wkdewey

  committed
  d4f45f0

• add rdf to default fields

  wkdewey

  committed
  f78ea73

• add spatial.title field

  wkdewey

  committed
  f5c9d62

• require byebug so it is in scope for posting etc.

  wkdewey

  committed
  a2b78f6

• remove inserted byebug

  wkdewey

  committed
  330fcd6

• require byebug so it is in scope for posting etc.

  wkdewey

  committed
  f7ebd87

• include full error message with backtrace

  wkdewey

  committed
  73e7b19

• updates gems and fixes test suite

  Show description for 4328acc
  

  jduss4

  authored and

  wkdewey

  committed
  4328acc

• update gems in preparation for release

  wkdewey

  committed
  1660fb1

• start adding new api fields

  wkdewey

  committed
  ee596cc

• update schema to match spreadsheet with new field names

  wkdewey

  committed
  2fe4238

• assemble json based on api version

  wkdewey

  committed
  e40a225

• add overrides for 2.0 fields

  wkdewey

  committed
  bfe1ca6

• change next and previous fields

  wkdewey

  committed
  9c8bc72

• add fig_location

  wkdewey

  committed
  9577bf3

• add abstract

  wkdewey

  committed
  3bd2eeb

• remove split-out assemble_text methods

  wkdewey

  committed
  d30f76b

• update gems and get rid of merge conflicts

  wkdewey

  committed
  48e316e

• add new fields

  wkdewey

  committed
  c9d8730

• correct field name

  wkdewey

  committed
  96d4199

• add fields to ead overrides

  wkdewey

  committed
  a3bbc14

• populate new fields in json

  wkdewey

  committed
  75651f7

• resolve merge conflict

  wkdewey

  committed
  f49f776

• add new fields

  wkdewey

  committed
  91c79d6

• update fields for related items, dates, order integers

  wkdewey

  committed
  95c6a29

• correct syntax errors

  wkdewey

  committed
  3c1ac53

• correct another syntax error

  wkdewey

  committed
  29f9bf2

• change keywords1 to plain keywords

  wkdewey

  committed
  a70d464

• add more specific message to es validation

  wkdewey

  committed
  f7aa3db

• remove extra byebug require

  wkdewey

  committed
  ddb6960

• remove byebug, change error message

  wkdewey

  committed
  b2ffb0c

• update schema under citations

  wkdewey

  committed
  8008c9b

• require fileutils to avoid errors in setup

  wkdewey

  committed
  7a7a0b1

• skip title_sort if title is nil

  wkdewey

  committed
  e983197

• return nil instead of empty string, addresses whitmanarchive/whitman-issues#157

  wkdewey

  committed
  f12291b

• add more nil checks for results of xpath methods

  wkdewey

  committed
  974ab39

• check the correct xpath fields

  wkdewey

  committed
  84d1a9e

• make sure input is in UTF-8

  wkdewey

  committed
  ec81ddb

• make changes for new api schema and revised xpath methods

  wkdewey

  committed
  a3d2fbf

• add a nil check for creators

  wkdewey

  committed
  9a18c1c

• Revert "make sure input is in UTF-8"

  Show description for 31d0a36
  wkdewey

  committed
  31d0a36

• change error handling to avoid method that isn't present

  wkdewey

  committed
  74cd809

• make sure person is an array

  wkdewey

  committed
  47533f9

• make sure settings hash is what elasticsearch expects

  wkdewey

  committed
  79d1455

• change where mappings are posted for es upgrade

  wkdewey

  committed
  aab8145

• add headers to ES requests for authorization

  wkdewey

  committed
  daeb437

• add method to construct basic auth header from options

  wkdewey

  committed
  04de44b

• remove debugging code

  wkdewey

  committed
  c713334

• update conditional logic for status code, dynamic_templates key

  wkdewey

  committed
  4da4062

• change endpoint for delete_by_query for ES8 compatibility

  wkdewey

  committed
  5bf8ca7

• upgrade to Ruby 3.0.4

  wkdewey

  committed
  a9d9fc0

• make keyword arguments compatible with Ruby 3

  wkdewey

  committed
  9c013b8

• go up to ruby 3.1.2

  wkdewey

  committed
  c70d01a

• add output if nested field is invalid

  wkdewey

  committed
  a784c1d

• don't use array method on person to avoid errors

  wkdewey

  committed
  2ec32e4

• update changelog for new version

  wkdewey

  committed
  1ff5b88

• update reference to ruby version

  wkdewey

  committed
  5bcfeb3

• make changes related to ES and API upgrade

  wkdewey

  committed
  51ff02f

• add links to more detailed documentation

  wkdewey

  committed
  bf82b87

• add link to elasticsearch documentation

  wkdewey

  committed
  e2d9d9f

• add conditional to creator for nil checking

  wkdewey

  committed
  bda6d7a

• Create schema_v2.md

  

  karindalziel

  authored and

  wkdewey

  committed
  214c777

• make sure webs_to_es fields can handle nil values

  wkdewey

  committed
  5d6d205

• add documentation steps for starting a new data collection

  wkdewey

  committed
  4c65fb0