-
Notifications
You must be signed in to change notification settings - Fork 5
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
API v2 support with new fields #231
Merged
Merged
Changes from 1 commit
Commits
Show all changes
168 commits
Select commit
Hold shift + click to select a range
419a8fd
moves code out of bin elasticsearch files and into module
jduss4 e452e3b
removes unnecessary dtd for french 17
jduss4 b01397a
combines some parameter gathering files
jduss4 4d02bee
updates gems and fixes test suite
jduss4 3308c75
in progress working on validator for es fields)
jduss4 8e995b4
creates validator for elasticsearch postings
jduss4 9f7eebf
whoops missed one
jduss4 44c3b80
refactored validator to handle nested field specific mapping
jduss4 2deb3ca
get rid of unnecessary variable definitions
jduss4 c863b32
put data methods in index class
wkdewey 282dabf
require_relative so that tests can be run from base directory
wkdewey d081aa0
move puts from bin methods into es classes
wkdewey ebbb8d2
change get_schema to return rather than puts
wkdewey a91a352
simplify regex
wkdewey 1c4b4eb
drop unnecessary conditional
wkdewey 3728424
return early if invalid nested field found
wkdewey 4d2c277
change coverage-spatial to spatial
wkdewey 1b3feeb
Update CHANGELOG.md
wkdewey 70f98a1
moves code out of bin elasticsearch files and into module
jduss4 ab2a750
in progress working on validator for es fields)
jduss4 8dc6ce8
creates validator for elasticsearch postings
jduss4 2d88276
add byebug and update gems
wkdewey 6d20c53
specify proper api_version, add xsl file for ead
wkdewey 6b63319
add ead to format_to_class
wkdewey 6e281f6
add date helpers from newer Datura version
wkdewey 37a7f07
add byebug to gemspec
wkdewey 5bc84bb
add gem
wkdewey 53bfcc3
add file_ead class
wkdewey 1f4361a
add EadToES class
wkdewey 52b38a0
add files for EadToEs
wkdewey 12b0897
add EadToEsItems class and associated files
wkdewey 21a55dc
add xsl file for ead (not functional yet)
wkdewey c39668c
remove gem doc that is messing things up
wkdewey 78fa867
print full error message, not just something went wrong
wkdewey 0df2257
fix xpath
wkdewey 3ce120d
add all require fields, including unfilled ones
wkdewey 812322d
fix xpaths hash
wkdewey 1f20314
make EadToEsItems a separate class
wkdewey 9f01cd0
add abstract field and fix bad xpaths
wkdewey f05e535
add a backtrace to error handling
wkdewey 1805a2d
grab 'items' at any nesting of the EAD
wkdewey 8f74d38
add xpaths and fields, and make sure eadtoesitems inherits from eadtoes
wkdewey fbcf656
change order of get id to fix bug
wkdewey 79e89e7
add documentation for adding new format
wkdewey 44a3790
adjust and add fields for items
wkdewey e2ab095
add items to repository xpaths
wkdewey aa9a9cb
fix image_url xpath
wkdewey 7a9cbeb
add puts statements for debugging
wkdewey 21b1adc
try another way to debug
wkdewey 0511b9f
test for nil specifically
wkdewey 52aba16
add debugging statements to get_schema
wkdewey 2bbdab1
try debugging with byebug
wkdewey 4358081
remove debugging info
wkdewey 5ae1fc0
add alternative field
wkdewey 73089c0
add relation field
wkdewey 428ea4c
add spatial field
wkdewey c5cb629
fix a get_text method
wkdewey 4ab0542
change post_es to match jessica's changes
wkdewey 193d69e
change CommonXML to Datura helpers
wkdewey 344381c
change xpaths to be less specific to Walt Whitman
wkdewey df7e896
refactor title fields and xpaths
wkdewey bf65724
add creator override for items so it is an array
wkdewey c367987
change creators to creator
wkdewey 480f95a
update gems in preparation for release
wkdewey 51d85b2
add rdf schema
wkdewey 45cb1a1
update schemas to include rdf fields
wkdewey 43c5c41
add rdf to default fields
wkdewey 6198a7a
add spatial.title field
wkdewey c145ad8
require byebug so it is in scope for posting etc.
wkdewey f089f42
remove inserted byebug
wkdewey d0bfc36
require byebug so it is in scope for posting etc.
wkdewey 0bd3c58
include full error message with backtrace
wkdewey cd639b4
updates gems and fixes test suite
jduss4 b40c22c
update gems in preparation for release
wkdewey 7efc8fc
start adding new api fields
wkdewey 685a563
update schema to match spreadsheet with new field names
wkdewey 5b52ab1
assemble json based on api version
wkdewey 3af4d47
add overrides for 2.0 fields
wkdewey 8a3634a
change next and previous fields
wkdewey 8e5f888
add fig_location
wkdewey 6397179
add abstract
wkdewey 2f80693
remove split-out assemble_text methods
wkdewey df573e1
update gems and get rid of merge conflicts
wkdewey ee079ae
add new fields
wkdewey c268f8c
correct field name
wkdewey 0eb14bc
add fields to ead overrides
wkdewey ccdede7
populate new fields in json
wkdewey dfa420c
resolve merge conflict
wkdewey e856812
add new fields
wkdewey b252e66
update fields for related items, dates, order integers
wkdewey 215da79
correct syntax errors
wkdewey db217d9
correct another syntax error
wkdewey 8065e3f
change keywords1 to plain keywords
wkdewey 05b5561
add more specific message to es validation
wkdewey fe9fb2b
remove extra byebug require
wkdewey 59013d2
remove byebug, change error message
wkdewey 3889306
update schema under citations
wkdewey f0c19d8
require fileutils to avoid errors in setup
wkdewey c0b734d
skip title_sort if title is nil
wkdewey 1ec9559
return nil instead of empty string, addresses https://github.com/whit…
wkdewey 3eddf9a
add more nil checks for results of xpath methods
wkdewey a1413ba
check the correct xpath fields
wkdewey b71d028
make sure input is in UTF-8
wkdewey 520bbaa
make changes for new api schema and revised xpath methods
wkdewey dd30d2f
add a nil check for creators
wkdewey 0ad2053
Revert "make sure input is in UTF-8"
wkdewey dc727d6
change error handling to avoid method that isn't present
wkdewey 2df4dde
make sure person is an array
wkdewey 436deed
make sure settings hash is what elasticsearch expects
wkdewey 8a35aa2
change where mappings are posted for es upgrade
wkdewey 2c7fa5c
add headers to ES requests for authorization
wkdewey 25f0a2b
add method to construct basic auth header from options
wkdewey 0ec2665
remove debugging code
wkdewey 7b90a09
update conditional logic for status code, dynamic_templates key
wkdewey f982173
change endpoint for delete_by_query for ES8 compatibility
wkdewey 8a99138
Merge pull request #210 from CDRH/elasticsearch_upgrade
techgique 602f6be
upgrade to Ruby 3.0.4
wkdewey c6e687d
make keyword arguments compatible with Ruby 3
wkdewey dbe6aaa
go up to ruby 3.1.2
wkdewey 6b1e468
add output if nested field is invalid
wkdewey de02891
don't use array method on person to avoid errors
wkdewey ba60f4c
update changelog for new version
wkdewey 0ddcc33
update reference to ruby version
wkdewey a614c6d
make changes related to ES and API upgrade
wkdewey fbc0897
add links to more detailed documentation
wkdewey 3cf2237
add link to elasticsearch documentation
wkdewey 3cadd26
add conditional to creator for nil checking
wkdewey 3498ebe
Create schema_v2.md
karindalziel 5b0f54f
make sure webs_to_es fields can handle nil values
wkdewey f286ab4
Merge pull request #211 from CDRH/ruby3upgrade
techgique 594e57f
add new fields to 2.0 schema
wkdewey e928e7e
fix errors
wkdewey aa0770b
create methods and overrides to transform pdf files to elasticsearch
wkdewey 16a421b
add pdf option for command line options
wkdewey d3f87be
fix variable names
wkdewey ab921b7
adjust and clarify documentation for adding new formats
wkdewey d0ba3f0
fix comments
wkdewey 13ba8f7
fix html transformation method
wkdewey 68842a9
clarify that this is not yet implement
wkdewey 1f612a7
add keywords5 to pdftoes fields
wkdewey 43a5821
add has_relation to pdftoes fields
wkdewey 9fe353c
add has_source to pdftoes fields
wkdewey caed7ba
add xpath for notes
wkdewey 91dcfc3
add note xpath to text fields
wkdewey 867cad7
truncate pdf text so it doesn't exceed ES limit
wkdewey 4dfd67b
Update solr_clear_index to use renamed method in parser code
techgique 5cc6c87
Merge pull request #217 from CDRH/pdftoes
techgique b680e70
Output schema in human readable format
techgique 4a1c478
make sure uri_html is blank so html is not loaded in rails
wkdewey e0bb62e
don't hardcode collection name for ead_to_es_items
wkdewey 0bb4c07
fix nested fields in vra that were making the schema not validate
wkdewey 0d39379
add keyword5 for default htmltoes overrides
wkdewey 7ae1583
check for nil values when variables depende on nokogiri methods
wkdewey 5088c14
add character limit and truncate text fields
wkdewey 10e45d0
update changelog
wkdewey 4fcb329
add ead to list of possible formats
wkdewey 22b2d32
fix incorrect field label and remove redundancy
wkdewey 40393d8
revert accidental commenting of subcategory override
wkdewey 67eedaa
Merge pull request #226 from CDRH/whitman-fixes
techgique dd21a6d
change version back to unreleased
wkdewey 8ba660c
replace dead link and link to private repo
wkdewey 20799c5
add pdf-reader to gemspec
wkdewey 544b59d
change link from private docs to ES site
wkdewey 18e1aad
remove file with redundant methods
wkdewey 2c9b4b4
remove uri_html definition which is blanked later
wkdewey 19e1bfe
move byebug into gemspec, don't restrict to dev only
wkdewey d0bac71
remove reference to info that was moved
wkdewey 93ba036
Merge branch 'dev' into new_fields
wkdewey File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please revise the text here to reflect that one doesn't need to look above for the auth info now.