Skip to content
This repository has been archived by the owner on Dec 18, 2019. It is now read-only.

Release History

Jump to bottom
Michael Della Bitta edited this page Jul 18, 2017 · 149 revisions

Effective v32.0.0, version numbers follow a semantic versioning scheme. Major numbers will be incremented whenever effort is required for upgrading, such as changing a configuration file, upgrading dependencies, and so on. Minor numbers will be incremented in most cases, for new features or combinations of features and bugfixes, and patchlevel numbers will be increased when the only changes are bugfixes.

v33.11.6.1 - July 18, 2017

  • Added set to Artstor profile

v33.11.5 - July 14, 2017

  • Imports cleanup
  • Sub in 'requests' lib in fetcher.py

v33.11.4 - June 29, 2017

  • Improvements to check-counts script + emails
  • Scripts to detect and prune records in the index that are leftover from older ingestion runs
  • MI mapper fixes
  • ME mapping

v33.11.3 - June 23, 2017

  • IA Fetcher rewrite
  • Removed obsolete IA code
  • Fix to PA mapping for intermediate providers
  • Added remove property enrichment
  • NCDHC mapping update

v33.11.2 - June 7, 2017

  • ESDN rights mapping changes
  • ESDN sets update
  • Tweaks to Maryland

v33.11.1 - May 24, 2017

  • Adds Maryland
  • Tweaks to Tennesee profile

v33.11.0 - May 17, 2017

  • Updates to Artstor sets
  • Adds Tennesee
  • Removes old Minnesota profile
  • Changes to Getty mapper
  • Adds Avro dumping

v33.10.4 - Apr 10, 2017

  • Rewrite of BHL mapping
  • Add tests for BHL
  • Update ArtStor sets
  • Update README with instructions for using Twofishes to perform geo-enrichments

v33.10.3 - Apr 06, 2017

  • Add 10 sets to ESDN
  • Add Pennsylvania as a provider
  • Select-id enrichment now accepts an optional parameter to exclude the provider prefix from DPLA id construction.

v33.10.2 -

v33.10.1 - Mar 15, 2017

  • Remove validate_mapv3 enrichment
  • Add new sets to ArtStor
  • Remove extraneous logging messages

v33.10.0 - Mar 1, 2017

  • Fixup bad variable name in WI mapper
  • Initial implementation of arvo saving
  • CDL mapping implementation based on new JSON/Solr-based harvest
  • Change BHL to MODS
  • Retrofit Twofishes onto geocode enrichment
  • Fix CDL enrichments to work with QA
  • Fix CDL mapping of thumbnail preview
  • Fix rights mapping for NCDHC
  • Fix IndexError exception in geocoder

v33.9.1 - Feb 10, 2017

  • Add Wisconsin as a provider
  • Add CDL fetcher
  • Fix bug in fetcher.py that was prevented harvesting from endpoints that use https://
  • Update to akara.conf.template to remove unneeded geocoding config

v33.9.0 -

v33.8.9 - Jan 24, 2017

  • Add ESDN as a provider
  • OAI harvester supports resumptionTokens in ListSets
  • Patched an error in the mapping of NYPL's standardized rightsstatements.org values
  • Update GPO endpoint in profile
  • Update copyright dates in project
  • Update SCDL collection mapping
  • Update Michigan date mapping

v33.8.8 - Dec 20, 2016

  • Add Michigan, Illinois and Wisconsin as providers
  • Add mapping for NYPL's rightsstatements.org values
  • Add some additional checking in enrich-type.py, enrich-format.py, utilities.py and shred.py for null values

v33.8.7 - Sep 23, 2016

  • Update the metadata prefix for Indiana and add the prefix to QDC metadata field mapping

v33.8.6 - Sep 12, 2016

  • Create QDC mapping for Indiana (refs #8514)
  • Add set SSDPLABennington to ArtStor (refs #8553)

v33.8.5 - Aug 08, 2016

  • Update data provider mappings for Harvard (refs #8453)
  • Add sets ihp, ward, and rubbings to Harvard (refs #8453)

v33.8.4 - Jul 05, 2016

  • Add SSDPLARhode set to ArtStor profile
  • Add set_params for SSDPLARhode

v33.8.3 - Mar 29, 2016

  • Fixup enrich_date delimited-month test
  • Add set_params for SSDPLALawrence, SSDPLASaintMary and SSDPLAPuget
  • Add SSDPLALawrence, SSDPLASaintMary and SSDPLAPuget sets

v33.8.2 - Feb 3, 2016

  • Fix broken Missouri mapping

v33.8.1 - Feb 1, 2016

  • Update Missouri and MWDL profiles. Fix MWDL timeouts and fix location of Missouri PMH endpoint.

v33.8.0 - Dec 21, 2015

  • Change MWDL blacklist (ref #8216) and modify bulk export steps

v33.7.3 - Dec 21, 2015

  • Harvard: change data provider for "cna" set (refs #8172)
  • Digital Commonwealth: don't map alternative title (refs #7990)

v33.7.2 - Dec 2, 2015

  • [IA] add Frick Art Reference Library

v33.7.0 - Oct 26, 2015

  • Map type 'mixed material' to ['image', 'text']. Refs #8043

v33.6.0 - Oct 15, 2015

  • Add usc_no_contributor filter for USC
  • Run build on Travis-CI containerized infrastructure

v33.5.2 - Sep 18, 2015

  • Add sets to MWDL exclusion list

v33.5.1 - July 24, 2015

  • Fix source of NYPL rights

v33.5.0 - July 20, 2015

  • Add Artstor sets
  • Log content for OAI parse error
  • Update Harvard mapping and sets
  • Remove problematic USC sets that have single-page records
  • Add traceback printing to couch module

v33.4.0 - June 9, 2015

  • Remove set from MWDL's exclusions in profile

v33.3.0 - June 2, 2015

  • Add MDL and CDL providers

v33.2.3 - April 13, 2015

  • Fix move_date_values to properly iterify incoming values
  • Add Indiana profile
  • Harvard: Properly iterate over incoming descriptions
  • NYPLMapper: fixup publisher and description setters
  • MARCMapper: improve handling in set_begin_end_dates()
  • Added test for pre-1900 dates in enrich_date (#7562)
  • Fix enrich_date to handle dates < 1900 (#7562)
  • Hotfix: Changed location of dict evaluation in ia_fetcher.request_records
  • Added "moving image" to type conversion keywords.

v33.2.2 - April 3, 2015

  • Update ARTstor, GPO, Hathitrust, Internet Archive, and Smithsonian profiles
  • Fix exception in AbsoluteURLFetcher
  • Map MWDL intermediateProvider
  • Change field for Getty collection

v33.2.1 - November 17, 2014

  • IAMapper: remove extraneous print function
  • Fix sitemap generation and use pyrax to sync to Rackspace

v33.2.0 - October 31, 2014

  • IA: generate sourceResource.title correctly (fixes #7708)
  • IA: Add YIVO Institute Library (fixes #7735)
  • Remove old SCDL-specific code (refs #7647)

v33.1.6 - October 1, 2014

  • Add Missouri Hub profile #7617

v33.1.4 - September 25, 2014

  • SCDL single-source profile #7647
  • Add strip_html pipeline module to all profiles #7636
  • Geocoding bug fixes: (refs #7705)
    • Add retries and logging to Geonames requests
    • Fix Geonames webservice URI
    • Fix critical bug in Place.merge_related()
  • MAPv3.1/temporal-related indexing fixes for MWDL/NYPL #7704

v33.1.1 - September 24, 2014

  • DigitalNC: Override DigitalNCMapper super's map_format (#29, refs #7656)

v33.1.0 - September 23, 2014

  • Geocoding refactor (#7664, #7643, #7640):
    • Skip 'USA' place names when Geocoding
    • Refactor geocode to use a Place class
    • Update enrichments to use Geonames first, before falling back on

v33.0.2 - September 22, 2014

  • Fix UTC timestamp regression #7699
  • Fix dedupe value exception to handle non-strings #7700
  • Allow "physical object" to type mapping and update validation #7663
  • Parallelize tests to isolate those connecting externally

Upgrading

Run pip install -r requirements.txt in addition to the usual steps, before restarting Akara.

v33.0.1 - September 15, 2014

  • IA: Add Montana State Library collection

v33.0.0 - September 11, 2014

  • Set log level for couch.py (fixes #7688)
  • Maintain Primo URL/params better (ref #7654)
  • Add module to strip HTML tags (#7636)
  • Refactor Couch object for sync_qa_views property (ref #7545)

Upgrading

Run pip install -r requirements.txt in addition to the usual steps, before restarting Akara.

Optional: set the new LogLevel property in akara.ini, per README.md.

v32.2.0 - September 8, 2014

  • Have save_records.py use sync_qa_views (fixes #7680)
  • GPO: Set URI for object property (refs #7519)
  • Cleanup unused/out of date code (fixes #7645, #7677)

v32.1.0 - September 4, 2014

  • Complete GPO profile (#7519)
  • Fix authority_condition in OAIMODSMapper.map_format() (#7673)
  • Hathi: fix removal of </collection> when fetching (#7678)
  • Enable/disable building of QA views (refs #7545)
  • Add docstrings to explain shred module (#7613)

Upgrading

Add the following line to your akara.ini file in the [CouchDb] section:

SyncQAViews=<Either True or False; if missing QA views will be synced by default>

v32.0.0 - August 29, 2014

  • Remove old Digital Commonwealth profile (#7633)
  • Have BPL enrichment remove finding aids (#7639)
  • Update geocode module to set 'name' property and add unit tests (#7607)
  • Add SCDLCharlestonMapper for object mapping (#7651)
  • UTC timestamps everywhere; fix sitemaps (#7657)

Upgrading

Add the following line to your akara.ini file in the [Sitemap] section:

SitemapPath=<Path to a writable local directory for sitemap files>

v31.2 - August 15, 2014

  • Fixed stripping of brackets in displayDate (#7598)
  • Updated date and temporal mapping (#7612)
  • Refactored views to eliminate separate *_count views (#7545)

Upgrading: This is being released alongside an update to the platform (API) application, which changes the QA interface's generation of reports. The sync_couch_views.py script should be run against the 'dpla' database, ideally before the views are actually needed.

v31.1 - August 7, 2014

  • Removed "nonSort" from BPL title mapping (#7570)
  • Removed cleanup and lookup language enrichments from Getty profile

v31.0 - August 7, 2014

  • Added Boston Library Consortium to IA intermediateProviders (#7595)
  • Changed JSON Schema for validation (#7590)
  • Fixed uiuc_book URL in test, and re-enabled test
  • Re-enabled fetcher tests
  • Refactored copy_prop module (#7130)
  • Removed uses_network attributes from tests that use local files
  • Refactored enrich_language module (#7618)
  • Fixed tests based on enrich_language refactoring
  • Refactored Primo and added Getty (#7603, #7604)
  • Fixed MWDL fetcher test

v30.0 - July 28, 2014

  • Added parsing of intermediateProvider, added MHL (#7588)
  • Changed NCDHC ("digitalnc") endpoint_url (#7552)

v29.2 - July 17, 2014

  • Updated mapv3 schema for validation
  • Updated fetcher, mapper, profile for MWDL upgrade (#6610)
  • Include all "content" type note values in description for NCDHC (#7594)

v29.1 - July 14, 2014

  • Improved date parsing in marc_mapper and enrich_date (#7530)
  • Added sets to Artstor (#7550)

v29.0 - July 8, 2014

v28.1 - June 30, 2014

  • Updated export file structure (#7540)
  • Minor refactoring, *-to-dpla modules (#6563)

v28.0 - June 26, 2014

  • Improved exception handling in couch module (#7486)

v27.1 - June 19, 2014

  • Refactor to-dpla modules into mapper classes (#6563)
  • Added test for pre-1900 dates in enrich_date (#7562)
  • Fixed problems with pipeline lacking "/" character (#7565)
  • Fix JSON-LD @context issues (#6919)

v27.0.1 - June 13, 2014

  • Fix enrich_date to handle dates < 1900 (#7562)

v27.0 - June 11, 2014

  • In couch.Couch._get_bulk_download_doc(), handle empty result or missing view
  • Fixed in hathi_fetcher: parsed_docs should always be a list (#7559)

v26.2 - May 31, 2014

  • Remove global views (refs #7493)

v26.1.1 - May 29, 2014

  • #7289, #7319: Add validation against MAPv3 JSON Schema and supporting views
  • #3062: Add views to report missing properties
  • #7493: Add global count views
  • #4586: remove operations on aggregatedCHO in Smithsonian profile
  • #7520: Added parsing capabilities to enrich_earliest_date
  • Reinstate human-readable byte count to bulk-file database record

Upgrading:

Run pip to install the new packages for JSON Schema support:

$ pip install -r requirements.txt

Update the "dpla" CouchDB database's views:

$ python scripts/sync_couch_views.py dpla

v26.0 - May 28, 2014

  • Changed database export's use of views (7155)
  • Have pip use DPLA mirror of Zepheira zenpub

Upgrading: Update the "dpla" CouchDB database's views:

$ python scripts/sync_couch_views.py dpla

v25.1 - May 21, 2014

  • Skip geocoding for generic "United States" (#6794)
  • Fix ARTStor format field (#4251)
  • Initial mapping for GPO and geocoding fixes
  • Add automation of bulk data upload (#4776)

Upgrading:

  • Create the "bulk_download" CouchDB database. For example, 
    $ curl -X PUT "http://user:pw@couchserver:5984/bulk_download"
  • Load that database's views: 
    $ python scripts/sync_couch_views.py bulk_download

v25.0 - May 15, 2014

  • Exception handling, OAI fetch; fixed MODS key error (#7524)
  • Print error when posting to the database fails (#7509)
  • Fixed geocode tests, because Bing response has changed
  • Added uses_network attribute to tests

v24.1 - May 5, 2014

  • Fixed OAI-PMH fetcher memory leak; added fetcher tests (#5483)
  • Added "photographer" to NYPL creator terms (#7512)

v24.0 - April 30, 2014

  • Fixed call to couch.sync_couch_views() in sync_couch_views.py
  • Added movingimage (w/out space) to type config
  • Updated Artstor profile to fix dataProvider, spatial
  • Changed artstor_identify_object recognition of thumbnail images
  • Fixed potentially undeclared variable in poll_storage

v23.3 - April 25, 2014

  • Added exclusion of IA "collection"-type records (#6731)
  • Added Cambridge Public Library to IA profile (#7502)
  • Fixed test for list in dpla-list-records.listrecords.
  • Lowered MWDL bulkSize parameter to 500, from 1000 (#7486)
  • Modified test_all_oai_verb_fetchers to fetch only a few profiles
  • Changed automated tests to retain Akara directory

v23.2 - April 17, 2014

  • Changes for new NYPL API, incl. multithreaded fetcher (#7773)
  • Changed primo-to-dpla (MWDL) to use facets/rsrctype for type (#7441)
  • Fixed tests

v23.1 - April 16, 2014

  • Added missing check_counts to poll_storage.py

v23.0 - April 12, 2014

  • Changed view syncing to speed up save, delete, and backup (#7487)
  • Changed Akara MaxServers and MaxRequestsPerServer
  • Added "moving image" to type conversion keywords.
  • Partial fix for 7491 ("sound recording" vs "sound")
  • Fix 7491: Delete sourceResource/type if type not found and default is None
  • Add enrich-type unit tests

v22.1 - April 4, 2014

Changes for new BPL, Digital Commonwealth sites; Geocoding fixes

  • Resolve #7330: BPL spatial data, "United States" generic value
  • Other geocoding improvements
  • Fix #7488: BPL dataProvider: add option to pull from recordInfo/recordContentSource
  • Fix #7330 and #6097: Created subject_and_spatial_transform_bpl to join spatial values on double-hyphen and add the coordinates
  • Fix #7329: Iterify OAI-PMH/ListRecords/record, fix location_transform_bpl

v22.0 - March 31, 2014

  • Added threading to enrich_records (#7282)
  • Added catch for incomplete read in database export (#7155)
  • Added catch for undefined sourceResource in enrich-type.
  • Fixed Google thumbnail image URL in test for hathi_identify_object

Upgrading:

Add the following lines to your akara.ini file:

[Enrichment]
QueueSize=4
ThreadCount=4

These are suggested numbers for a start. You can experiment with increasing them.

v21.0 - March 19, 2014

  • Fixed #7329 - Iterify OAI-PMH/ListRecords/record & fix location_transform_bpl
  • Fixed #7256 - Smithsonian missing titles
  • Resolved #7315 - Print counts and error if alert email fails
  • Set blacklist for BPL metasets; lower threshold for old DC endpoint
  • Removed obsolete scripts
  • Added speed.py to report on elapsed time from access log

v20.5 - March 13, 2014

  • Fixed #7197 - Improved type mapping for enrich-type module

v20.4.1 - March 12, 2014

  • Refs #7169 - add regionaldigitzationmass to Internet Archive profile
  • Fix an error with the ia_fetcher

v20.3 - March 12, 2014

  • Fixed 7183 - Create an alert if an ingestion adds, deletes, or changes a certain number of records

Notes:

  • Add an Alert section to the akara.ini file with To and From parameters:
[Alert]
To=<Comma-separated email addresses to receive alert emails>
From=<Email address of alert email sender, ie no-reply@dpla>

v20.2 - March 11, 2014

  • Fixed 7226 - Change Akara logging to produce smaller logs by default
  • Fixed 6623 - SCDL - Update ingest process according to crosswalk rationalization
  • Fixed 6949 - Create instructions for setting up an ingestion server from scratch
  • Fixed part of #7155 - Added ret. val. checking, db export per-source file generation

Upgrading:

  • Add this line to the Akara section of your akara.ini file:
LogLevel=<level>

where <level> is DEBUG or INFO (or another logging level, but those are the recommended ones)

  • Re-install Akara so that it uses the DPLA version. Do pip uninstall akara and then pip install -r requirements.txt.

  • Run the usual python setup.py install

v20.1 - March 5, 2014

  • Fixed 7254 - Fixed class name of instantiated HathiFetcher in create_fetcher
  • Fixed 6624 - UIUC (OAI_DC) - Update ingest process according to crosswalk rationalization
  • Fixed 6622 - MDL - Update ingest process according to crosswalk rationalization

v20.0 - March 4, 2014

  • Fixed 6966 - Run sitemap creation process after Week 4 ingest
  • Fixed 7218 - HathiTrust: tweak to ingest script to remove double commas
  • Fixed 7231 - Add database-export view to views synced by "couch" ingestion module
  • Fixed 7219 - Ingestion: Handle XML parsing exception and output bad lines
  • Fixed 6944 - IA - Fetch process hangs

Notes:

  • Create a Sitemap section in akara.ini
[Sitemap]
SitemapURI=http://sitemaps.dp.la

v19.1 - February 26, 2014

  • Issue 4265 - Added determination of Smithsonian type for physical format
  • Fixed edan_to_dpla.transform_date exception for non-iterable dates

v19.0 - February 25, 2014

  • Fixed 7122 - Digital Commonwealth contributing institution values gone

v18.1 - February 14, 2014

  • Fixed 7122 - Digital Commonwealth contributing institution values gone

v18.0 - February 3, 2014

  • Fixed 5976 - Install sitemap to allow search engines to crawl individual item pages

Notes: In the Rackspace section of akara.ini change ContainerName parameter (not the value) to DPLAContainer and add parameter SitemapContainer with value Sitemap.

v17.2 - January 31, 2014

  • Uncomment QA views to allow building during sync method

v17.1 - January 30, 2014

  • Fixed 6686 - ARTstor: "from" and "until" dates in set_params need to be changed
  • Fixed 6208 - Rollback process should remove dashboard documents
  • Fixed 5789 - Restrict ingested data to our schema to stop current bloating

v17.0 - January 28, 2014

  • Fixed 6786 - Enrichment errors from last Smithsonian ingest
  • Fixed 4675 - Digital Commonwealth - "Finding Aid" in title
  • Fixed 6267 - HATHI: Permanently suppress ALL MDL records
  • Fixed 6880 - MODS to DPLA and OAI/MODS to DPLA set hasView.@id incorrectly
  • Fixed 6752 - Geocode module fails if wrong version of geopy is used
  • Fixed 6758 - SCDL: Fix geocoding of forcibly set coordinates for South Carolina regions
  • Fixed README formatting

v16.0 - January 7, 2014

  • Fixed 6625 - IA - Update ingest process according to crosswalk rationalization
  • Crosswalk fixes for NARA, Hathi, ARTstor, UIUC MARC, and PTH
  • Added handling of error tag in OAI-PMH response

v15.0 - December 31, 2013

  • Fixed Hathi thumbnail URL for HVD
  • Crosswalk fixes for Smithsonian, USC, BHL, KDL and MWDL
  • Hardcoded provider for Hathi

v14.3 - December 19, 2013

  • Hardcoded general CouchDB settings
  • Updated README
  • Changed URL scheme to https in requirements.txt
  • Fixed poll_storage
  • More compare_with_schema module fixes

v14.2 - December 15, 2013

  • Fixed compare_with_schema module

v14.1 - December 12, 2013

  • Added stateLocatedIn enrichment for Smithsonian
  • Fixed scdl_enrich_location
  • Updated poll_storage to handle pipe method return tuple

v14.0 - December 11, 2013

  • Fixed 6491 - Digital Commonwealth - Crosswalk update
  • Fixed 6209 - Dashboard database should only contain item-level documents for the last 3 ingestion sequences
  • Fixed 5651 - Hathi data fix post production release

v13.1.1 - December 3, 2013

  • Added handling of non-string values in move_date_values module
  • Fix test_parse_profiles

v13.1 - December 3, 2013

  • Fixed 6515 - Don't send deleted records through enrichment pipeline
  • Blacklisted all non-partner collections for PTH

v13.0 - December 2, 2013

  • Fixed 6510 - NARA - Records without collections should still be ingested
  • Fixed 5085 - NARA - new data set

v12.1 - November 27, 2013

  • Fixed 6223 - IA - URLs which timeout or return a 404 response should be retried at the end of the fetch process
  • Fixed 6129 - DPLA: Fix Portal to Texas History ID mapping
  • Fixed 6490 - Log number of records that failed ingest
  • Added retry logic for MWDL

Notes: Add LogLevel=ERROR in the CouchDb section of the akara.ini file.

v12.0.1 - November 21, 2013

  • Fixed poll_storage script

v12.0 - November 21, 2013

  • Updated set_prop to handle dictionary values
  • Fixed setting of provider field in ia_to_dpla
  • Added --no-backup option to save_records

v11.3 - November 14, 2013

  • Fixed Smithsonian stateLocatedIn mapping

v11.2 - November 13, 2013

  • Fixed 6209: Dashboard database should only contain item-level documents for the last 3 ingestion sequences
  • Fixed 5789: Restrict ingested data to our schema to stop current bloating
  • Fixed startkey and endkey doc._id in iterview queries

v11.1 - November 11, 2013

  • Fixed 6225: Regenerate views on update
  • Fixed 5769: Change Hathi/ UIUC mapping
  • Fixed 5856: Refactor Hathi fetcher for new poll_profiles
  • Fixed 5651: Hathi data fix post production release

v11.0 - November 5, 2013

  • Fixed 5459: Reingest NYPL with new crosswalk
  • Fixed 6224: Smithsonian/Global - Update fetchers to return only records and not collection information
  • Updated all post/queries to CouchDB to use batch_size

Notes: Rename IterviewBatch to BatchSize in the CouchDb section of the akara.ini file and set its value to 500.

v10.7 - October 22, 2013

  • Fixed 5855: Ingest North Carolina Digital Heritage
  • Fixed USC dataProvider

v10.6 - October 21, 2013

  • Fixed 5933: ArtStor - New Collection (SSDPLAWashington)
  • Updated IAFetcher to reset page on new collection

v10.5 - October 20, 2013

  • Fixed date transform in oai_untl_to_dpla module
  • Updated couch module to not save unchanged collection documents

v10.4 - October 18, 2013

  • Added collection statelibrarynorthcarolina to IA profile
  • Fixed stateLocatedIn for various providers

v10.3 - October 17, 2013

  • Fixed 5970: Slight change to DLG ingest script
  • Fixed UVA stateLocatedIn field

v10.2 - October 15, 2013

  • Added profile for Portal to Texas History
  • Added handling of OAI-PMH/error in ListRecords response
  • Added oai_untl_to_dpla module
  • Updated couch.py to not iterate over all DPLA database ids
  • Restored UVA fetcher unit tests

v10.1 - October 14, 2013

  • Fixed request_more in MWDLFetcher
  • Compressed fetch and enrich directories in the enrich and save scripts

v10.0 - October 14, 2013

  • Fixed geocode unit tests
  • Commented out UIUC book profile from test_all_oai_verb_fetchers test
  • Fixed expected_records in IAFetcher

v9.6 - October 12, 2013

  • Updated IAFetcher to use fetch_url method from internet_archive.py

v9.5 - October 11, 2013

  • Fixed UVA spatial transform
  • Minimized CouchDB post requests in the save_records script

v9.4 - October 10, 2013

  • Fixed 5084: UVA - ingest uva-lib:628506
  • Fixed 5909: Investigate missing UVA records
  • Excluded uva1 fetcher test from Travis

v9.3 - October 7, 2013

  • Fixed BPL description transform to handle lists with both string and dictionary values

v9.2 - October 4, 2013

  • Updated enrich_location module to handle lists with both string and dictionary values
  • Added a BPL specific description transform in oai_mods_to_dpla.py

v9.1 - October 4, 2013

  • Updated EDANFetcher to parse Smithsonian XML files in parts

v9.0 - October 2, 2013

  • Fixed error_msg in fetch_records script
  • Updated oai.py to handle dictionary resumption_token
  • Fixed ingestion document ID references in ingestion scripts

v8.4 - September 26, 2013

  • Updated export_database script and view to use provider.name

v8.3 - September 26, 2013

  • Added BPL profile and updated oai_mods_to_dpla to map BPL
  • Fixed UIUC books dataProvider

v8.2 - September 24, 2013

  • Updated export_database script
  • Updated couch module to post records with iterview_batch as limit

v8.1 - September 23, 2013

  • Removed log_json references in contentdm_identify_object module
  • Appended to UIUC book dataProvider
  • Fixed ingestion doc id references in scripts

v8.0 - September 19, 2013

  • Fixed marc_to_dpla UIUC book provider
  • Fixed overriding of OAI fetcher metadataPrefix

v7.2.1 - September 13, 2013

  • Fixed UIUC book profile name field

v7.2 - September 13, 2013

  • Fixed 4789: UIUC - Book collections ingest

v7.1 - September 9, 2013

  • Fixed ISBN extraction in hathi_identify_object module

v7.0 - September 9, 2013

  • Fixed 4644: Refactor poll_profiles
  • Fixed 5314: Split out "Fetch" process from poll_profiles
  • Fixed 5315: Split out "Enrich" process from poll_profiles
  • Fixed 5316: Split out "Save" process from poll_profiles
  • Fixed 5683: Hathi: Add thumbnails

v6.2 - August 25, 2013

  • Fixed Hathi sourceResource.spatial field

v6.1.1 - August 21, 2013

  • Added unit test for getting last ingestion document

v6.1 - August 21, 2013

  • Fixed 5567: Couch module function _get_last_ingestion_doc_for does not return last ingestion document

v6.0 - August 19, 2013

  • Fixed 5545: Update test/server_support to support Geonames token
  • Fixed 4890: Ingest HathiTrust data into a dev environment

v5.4.2 - August 14, 2013

  • Fixed simple service URL for geocode module

v5.4.1 - August 14, 2013

  • Fixed 5507: USC: Fix Geonames URL

v5.4 - August 13, 2013

  • Fixed 5497: USC: Add latitude check

v5.3 - August 12, 2013

  • Fixed 5488: Hathi: Fix parsing of XML files so as not to use so much memory

v5.2 - August 9, 2013

  • Fixed 5475: USC: Fix location coordinates

v5.1 - August 8, 2013

  • Fixed 5465: USC: Fix dates

v5.0 - August 7, 2013

  • Fixed 5391: USC: Better geocoding

v4.6 - August 1, 2013

  • Fixed 5180: Change UIUC endpoint

v4.5 - July 30, 2013

  • Fixed 5390: USC - Update endpoint URL and remove geocode from pipeline

v4.4 - July 30, 2013

  • Fixed 5366: Update poll_storage to use couch module
  • Fixed 5365: USC: Extract thumbnails
  • Fixed 5382: Create script to delete all of a provider's documents

v4.3 - July 25, 2013

  • Fixed 4890 - Ingest HathiTrust data into a dev environment

v4.2 - July 25, 2013

  • Fixed 5009 - Ingest USC data

v4.1 - July 18, 2013

  • Changed Virginia books profile "name" field from "virginia" to "virginia_books"

v4.0 - July 18, 2013

  • Fixed 4807: UVA - Ingest book collection
  • Fixed 4737: UVA - Ingest additional collection
  • Fixed 4656: UVA - Use different domain name for ingestion
  • Fixed 4723: UIUC - Add additional sets for ingestion
  • Fixed 5055: Refactor the mods_to_dpla module
  • Fixed 5278: David Rumsey profile should use production URL for ingest
  • Fixed 5277: Artstor thumbnail parsing fails
  • Fixed 5250: DLG - Thumbnail extraction fails if underscores are embedded in the item identifier part of the identifier
  • Fixed 4250: Ingestions are bloating ElasticSearch schemas
  • Fixed 3779: SCDL - Inconsistency in case/pluralization for object formats
Clone this wiki locally