Actions #50
Merged
Actions #50
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
joeleehen
added a commit
that referenced
this pull request
Dec 17, 2024
* added PubMed (refactored to OOP) module and scaffolding for IEEE module * initialized poetry project * added (blank) __init__.py to package and test directories * added testing, logging to PubMed.py (WIP) * further testing * added python venv dirs to .gitignore * .gitignore additions, formatting/linter changes * Added arXiv python file * Added python file for Springer API * Added python file for Elsevier API * Added python file for Wiley API * Added test for arXvip and updated code for arXvip to standardize name variations * Git added test for Spinger API * Updated code for test_Springer and Spinger.py for name Normalizion * Updated code to be in class format * deleted redundant comments * Updated code for Elsevier.py * Fixed Commenting * Updated code for Wiley.py normalize_author_name * added docstrings, better testing coverage * better error logging in PubMed.py * Fixed code format * Fixed standardize author name for all API * Hendrix additions (#1) * added PubMed (refactored to OOP) module and scaffolding for IEEE module * initialized poetry project * added (blank) __init__.py to package and test directories * added testing, logging to PubMed.py (WIP) * further testing * added python venv dirs to .gitignore * .gitignore additions, formatting/linter changes * deleted redundant comments * added docstrings, better testing coverage * better error logging in PubMed.py --------- Co-authored-by: Joseph Hendrix <[email protected]> Co-authored-by: Joseph Hendrix <[email protected]> * Fixed .gitignore file * Updated test_PubMed.py yo mach with recent changes * Updated to PubMed.py for merging with dev * Fixed returning authors for arXiv * Fixed standardize_author_name for Elsevier * Update Elsevier.py * Added try/catch for Elsevier and arXiv * Added try/catch for PubMed * Added try/catch to Springer * add try/catch for Wiley api * fixed imports and API keys * Added PLOS API along with try/catch * pyproject changes ensure extending from a Base class doesn't break pytest * removed old pubmed script at root of project * API modules extend base class, added secrets template * moved URLs to config.py * added functionality to main.py (WIP) * added bumpversion, poetry script entry point * main.py can now accept command line arguments (WIP) * fixed erroneous method call in MDPI class, added root logger object to main.py * Updated arXive test and code * Added more test for arXiv * Updated formate * Base class, main CLI script (#5) * pyproject changes ensure extending from a Base class doesn't break pytest * removed old pubmed script at root of project * API modules extend base class, added secrets template * moved URLs to config.py * added functionality to main.py (WIP) * added bumpversion, poetry script entry point * main.py can now accept command line arguments (WIP) * fixed erroneous method call in MDPI class, added root logger object to main.py * Updated with baseclass * Fxied main * Added Wiley Api and fixed nameing convention of DOI's * Updated config with urls also connecting new api files with Base * Cli additions (#9) * todo comments * added example input file, author names gathered from file * main script writes API results to (user configurable) file * added api-keyed endpoints to main script * updated .gitignore * crossref into cli_additions (#8) * added CrossRef module and tests (WIP) * crossref data extraction (WIP) * barebones crossref functionality * added full CrossRef functionality, began testing * CrossRef tests and accompanying module changes --------- Co-authored-by: Joseph Hendrix <[email protected]> * added functionality to specify targeted APIs through command line * removed obsolete requirements.txt * Updated README.md updated README.md to reflect command line changes * minor updates to secret.sample for clarity * updated TODO comments in main --------- Co-authored-by: Joseph Hendrix <[email protected]> Co-authored-by: Magret Adekunle <[email protected]> * Better testing (#11) * created dev dependency group * removed extraneous config from project root * added main.py tests * added description to pyproject.toml * better pubmd tests (WIP) * mock test for PubMed module * pubmed tests no longer query PubMed API * Springer tests no longer query Springer API * removed bad test from test_CrossRef.py (testing API rather than pubscraper functionality) * cleaned some comments * List apis (#12) * added --list flag to list APIs available to query * added help message for --apis and --list * prettier --list output * Updated Main to have PLOS/Weily and testing for PLOS/arXiv * Added test for Elsevier * Child class changes (#14) * privatized crossref methods, added DOI * linter changes, privatized Elsevier methods, changed result format * privatized MDPI methods, added DOI * PLOS linter changes, added DOI * added DOI to PubMed, privatized methods * privatized Springer methods, changed result formatting * privatized Wiley methods, changed result formatting * privatized arXiv methods, changed output formatting * added tests for --list CLI flag * minor typo fixes in comments * better logging on startup * Containerize (#15) * added Dockerfile (builds, no run) * changed some import statements to fix ModuleNotFoundErrors when running from wheel build * Added tablib to export data * Crossref fixes (#16) * privatized crossref methods, added DOI * linter changes, privatized Elsevier methods, changed result format * privatized MDPI methods, added DOI * PLOS linter changes, added DOI * added DOI to PubMed, privatized methods * privatized Springer methods, changed result formatting * privatized Wiley methods, changed result formatting * privatized arXiv methods, changed output formatting * added tests for --list CLI flag * minor typo fixes in comments * aggregating results is broken, all other tests pass * fixed a bad check in CrossRef.py where 0 valid publications would mistakenly break out of query loop * Aded changes to README.md * Update README.md * Update README.md * Updated MDPI and main.py * Updated MDPI.py and main.py * Update README.md * Main fixes (#20) * minor Dockerfile fixes in COPY statements * updated CLI options and their tests * changed input file and input parsing to better reflect expected input * Update README.md * Updated main * Updted main and sub class for logger * Updated ArXive date * Standardize CrossRef publication date * Standardize Elsevier publication date * Standardize MDPI publication date * Standardize PLOS publication date * Standardize PubMed publication date * Standardize Springer publication date * Standardize Wiley publication date * Standardize the publication date to "YYYY-MM-DD" * Updated main.py to allows user to pull publication from cutoff date and test for main.py * Added Documentations for xlsx * updated README for flag `--cutoff_date` or `-cd` * Update README.md * Update README.md * Added mock responses for test_Elsevier.py * Added Test for test_CrossRef.py * Update README.md * Docker io (#35) * script now reads a (default) xlsx file instead of a csv * added run.sh script (WIP) * added .env support and example for storing secrets * Update README.md * refactored API list to a const in main.py * changed --input to click.Path object * removed old files from Dockerfile * moved input and default input/output files value to /data directory, mount /data directory to docker container instead of the whole project * quick poetry.lock fix * Updated MDPI for pull date * Added logging.debug for MDPI.py * Updated code for pull date (CrossRef.py) * Updated code for Elsevire.py (date and added logging.debug) * Added code for loggin.debug * Added logging.debug for PLOS.py * Updated pull date from PubMed.py * Added logging.debug to Spinger.py * Added loggin.debug to Wiley.py * Updated authors naming * Bumpversion fixes (#37) * added .env version spec to pyproject; bumpversion now updates .env * Update README.md * got rid of /data folder (too confusing) * Update README.md * Added testing for test_MDPI.py * Updated test_Elsevier.py to use .env * Added testting for test_Elsevire.py * Wiley tests (#42) * added Wiley tests (WIP) * added Wiley tests * Development (#41) * Added testing for test_MDPI.py * Updated test_Elsevier.py to use .env * Added testting for test_Elsevire.py --------- Co-authored-by: Magret <[email protected]> Co-authored-by: Magret Adekunle <[email protected]> * fixed main.py tests (all tests pass!) --------- Co-authored-by: Magret <[email protected]> Co-authored-by: Magret Adekunle <[email protected]> * Runner versioning (#43) * run.sh now properly reads IMAGEVERSION environment variable from .env for building the container * removed old secrets sample, edited IMAGEVERSION in .env.sample to match our versioning scheme * changed help text for --list option for clarity * updated help text for test_main * Cleanup (#45) * added callback for --list option * Update README.md * Updated hard-coded vars to config.py * bumped version * added actions yaml files (#48) * Actions (#50) * added actions yaml files * corrected job name in docker-image.yml --------- Co-authored-by: Joseph Hendrix <[email protected]> Co-authored-by: Joseph Hendrix <[email protected]> Co-authored-by: Magret <[email protected]> Co-authored-by: Magret Adekunle <[email protected]>
joeleehen
added a commit
that referenced
this pull request
Dec 17, 2024
* added PubMed (refactored to OOP) module and scaffolding for IEEE module * initialized poetry project * added (blank) __init__.py to package and test directories * added testing, logging to PubMed.py (WIP) * further testing * added python venv dirs to .gitignore * .gitignore additions, formatting/linter changes * Added arXiv python file * Added python file for Springer API * Added python file for Elsevier API * Added python file for Wiley API * Added test for arXvip and updated code for arXvip to standardize name variations * Git added test for Spinger API * Updated code for test_Springer and Spinger.py for name Normalizion * Updated code to be in class format * deleted redundant comments * Updated code for Elsevier.py * Fixed Commenting * Updated code for Wiley.py normalize_author_name * added docstrings, better testing coverage * better error logging in PubMed.py * Fixed code format * Fixed standardize author name for all API * Hendrix additions (#1) * added PubMed (refactored to OOP) module and scaffolding for IEEE module * initialized poetry project * added (blank) __init__.py to package and test directories * added testing, logging to PubMed.py (WIP) * further testing * added python venv dirs to .gitignore * .gitignore additions, formatting/linter changes * deleted redundant comments * added docstrings, better testing coverage * better error logging in PubMed.py --------- Co-authored-by: Joseph Hendrix <[email protected]> Co-authored-by: Joseph Hendrix <[email protected]> * Fixed .gitignore file * Updated test_PubMed.py yo mach with recent changes * Updated to PubMed.py for merging with dev * Fixed returning authors for arXiv * Fixed standardize_author_name for Elsevier * Update Elsevier.py * Added try/catch for Elsevier and arXiv * Added try/catch for PubMed * Added try/catch to Springer * add try/catch for Wiley api * fixed imports and API keys * Added PLOS API along with try/catch * pyproject changes ensure extending from a Base class doesn't break pytest * removed old pubmed script at root of project * API modules extend base class, added secrets template * moved URLs to config.py * added functionality to main.py (WIP) * added bumpversion, poetry script entry point * main.py can now accept command line arguments (WIP) * fixed erroneous method call in MDPI class, added root logger object to main.py * Updated arXive test and code * Added more test for arXiv * Updated formate * Base class, main CLI script (#5) * pyproject changes ensure extending from a Base class doesn't break pytest * removed old pubmed script at root of project * API modules extend base class, added secrets template * moved URLs to config.py * added functionality to main.py (WIP) * added bumpversion, poetry script entry point * main.py can now accept command line arguments (WIP) * fixed erroneous method call in MDPI class, added root logger object to main.py * Updated with baseclass * Fxied main * Added Wiley Api and fixed nameing convention of DOI's * Updated config with urls also connecting new api files with Base * Cli additions (#9) * todo comments * added example input file, author names gathered from file * main script writes API results to (user configurable) file * added api-keyed endpoints to main script * updated .gitignore * crossref into cli_additions (#8) * added CrossRef module and tests (WIP) * crossref data extraction (WIP) * barebones crossref functionality * added full CrossRef functionality, began testing * CrossRef tests and accompanying module changes --------- Co-authored-by: Joseph Hendrix <[email protected]> * added functionality to specify targeted APIs through command line * removed obsolete requirements.txt * Updated README.md updated README.md to reflect command line changes * minor updates to secret.sample for clarity * updated TODO comments in main --------- Co-authored-by: Joseph Hendrix <[email protected]> Co-authored-by: Magret Adekunle <[email protected]> * Better testing (#11) * created dev dependency group * removed extraneous config from project root * added main.py tests * added description to pyproject.toml * better pubmd tests (WIP) * mock test for PubMed module * pubmed tests no longer query PubMed API * Springer tests no longer query Springer API * removed bad test from test_CrossRef.py (testing API rather than pubscraper functionality) * cleaned some comments * List apis (#12) * added --list flag to list APIs available to query * added help message for --apis and --list * prettier --list output * Updated Main to have PLOS/Weily and testing for PLOS/arXiv * Added test for Elsevier * Child class changes (#14) * privatized crossref methods, added DOI * linter changes, privatized Elsevier methods, changed result format * privatized MDPI methods, added DOI * PLOS linter changes, added DOI * added DOI to PubMed, privatized methods * privatized Springer methods, changed result formatting * privatized Wiley methods, changed result formatting * privatized arXiv methods, changed output formatting * added tests for --list CLI flag * minor typo fixes in comments * better logging on startup * Containerize (#15) * added Dockerfile (builds, no run) * changed some import statements to fix ModuleNotFoundErrors when running from wheel build * Added tablib to export data * Crossref fixes (#16) * privatized crossref methods, added DOI * linter changes, privatized Elsevier methods, changed result format * privatized MDPI methods, added DOI * PLOS linter changes, added DOI * added DOI to PubMed, privatized methods * privatized Springer methods, changed result formatting * privatized Wiley methods, changed result formatting * privatized arXiv methods, changed output formatting * added tests for --list CLI flag * minor typo fixes in comments * aggregating results is broken, all other tests pass * fixed a bad check in CrossRef.py where 0 valid publications would mistakenly break out of query loop * Aded changes to README.md * Update README.md * Update README.md * Updated MDPI and main.py * Updated MDPI.py and main.py * Update README.md * Main fixes (#20) * minor Dockerfile fixes in COPY statements * updated CLI options and their tests * changed input file and input parsing to better reflect expected input * Update README.md * Updated main * Updted main and sub class for logger * Updated ArXive date * Standardize CrossRef publication date * Standardize Elsevier publication date * Standardize MDPI publication date * Standardize PLOS publication date * Standardize PubMed publication date * Standardize Springer publication date * Standardize Wiley publication date * Standardize the publication date to "YYYY-MM-DD" * Updated main.py to allows user to pull publication from cutoff date and test for main.py * Added Documentations for xlsx * updated README for flag `--cutoff_date` or `-cd` * Update README.md * Update README.md * Added mock responses for test_Elsevier.py * Added Test for test_CrossRef.py * Update README.md * Docker io (#35) * script now reads a (default) xlsx file instead of a csv * added run.sh script (WIP) * added .env support and example for storing secrets * Update README.md * refactored API list to a const in main.py * changed --input to click.Path object * removed old files from Dockerfile * moved input and default input/output files value to /data directory, mount /data directory to docker container instead of the whole project * quick poetry.lock fix * Updated MDPI for pull date * Added logging.debug for MDPI.py * Updated code for pull date (CrossRef.py) * Updated code for Elsevire.py (date and added logging.debug) * Added code for loggin.debug * Added logging.debug for PLOS.py * Updated pull date from PubMed.py * Added logging.debug to Spinger.py * Added loggin.debug to Wiley.py * Updated authors naming * Bumpversion fixes (#37) * added .env version spec to pyproject; bumpversion now updates .env * Update README.md * got rid of /data folder (too confusing) * Update README.md * Added testing for test_MDPI.py * Updated test_Elsevier.py to use .env * Added testting for test_Elsevire.py * Wiley tests (#42) * added Wiley tests (WIP) * added Wiley tests * Development (#41) * Added testing for test_MDPI.py * Updated test_Elsevier.py to use .env * Added testting for test_Elsevire.py --------- Co-authored-by: Magret <[email protected]> Co-authored-by: Magret Adekunle <[email protected]> * fixed main.py tests (all tests pass!) --------- Co-authored-by: Magret <[email protected]> Co-authored-by: Magret Adekunle <[email protected]> * Runner versioning (#43) * run.sh now properly reads IMAGEVERSION environment variable from .env for building the container * removed old secrets sample, edited IMAGEVERSION in .env.sample to match our versioning scheme * changed help text for --list option for clarity * updated help text for test_main * Cleanup (#45) * added callback for --list option * Update README.md * Updated hard-coded vars to config.py * bumped version * added actions yaml files (#48) * Actions (#50) * added actions yaml files * corrected job name in docker-image.yml * bumped version --------- Co-authored-by: Joseph Hendrix <[email protected]> Co-authored-by: Joseph Hendrix <[email protected]> Co-authored-by: Magret <[email protected]> Co-authored-by: Magret Adekunle <[email protected]>
joeleehen
added a commit
that referenced
this pull request
Dec 17, 2024
* Development (#47) * added PubMed (refactored to OOP) module and scaffolding for IEEE module * initialized poetry project * added (blank) __init__.py to package and test directories * added testing, logging to PubMed.py (WIP) * further testing * added python venv dirs to .gitignore * .gitignore additions, formatting/linter changes * Added arXiv python file * Added python file for Springer API * Added python file for Elsevier API * Added python file for Wiley API * Added test for arXvip and updated code for arXvip to standardize name variations * Git added test for Spinger API * Updated code for test_Springer and Spinger.py for name Normalizion * Updated code to be in class format * deleted redundant comments * Updated code for Elsevier.py * Fixed Commenting * Updated code for Wiley.py normalize_author_name * added docstrings, better testing coverage * better error logging in PubMed.py * Fixed code format * Fixed standardize author name for all API * Hendrix additions (#1) * added PubMed (refactored to OOP) module and scaffolding for IEEE module * initialized poetry project * added (blank) __init__.py to package and test directories * added testing, logging to PubMed.py (WIP) * further testing * added python venv dirs to .gitignore * .gitignore additions, formatting/linter changes * deleted redundant comments * added docstrings, better testing coverage * better error logging in PubMed.py --------- Co-authored-by: Joseph Hendrix <[email protected]> Co-authored-by: Joseph Hendrix <[email protected]> * Fixed .gitignore file * Updated test_PubMed.py yo mach with recent changes * Updated to PubMed.py for merging with dev * Fixed returning authors for arXiv * Fixed standardize_author_name for Elsevier * Update Elsevier.py * Added try/catch for Elsevier and arXiv * Added try/catch for PubMed * Added try/catch to Springer * add try/catch for Wiley api * fixed imports and API keys * Added PLOS API along with try/catch * pyproject changes ensure extending from a Base class doesn't break pytest * removed old pubmed script at root of project * API modules extend base class, added secrets template * moved URLs to config.py * added functionality to main.py (WIP) * added bumpversion, poetry script entry point * main.py can now accept command line arguments (WIP) * fixed erroneous method call in MDPI class, added root logger object to main.py * Updated arXive test and code * Added more test for arXiv * Updated formate * Base class, main CLI script (#5) * pyproject changes ensure extending from a Base class doesn't break pytest * removed old pubmed script at root of project * API modules extend base class, added secrets template * moved URLs to config.py * added functionality to main.py (WIP) * added bumpversion, poetry script entry point * main.py can now accept command line arguments (WIP) * fixed erroneous method call in MDPI class, added root logger object to main.py * Updated with baseclass * Fxied main * Added Wiley Api and fixed nameing convention of DOI's * Updated config with urls also connecting new api files with Base * Cli additions (#9) * todo comments * added example input file, author names gathered from file * main script writes API results to (user configurable) file * added api-keyed endpoints to main script * updated .gitignore * crossref into cli_additions (#8) * added CrossRef module and tests (WIP) * crossref data extraction (WIP) * barebones crossref functionality * added full CrossRef functionality, began testing * CrossRef tests and accompanying module changes --------- Co-authored-by: Joseph Hendrix <[email protected]> * added functionality to specify targeted APIs through command line * removed obsolete requirements.txt * Updated README.md updated README.md to reflect command line changes * minor updates to secret.sample for clarity * updated TODO comments in main --------- Co-authored-by: Joseph Hendrix <[email protected]> Co-authored-by: Magret Adekunle <[email protected]> * Better testing (#11) * created dev dependency group * removed extraneous config from project root * added main.py tests * added description to pyproject.toml * better pubmd tests (WIP) * mock test for PubMed module * pubmed tests no longer query PubMed API * Springer tests no longer query Springer API * removed bad test from test_CrossRef.py (testing API rather than pubscraper functionality) * cleaned some comments * List apis (#12) * added --list flag to list APIs available to query * added help message for --apis and --list * prettier --list output * Updated Main to have PLOS/Weily and testing for PLOS/arXiv * Added test for Elsevier * Child class changes (#14) * privatized crossref methods, added DOI * linter changes, privatized Elsevier methods, changed result format * privatized MDPI methods, added DOI * PLOS linter changes, added DOI * added DOI to PubMed, privatized methods * privatized Springer methods, changed result formatting * privatized Wiley methods, changed result formatting * privatized arXiv methods, changed output formatting * added tests for --list CLI flag * minor typo fixes in comments * better logging on startup * Containerize (#15) * added Dockerfile (builds, no run) * changed some import statements to fix ModuleNotFoundErrors when running from wheel build * Added tablib to export data * Crossref fixes (#16) * privatized crossref methods, added DOI * linter changes, privatized Elsevier methods, changed result format * privatized MDPI methods, added DOI * PLOS linter changes, added DOI * added DOI to PubMed, privatized methods * privatized Springer methods, changed result formatting * privatized Wiley methods, changed result formatting * privatized arXiv methods, changed output formatting * added tests for --list CLI flag * minor typo fixes in comments * aggregating results is broken, all other tests pass * fixed a bad check in CrossRef.py where 0 valid publications would mistakenly break out of query loop * Aded changes to README.md * Update README.md * Update README.md * Updated MDPI and main.py * Updated MDPI.py and main.py * Update README.md * Main fixes (#20) * minor Dockerfile fixes in COPY statements * updated CLI options and their tests * changed input file and input parsing to better reflect expected input * Update README.md * Updated main * Updted main and sub class for logger * Updated ArXive date * Standardize CrossRef publication date * Standardize Elsevier publication date * Standardize MDPI publication date * Standardize PLOS publication date * Standardize PubMed publication date * Standardize Springer publication date * Standardize Wiley publication date * Standardize the publication date to "YYYY-MM-DD" * Updated main.py to allows user to pull publication from cutoff date and test for main.py * Added Documentations for xlsx * updated README for flag `--cutoff_date` or `-cd` * Update README.md * Update README.md * Added mock responses for test_Elsevier.py * Added Test for test_CrossRef.py * Update README.md * Docker io (#35) * script now reads a (default) xlsx file instead of a csv * added run.sh script (WIP) * added .env support and example for storing secrets * Update README.md * refactored API list to a const in main.py * changed --input to click.Path object * removed old files from Dockerfile * moved input and default input/output files value to /data directory, mount /data directory to docker container instead of the whole project * quick poetry.lock fix * Updated MDPI for pull date * Added logging.debug for MDPI.py * Updated code for pull date (CrossRef.py) * Updated code for Elsevire.py (date and added logging.debug) * Added code for loggin.debug * Added logging.debug for PLOS.py * Updated pull date from PubMed.py * Added logging.debug to Spinger.py * Added loggin.debug to Wiley.py * Updated authors naming * Bumpversion fixes (#37) * added .env version spec to pyproject; bumpversion now updates .env * Update README.md * got rid of /data folder (too confusing) * Update README.md * Added testing for test_MDPI.py * Updated test_Elsevier.py to use .env * Added testting for test_Elsevire.py * Wiley tests (#42) * added Wiley tests (WIP) * added Wiley tests * Development (#41) * Added testing for test_MDPI.py * Updated test_Elsevier.py to use .env * Added testting for test_Elsevire.py --------- Co-authored-by: Magret <[email protected]> Co-authored-by: Magret Adekunle <[email protected]> * fixed main.py tests (all tests pass!) --------- Co-authored-by: Magret <[email protected]> Co-authored-by: Magret Adekunle <[email protected]> * Runner versioning (#43) * run.sh now properly reads IMAGEVERSION environment variable from .env for building the container * removed old secrets sample, edited IMAGEVERSION in .env.sample to match our versioning scheme * changed help text for --list option for clarity * updated help text for test_main * Cleanup (#45) * added callback for --list option * Update README.md * Updated hard-coded vars to config.py * bumped version --------- Co-authored-by: Joseph Hendrix <[email protected]> Co-authored-by: Joseph Hendrix <[email protected]> Co-authored-by: Magret <[email protected]> Co-authored-by: Magret Adekunle <[email protected]> * Development (#49) * added PubMed (refactored to OOP) module and scaffolding for IEEE module * initialized poetry project * added (blank) __init__.py to package and test directories * added testing, logging to PubMed.py (WIP) * further testing * added python venv dirs to .gitignore * .gitignore additions, formatting/linter changes * Added arXiv python file * Added python file for Springer API * Added python file for Elsevier API * Added python file for Wiley API * Added test for arXvip and updated code for arXvip to standardize name variations * Git added test for Spinger API * Updated code for test_Springer and Spinger.py for name Normalizion * Updated code to be in class format * deleted redundant comments * Updated code for Elsevier.py * Fixed Commenting * Updated code for Wiley.py normalize_author_name * added docstrings, better testing coverage * better error logging in PubMed.py * Fixed code format * Fixed standardize author name for all API * Hendrix additions (#1) * added PubMed (refactored to OOP) module and scaffolding for IEEE module * initialized poetry project * added (blank) __init__.py to package and test directories * added testing, logging to PubMed.py (WIP) * further testing * added python venv dirs to .gitignore * .gitignore additions, formatting/linter changes * deleted redundant comments * added docstrings, better testing coverage * better error logging in PubMed.py --------- Co-authored-by: Joseph Hendrix <[email protected]> Co-authored-by: Joseph Hendrix <[email protected]> * Fixed .gitignore file * Updated test_PubMed.py yo mach with recent changes * Updated to PubMed.py for merging with dev * Fixed returning authors for arXiv * Fixed standardize_author_name for Elsevier * Update Elsevier.py * Added try/catch for Elsevier and arXiv * Added try/catch for PubMed * Added try/catch to Springer * add try/catch for Wiley api * fixed imports and API keys * Added PLOS API along with try/catch * pyproject changes ensure extending from a Base class doesn't break pytest * removed old pubmed script at root of project * API modules extend base class, added secrets template * moved URLs to config.py * added functionality to main.py (WIP) * added bumpversion, poetry script entry point * main.py can now accept command line arguments (WIP) * fixed erroneous method call in MDPI class, added root logger object to main.py * Updated arXive test and code * Added more test for arXiv * Updated formate * Base class, main CLI script (#5) * pyproject changes ensure extending from a Base class doesn't break pytest * removed old pubmed script at root of project * API modules extend base class, added secrets template * moved URLs to config.py * added functionality to main.py (WIP) * added bumpversion, poetry script entry point * main.py can now accept command line arguments (WIP) * fixed erroneous method call in MDPI class, added root logger object to main.py * Updated with baseclass * Fxied main * Added Wiley Api and fixed nameing convention of DOI's * Updated config with urls also connecting new api files with Base * Cli additions (#9) * todo comments * added example input file, author names gathered from file * main script writes API results to (user configurable) file * added api-keyed endpoints to main script * updated .gitignore * crossref into cli_additions (#8) * added CrossRef module and tests (WIP) * crossref data extraction (WIP) * barebones crossref functionality * added full CrossRef functionality, began testing * CrossRef tests and accompanying module changes --------- Co-authored-by: Joseph Hendrix <[email protected]> * added functionality to specify targeted APIs through command line * removed obsolete requirements.txt * Updated README.md updated README.md to reflect command line changes * minor updates to secret.sample for clarity * updated TODO comments in main --------- Co-authored-by: Joseph Hendrix <[email protected]> Co-authored-by: Magret Adekunle <[email protected]> * Better testing (#11) * created dev dependency group * removed extraneous config from project root * added main.py tests * added description to pyproject.toml * better pubmd tests (WIP) * mock test for PubMed module * pubmed tests no longer query PubMed API * Springer tests no longer query Springer API * removed bad test from test_CrossRef.py (testing API rather than pubscraper functionality) * cleaned some comments * List apis (#12) * added --list flag to list APIs available to query * added help message for --apis and --list * prettier --list output * Updated Main to have PLOS/Weily and testing for PLOS/arXiv * Added test for Elsevier * Child class changes (#14) * privatized crossref methods, added DOI * linter changes, privatized Elsevier methods, changed result format * privatized MDPI methods, added DOI * PLOS linter changes, added DOI * added DOI to PubMed, privatized methods * privatized Springer methods, changed result formatting * privatized Wiley methods, changed result formatting * privatized arXiv methods, changed output formatting * added tests for --list CLI flag * minor typo fixes in comments * better logging on startup * Containerize (#15) * added Dockerfile (builds, no run) * changed some import statements to fix ModuleNotFoundErrors when running from wheel build * Added tablib to export data * Crossref fixes (#16) * privatized crossref methods, added DOI * linter changes, privatized Elsevier methods, changed result format * privatized MDPI methods, added DOI * PLOS linter changes, added DOI * added DOI to PubMed, privatized methods * privatized Springer methods, changed result formatting * privatized Wiley methods, changed result formatting * privatized arXiv methods, changed output formatting * added tests for --list CLI flag * minor typo fixes in comments * aggregating results is broken, all other tests pass * fixed a bad check in CrossRef.py where 0 valid publications would mistakenly break out of query loop * Aded changes to README.md * Update README.md * Update README.md * Updated MDPI and main.py * Updated MDPI.py and main.py * Update README.md * Main fixes (#20) * minor Dockerfile fixes in COPY statements * updated CLI options and their tests * changed input file and input parsing to better reflect expected input * Update README.md * Updated main * Updted main and sub class for logger * Updated ArXive date * Standardize CrossRef publication date * Standardize Elsevier publication date * Standardize MDPI publication date * Standardize PLOS publication date * Standardize PubMed publication date * Standardize Springer publication date * Standardize Wiley publication date * Standardize the publication date to "YYYY-MM-DD" * Updated main.py to allows user to pull publication from cutoff date and test for main.py * Added Documentations for xlsx * updated README for flag `--cutoff_date` or `-cd` * Update README.md * Update README.md * Added mock responses for test_Elsevier.py * Added Test for test_CrossRef.py * Update README.md * Docker io (#35) * script now reads a (default) xlsx file instead of a csv * added run.sh script (WIP) * added .env support and example for storing secrets * Update README.md * refactored API list to a const in main.py * changed --input to click.Path object * removed old files from Dockerfile * moved input and default input/output files value to /data directory, mount /data directory to docker container instead of the whole project * quick poetry.lock fix * Updated MDPI for pull date * Added logging.debug for MDPI.py * Updated code for pull date (CrossRef.py) * Updated code for Elsevire.py (date and added logging.debug) * Added code for loggin.debug * Added logging.debug for PLOS.py * Updated pull date from PubMed.py * Added logging.debug to Spinger.py * Added loggin.debug to Wiley.py * Updated authors naming * Bumpversion fixes (#37) * added .env version spec to pyproject; bumpversion now updates .env * Update README.md * got rid of /data folder (too confusing) * Update README.md * Added testing for test_MDPI.py * Updated test_Elsevier.py to use .env * Added testting for test_Elsevire.py * Wiley tests (#42) * added Wiley tests (WIP) * added Wiley tests * Development (#41) * Added testing for test_MDPI.py * Updated test_Elsevier.py to use .env * Added testting for test_Elsevire.py --------- Co-authored-by: Magret <[email protected]> Co-authored-by: Magret Adekunle <[email protected]> * fixed main.py tests (all tests pass!) --------- Co-authored-by: Magret <[email protected]> Co-authored-by: Magret Adekunle <[email protected]> * Runner versioning (#43) * run.sh now properly reads IMAGEVERSION environment variable from .env for building the container * removed old secrets sample, edited IMAGEVERSION in .env.sample to match our versioning scheme * changed help text for --list option for clarity * updated help text for test_main * Cleanup (#45) * added callback for --list option * Update README.md * Updated hard-coded vars to config.py * bumped version * added actions yaml files (#48) * Actions (#50) * added actions yaml files * corrected job name in docker-image.yml --------- Co-authored-by: Joseph Hendrix <[email protected]> Co-authored-by: Joseph Hendrix <[email protected]> Co-authored-by: Magret <[email protected]> Co-authored-by: Magret Adekunle <[email protected]> * env fixes * env directory fixes --------- Co-authored-by: Joseph Hendrix <[email protected]> Co-authored-by: Joseph Hendrix <[email protected]> Co-authored-by: Magret <[email protected]> Co-authored-by: Magret Adekunle <[email protected]>
joeleehen
added a commit
that referenced
this pull request
Dec 17, 2024
* added PubMed (refactored to OOP) module and scaffolding for IEEE module * initialized poetry project * added (blank) __init__.py to package and test directories * added testing, logging to PubMed.py (WIP) * further testing * added python venv dirs to .gitignore * .gitignore additions, formatting/linter changes * Added arXiv python file * Added python file for Springer API * Added python file for Elsevier API * Added python file for Wiley API * Added test for arXvip and updated code for arXvip to standardize name variations * Git added test for Spinger API * Updated code for test_Springer and Spinger.py for name Normalizion * Updated code to be in class format * deleted redundant comments * Updated code for Elsevier.py * Fixed Commenting * Updated code for Wiley.py normalize_author_name * added docstrings, better testing coverage * better error logging in PubMed.py * Fixed code format * Fixed standardize author name for all API * Hendrix additions (#1) * added PubMed (refactored to OOP) module and scaffolding for IEEE module * initialized poetry project * added (blank) __init__.py to package and test directories * added testing, logging to PubMed.py (WIP) * further testing * added python venv dirs to .gitignore * .gitignore additions, formatting/linter changes * deleted redundant comments * added docstrings, better testing coverage * better error logging in PubMed.py --------- Co-authored-by: Joseph Hendrix <[email protected]> Co-authored-by: Joseph Hendrix <[email protected]> * Fixed .gitignore file * Updated test_PubMed.py yo mach with recent changes * Updated to PubMed.py for merging with dev * Fixed returning authors for arXiv * Fixed standardize_author_name for Elsevier * Update Elsevier.py * Added try/catch for Elsevier and arXiv * Added try/catch for PubMed * Added try/catch to Springer * add try/catch for Wiley api * fixed imports and API keys * Added PLOS API along with try/catch * pyproject changes ensure extending from a Base class doesn't break pytest * removed old pubmed script at root of project * API modules extend base class, added secrets template * moved URLs to config.py * added functionality to main.py (WIP) * added bumpversion, poetry script entry point * main.py can now accept command line arguments (WIP) * fixed erroneous method call in MDPI class, added root logger object to main.py * Updated arXive test and code * Added more test for arXiv * Updated formate * Base class, main CLI script (#5) * pyproject changes ensure extending from a Base class doesn't break pytest * removed old pubmed script at root of project * API modules extend base class, added secrets template * moved URLs to config.py * added functionality to main.py (WIP) * added bumpversion, poetry script entry point * main.py can now accept command line arguments (WIP) * fixed erroneous method call in MDPI class, added root logger object to main.py * Updated with baseclass * Fxied main * Added Wiley Api and fixed nameing convention of DOI's * Updated config with urls also connecting new api files with Base * Cli additions (#9) * todo comments * added example input file, author names gathered from file * main script writes API results to (user configurable) file * added api-keyed endpoints to main script * updated .gitignore * crossref into cli_additions (#8) * added CrossRef module and tests (WIP) * crossref data extraction (WIP) * barebones crossref functionality * added full CrossRef functionality, began testing * CrossRef tests and accompanying module changes --------- Co-authored-by: Joseph Hendrix <[email protected]> * added functionality to specify targeted APIs through command line * removed obsolete requirements.txt * Updated README.md updated README.md to reflect command line changes * minor updates to secret.sample for clarity * updated TODO comments in main --------- Co-authored-by: Joseph Hendrix <[email protected]> Co-authored-by: Magret Adekunle <[email protected]> * Better testing (#11) * created dev dependency group * removed extraneous config from project root * added main.py tests * added description to pyproject.toml * better pubmd tests (WIP) * mock test for PubMed module * pubmed tests no longer query PubMed API * Springer tests no longer query Springer API * removed bad test from test_CrossRef.py (testing API rather than pubscraper functionality) * cleaned some comments * List apis (#12) * added --list flag to list APIs available to query * added help message for --apis and --list * prettier --list output * Updated Main to have PLOS/Weily and testing for PLOS/arXiv * Added test for Elsevier * Child class changes (#14) * privatized crossref methods, added DOI * linter changes, privatized Elsevier methods, changed result format * privatized MDPI methods, added DOI * PLOS linter changes, added DOI * added DOI to PubMed, privatized methods * privatized Springer methods, changed result formatting * privatized Wiley methods, changed result formatting * privatized arXiv methods, changed output formatting * added tests for --list CLI flag * minor typo fixes in comments * better logging on startup * Containerize (#15) * added Dockerfile (builds, no run) * changed some import statements to fix ModuleNotFoundErrors when running from wheel build * Added tablib to export data * Crossref fixes (#16) * privatized crossref methods, added DOI * linter changes, privatized Elsevier methods, changed result format * privatized MDPI methods, added DOI * PLOS linter changes, added DOI * added DOI to PubMed, privatized methods * privatized Springer methods, changed result formatting * privatized Wiley methods, changed result formatting * privatized arXiv methods, changed output formatting * added tests for --list CLI flag * minor typo fixes in comments * aggregating results is broken, all other tests pass * fixed a bad check in CrossRef.py where 0 valid publications would mistakenly break out of query loop * Aded changes to README.md * Update README.md * Update README.md * Updated MDPI and main.py * Updated MDPI.py and main.py * Update README.md * Main fixes (#20) * minor Dockerfile fixes in COPY statements * updated CLI options and their tests * changed input file and input parsing to better reflect expected input * Update README.md * Updated main * Updted main and sub class for logger * Updated ArXive date * Standardize CrossRef publication date * Standardize Elsevier publication date * Standardize MDPI publication date * Standardize PLOS publication date * Standardize PubMed publication date * Standardize Springer publication date * Standardize Wiley publication date * Standardize the publication date to "YYYY-MM-DD" * Updated main.py to allows user to pull publication from cutoff date and test for main.py * Added Documentations for xlsx * updated README for flag `--cutoff_date` or `-cd` * Update README.md * Update README.md * Added mock responses for test_Elsevier.py * Added Test for test_CrossRef.py * Update README.md * Docker io (#35) * script now reads a (default) xlsx file instead of a csv * added run.sh script (WIP) * added .env support and example for storing secrets * Update README.md * refactored API list to a const in main.py * changed --input to click.Path object * removed old files from Dockerfile * moved input and default input/output files value to /data directory, mount /data directory to docker container instead of the whole project * quick poetry.lock fix * Updated MDPI for pull date * Added logging.debug for MDPI.py * Updated code for pull date (CrossRef.py) * Updated code for Elsevire.py (date and added logging.debug) * Added code for loggin.debug * Added logging.debug for PLOS.py * Updated pull date from PubMed.py * Added logging.debug to Spinger.py * Added loggin.debug to Wiley.py * Updated authors naming * Bumpversion fixes (#37) * added .env version spec to pyproject; bumpversion now updates .env * Update README.md * got rid of /data folder (too confusing) * Update README.md * Added testing for test_MDPI.py * Updated test_Elsevier.py to use .env * Added testting for test_Elsevire.py * Wiley tests (#42) * added Wiley tests (WIP) * added Wiley tests * Development (#41) * Added testing for test_MDPI.py * Updated test_Elsevier.py to use .env * Added testting for test_Elsevire.py --------- Co-authored-by: Magret <[email protected]> Co-authored-by: Magret Adekunle <[email protected]> * fixed main.py tests (all tests pass!) --------- Co-authored-by: Magret <[email protected]> Co-authored-by: Magret Adekunle <[email protected]> * Runner versioning (#43) * run.sh now properly reads IMAGEVERSION environment variable from .env for building the container * removed old secrets sample, edited IMAGEVERSION in .env.sample to match our versioning scheme * changed help text for --list option for clarity * updated help text for test_main * Cleanup (#45) * added callback for --list option * Update README.md * Updated hard-coded vars to config.py * bumped version * added actions yaml files (#48) * Actions (#50) * added actions yaml files * corrected job name in docker-image.yml * bumped version * Actions fix (#53) * Development (#47) * added PubMed (refactored to OOP) module and scaffolding for IEEE module * initialized poetry project * added (blank) __init__.py to package and test directories * added testing, logging to PubMed.py (WIP) * further testing * added python venv dirs to .gitignore * .gitignore additions, formatting/linter changes * Added arXiv python file * Added python file for Springer API * Added python file for Elsevier API * Added python file for Wiley API * Added test for arXvip and updated code for arXvip to standardize name variations * Git added test for Spinger API * Updated code for test_Springer and Spinger.py for name Normalizion * Updated code to be in class format * deleted redundant comments * Updated code for Elsevier.py * Fixed Commenting * Updated code for Wiley.py normalize_author_name * added docstrings, better testing coverage * better error logging in PubMed.py * Fixed code format * Fixed standardize author name for all API * Hendrix additions (#1) * added PubMed (refactored to OOP) module and scaffolding for IEEE module * initialized poetry project * added (blank) __init__.py to package and test directories * added testing, logging to PubMed.py (WIP) * further testing * added python venv dirs to .gitignore * .gitignore additions, formatting/linter changes * deleted redundant comments * added docstrings, better testing coverage * better error logging in PubMed.py --------- Co-authored-by: Joseph Hendrix <[email protected]> Co-authored-by: Joseph Hendrix <[email protected]> * Fixed .gitignore file * Updated test_PubMed.py yo mach with recent changes * Updated to PubMed.py for merging with dev * Fixed returning authors for arXiv * Fixed standardize_author_name for Elsevier * Update Elsevier.py * Added try/catch for Elsevier and arXiv * Added try/catch for PubMed * Added try/catch to Springer * add try/catch for Wiley api * fixed imports and API keys * Added PLOS API along with try/catch * pyproject changes ensure extending from a Base class doesn't break pytest * removed old pubmed script at root of project * API modules extend base class, added secrets template * moved URLs to config.py * added functionality to main.py (WIP) * added bumpversion, poetry script entry point * main.py can now accept command line arguments (WIP) * fixed erroneous method call in MDPI class, added root logger object to main.py * Updated arXive test and code * Added more test for arXiv * Updated formate * Base class, main CLI script (#5) * pyproject changes ensure extending from a Base class doesn't break pytest * removed old pubmed script at root of project * API modules extend base class, added secrets template * moved URLs to config.py * added functionality to main.py (WIP) * added bumpversion, poetry script entry point * main.py can now accept command line arguments (WIP) * fixed erroneous method call in MDPI class, added root logger object to main.py * Updated with baseclass * Fxied main * Added Wiley Api and fixed nameing convention of DOI's * Updated config with urls also connecting new api files with Base * Cli additions (#9) * todo comments * added example input file, author names gathered from file * main script writes API results to (user configurable) file * added api-keyed endpoints to main script * updated .gitignore * crossref into cli_additions (#8) * added CrossRef module and tests (WIP) * crossref data extraction (WIP) * barebones crossref functionality * added full CrossRef functionality, began testing * CrossRef tests and accompanying module changes --------- Co-authored-by: Joseph Hendrix <[email protected]> * added functionality to specify targeted APIs through command line * removed obsolete requirements.txt * Updated README.md updated README.md to reflect command line changes * minor updates to secret.sample for clarity * updated TODO comments in main --------- Co-authored-by: Joseph Hendrix <[email protected]> Co-authored-by: Magret Adekunle <[email protected]> * Better testing (#11) * created dev dependency group * removed extraneous config from project root * added main.py tests * added description to pyproject.toml * better pubmd tests (WIP) * mock test for PubMed module * pubmed tests no longer query PubMed API * Springer tests no longer query Springer API * removed bad test from test_CrossRef.py (testing API rather than pubscraper functionality) * cleaned some comments * List apis (#12) * added --list flag to list APIs available to query * added help message for --apis and --list * prettier --list output * Updated Main to have PLOS/Weily and testing for PLOS/arXiv * Added test for Elsevier * Child class changes (#14) * privatized crossref methods, added DOI * linter changes, privatized Elsevier methods, changed result format * privatized MDPI methods, added DOI * PLOS linter changes, added DOI * added DOI to PubMed, privatized methods * privatized Springer methods, changed result formatting * privatized Wiley methods, changed result formatting * privatized arXiv methods, changed output formatting * added tests for --list CLI flag * minor typo fixes in comments * better logging on startup * Containerize (#15) * added Dockerfile (builds, no run) * changed some import statements to fix ModuleNotFoundErrors when running from wheel build * Added tablib to export data * Crossref fixes (#16) * privatized crossref methods, added DOI * linter changes, privatized Elsevier methods, changed result format * privatized MDPI methods, added DOI * PLOS linter changes, added DOI * added DOI to PubMed, privatized methods * privatized Springer methods, changed result formatting * privatized Wiley methods, changed result formatting * privatized arXiv methods, changed output formatting * added tests for --list CLI flag * minor typo fixes in comments * aggregating results is broken, all other tests pass * fixed a bad check in CrossRef.py where 0 valid publications would mistakenly break out of query loop * Aded changes to README.md * Update README.md * Update README.md * Updated MDPI and main.py * Updated MDPI.py and main.py * Update README.md * Main fixes (#20) * minor Dockerfile fixes in COPY statements * updated CLI options and their tests * changed input file and input parsing to better reflect expected input * Update README.md * Updated main * Updted main and sub class for logger * Updated ArXive date * Standardize CrossRef publication date * Standardize Elsevier publication date * Standardize MDPI publication date * Standardize PLOS publication date * Standardize PubMed publication date * Standardize Springer publication date * Standardize Wiley publication date * Standardize the publication date to "YYYY-MM-DD" * Updated main.py to allows user to pull publication from cutoff date and test for main.py * Added Documentations for xlsx * updated README for flag `--cutoff_date` or `-cd` * Update README.md * Update README.md * Added mock responses for test_Elsevier.py * Added Test for test_CrossRef.py * Update README.md * Docker io (#35) * script now reads a (default) xlsx file instead of a csv * added run.sh script (WIP) * added .env support and example for storing secrets * Update README.md * refactored API list to a const in main.py * changed --input to click.Path object * removed old files from Dockerfile * moved input and default input/output files value to /data directory, mount /data directory to docker container instead of the whole project * quick poetry.lock fix * Updated MDPI for pull date * Added logging.debug for MDPI.py * Updated code for pull date (CrossRef.py) * Updated code for Elsevire.py (date and added logging.debug) * Added code for loggin.debug * Added logging.debug for PLOS.py * Updated pull date from PubMed.py * Added logging.debug to Spinger.py * Added loggin.debug to Wiley.py * Updated authors naming * Bumpversion fixes (#37) * added .env version spec to pyproject; bumpversion now updates .env * Update README.md * got rid of /data folder (too confusing) * Update README.md * Added testing for test_MDPI.py * Updated test_Elsevier.py to use .env * Added testting for test_Elsevire.py * Wiley tests (#42) * added Wiley tests (WIP) * added Wiley tests * Development (#41) * Added testing for test_MDPI.py * Updated test_Elsevier.py to use .env * Added testting for test_Elsevire.py --------- Co-authored-by: Magret <[email protected]> Co-authored-by: Magret Adekunle <[email protected]> * fixed main.py tests (all tests pass!) --------- Co-authored-by: Magret <[email protected]> Co-authored-by: Magret Adekunle <[email protected]> * Runner versioning (#43) * run.sh now properly reads IMAGEVERSION environment variable from .env for building the container * removed old secrets sample, edited IMAGEVERSION in .env.sample to match our versioning scheme * changed help text for --list option for clarity * updated help text for test_main * Cleanup (#45) * added callback for --list option * Update README.md * Updated hard-coded vars to config.py * bumped version --------- Co-authored-by: Joseph Hendrix <[email protected]> Co-authored-by: Joseph Hendrix <[email protected]> Co-authored-by: Magret <[email protected]> Co-authored-by: Magret Adekunle <[email protected]> * Development (#49) * added PubMed (refactored to OOP) module and scaffolding for IEEE module * initialized poetry project * added (blank) __init__.py to package and test directories * added testing, logging to PubMed.py (WIP) * further testing * added python venv dirs to .gitignore * .gitignore additions, formatting/linter changes * Added arXiv python file * Added python file for Springer API * Added python file for Elsevier API * Added python file for Wiley API * Added test for arXvip and updated code for arXvip to standardize name variations * Git added test for Spinger API * Updated code for test_Springer and Spinger.py for name Normalizion * Updated code to be in class format * deleted redundant comments * Updated code for Elsevier.py * Fixed Commenting * Updated code for Wiley.py normalize_author_name * added docstrings, better testing coverage * better error logging in PubMed.py * Fixed code format * Fixed standardize author name for all API * Hendrix additions (#1) * added PubMed (refactored to OOP) module and scaffolding for IEEE module * initialized poetry project * added (blank) __init__.py to package and test directories * added testing, logging to PubMed.py (WIP) * further testing * added python venv dirs to .gitignore * .gitignore additions, formatting/linter changes * deleted redundant comments * added docstrings, better testing coverage * better error logging in PubMed.py --------- Co-authored-by: Joseph Hendrix <[email protected]> Co-authored-by: Joseph Hendrix <[email protected]> * Fixed .gitignore file * Updated test_PubMed.py yo mach with recent changes * Updated to PubMed.py for merging with dev * Fixed returning authors for arXiv * Fixed standardize_author_name for Elsevier * Update Elsevier.py * Added try/catch for Elsevier and arXiv * Added try/catch for PubMed * Added try/catch to Springer * add try/catch for Wiley api * fixed imports and API keys * Added PLOS API along with try/catch * pyproject changes ensure extending from a Base class doesn't break pytest * removed old pubmed script at root of project * API modules extend base class, added secrets template * moved URLs to config.py * added functionality to main.py (WIP) * added bumpversion, poetry script entry point * main.py can now accept command line arguments (WIP) * fixed erroneous method call in MDPI class, added root logger object to main.py * Updated arXive test and code * Added more test for arXiv * Updated formate * Base class, main CLI script (#5) * pyproject changes ensure extending from a Base class doesn't break pytest * removed old pubmed script at root of project * API modules extend base class, added secrets template * moved URLs to config.py * added functionality to main.py (WIP) * added bumpversion, poetry script entry point * main.py can now accept command line arguments (WIP) * fixed erroneous method call in MDPI class, added root logger object to main.py * Updated with baseclass * Fxied main * Added Wiley Api and fixed nameing convention of DOI's * Updated config with urls also connecting new api files with Base * Cli additions (#9) * todo comments * added example input file, author names gathered from file * main script writes API results to (user configurable) file * added api-keyed endpoints to main script * updated .gitignore * crossref into cli_additions (#8) * added CrossRef module and tests (WIP) * crossref data extraction (WIP) * barebones crossref functionality * added full CrossRef functionality, began testing * CrossRef tests and accompanying module changes --------- Co-authored-by: Joseph Hendrix <[email protected]> * added functionality to specify targeted APIs through command line * removed obsolete requirements.txt * Updated README.md updated README.md to reflect command line changes * minor updates to secret.sample for clarity * updated TODO comments in main --------- Co-authored-by: Joseph Hendrix <[email protected]> Co-authored-by: Magret Adekunle <[email protected]> * Better testing (#11) * created dev dependency group * removed extraneous config from project root * added main.py tests * added description to pyproject.toml * better pubmd tests (WIP) * mock test for PubMed module * pubmed tests no longer query PubMed API * Springer tests no longer query Springer API * removed bad test from test_CrossRef.py (testing API rather than pubscraper functionality) * cleaned some comments * List apis (#12) * added --list flag to list APIs available to query * added help message for --apis and --list * prettier --list output * Updated Main to have PLOS/Weily and testing for PLOS/arXiv * Added test for Elsevier * Child class changes (#14) * privatized crossref methods, added DOI * linter changes, privatized Elsevier methods, changed result format * privatized MDPI methods, added DOI * PLOS linter changes, added DOI * added DOI to PubMed, privatized methods * privatized Springer methods, changed result formatting * privatized Wiley methods, changed result formatting * privatized arXiv methods, changed output formatting * added tests for --list CLI flag * minor typo fixes in comments * better logging on startup * Containerize (#15) * added Dockerfile (builds, no run) * changed some import statements to fix ModuleNotFoundErrors when running from wheel build * Added tablib to export data * Crossref fixes (#16) * privatized crossref methods, added DOI * linter changes, privatized Elsevier methods, changed result format * privatized MDPI methods, added DOI * PLOS linter changes, added DOI * added DOI to PubMed, privatized methods * privatized Springer methods, changed result formatting * privatized Wiley methods, changed result formatting * privatized arXiv methods, changed output formatting * added tests for --list CLI flag * minor typo fixes in comments * aggregating results is broken, all other tests pass * fixed a bad check in CrossRef.py where 0 valid publications would mistakenly break out of query loop * Aded changes to README.md * Update README.md * Update README.md * Updated MDPI and main.py * Updated MDPI.py and main.py * Update README.md * Main fixes (#20) * minor Dockerfile fixes in COPY statements * updated CLI options and their tests * changed input file and input parsing to better reflect expected input * Update README.md * Updated main * Updted main and sub class for logger * Updated ArXive date * Standardize CrossRef publication date * Standardize Elsevier publication date * Standardize MDPI publication date * Standardize PLOS publication date * Standardize PubMed publication date * Standardize Springer publication date * Standardize Wiley publication date * Standardize the publication date to "YYYY-MM-DD" * Updated main.py to allows user to pull publication from cutoff date and test for main.py * Added Documentations for xlsx * updated README for flag `--cutoff_date` or `-cd` * Update README.md * Update README.md * Added mock responses for test_Elsevier.py * Added Test for test_CrossRef.py * Update README.md * Docker io (#35) * script now reads a (default) xlsx file instead of a csv * added run.sh script (WIP) * added .env support and example for storing secrets * Update README.md * refactored API list to a const in main.py * changed --input to click.Path object * removed old files from Dockerfile * moved input and default input/output files value to /data directory, mount /data directory to docker container instead of the whole project * quick poetry.lock fix * Updated MDPI for pull date * Added logging.debug for MDPI.py * Updated code for pull date (CrossRef.py) * Updated code for Elsevire.py (date and added logging.debug) * Added code for loggin.debug * Added logging.debug for PLOS.py * Updated pull date from PubMed.py * Added logging.debug to Spinger.py * Added loggin.debug to Wiley.py * Updated authors naming * Bumpversion fixes (#37) * added .env version spec to pyproject; bumpversion now updates .env * Update README.md * got rid of /data folder (too confusing) * Update README.md * Added testing for test_MDPI.py * Updated test_Elsevier.py to use .env * Added testting for test_Elsevire.py * Wiley tests (#42) * added Wiley tests (WIP) * added Wiley tests * Development (#41) * Added testing for test_MDPI.py * Updated test_Elsevier.py to use .env * Added testting for test_Elsevire.py --------- Co-authored-by: Magret <[email protected]> Co-authored-by: Magret Adekunle <[email protected]> * fixed main.py tests (all tests pass!) --------- Co-authored-by: Magret <[email protected]> Co-authored-by: Magret Adekunle <[email protected]> * Runner versioning (#43) * run.sh now properly reads IMAGEVERSION environment variable from .env for building the container * removed old secrets sample, edited IMAGEVERSION in .env.sample to match our versioning scheme * changed help text for --list option for clarity * updated help text for test_main * Cleanup (#45) * added callback for --list option * Update README.md * Updated hard-coded vars to config.py * bumped version * added actions yaml files (#48) * Actions (#50) * added actions yaml files * corrected job name in docker-image.yml --------- Co-authored-by: Joseph Hendrix <[email protected]> Co-authored-by: Joseph Hendrix <[email protected]> Co-authored-by: Magret <[email protected]> Co-authored-by: Magret Adekunle <[email protected]> * env fixes * env directory fixes --------- Co-authored-by: Joseph Hendrix <[email protected]> Co-authored-by: Joseph Hendrix <[email protected]> Co-authored-by: Magret <[email protected]> Co-authored-by: Magret Adekunle <[email protected]> * patched versions --------- Co-authored-by: Joseph Hendrix <[email protected]> Co-authored-by: Joseph Hendrix <[email protected]> Co-authored-by: Magret <[email protected]> Co-authored-by: Magret Adekunle <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.