Skip to content

An application that scrapes a web page, generates reports and exposes a RESTful web service for querying scraped data.

License

Notifications You must be signed in to change notification settings

clearfountain/Web-scraper

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

4 Commits
 
 
 
 
 
 
 
 

Repository files navigation

Web-scraper

An application that scrapes a web page, generates reports and exposes a RESTful web service for querying scraped data.

Architecture

Built following Domain Driven Design and Clean Architecture. Application layer contains logic.\ operated on entities in the domain layer. Domain layer consists of entities relating to real life domain. Infrastructure layer consists of persistence and is used from the applicztion layer.

#Endpoints /api/Reports - Generate excel file from json input. /api/Reports/get-hotel-rates - Get hotel rates from json file. /api/scrape - Scrape web page and extract hotel date.

How to run

Open project in Visual Studio and Run, project runs at http://localhost:5151/swagger/index.html showing a swagger page of the endpoints.

Suggestion

Reports can be generated periodically using a background job via Hangfire.

Extra

Generated files appear inf 'files' folder in API folder

About

An application that scrapes a web page, generates reports and exposes a RESTful web service for querying scraped data.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published