Skip to content

Commit

Permalink
Feature/frontend (#10)
Browse files Browse the repository at this point in the history
* (ui) initial commit for UI app
* (ci) added npm
* (docs) updated screenshot
* (ui) protocol on embed
  • Loading branch information
kevincobain2000 authored May 5, 2024
1 parent ee83faa commit bc01ec9
Show file tree
Hide file tree
Showing 44 changed files with 8,794 additions and 21 deletions.
45 changes: 45 additions & 0 deletions .air.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# .air.conf
# Config file for [Air](https://github.com/cosmtrek/air) in TOML format

# Working directory
# . or absolute path, please note that the directories following must be under root.
root = "."
tmp_dir = "tmp"

[build]
# Just plain old shell command. You could use `make` as well.
cmd = "go mod tidy && go build -o ./tmp/main main.go"
# Binary file yields from `cmd`.
bin = "tmp/main"
# Customize binary.
# allow cors for astro
full_bin = "PP_USER=air ./tmp/main --cors=4321"
# Watch these filename extensions.
include_ext = ["go", "tpl", "tmpl", "html", "env", "conf"]
# Ignore these filename extensions or directories.
exclude_dir = ["assets", "tmp", "vendor", "dist", "node_modules", "public", "frontend"]
# Watch these directories if you specified.
include_dir = []
# Exclude files.
exclude_file = []
# It's not necessary to trigger build each time file changes if it's too frequent.
delay = 1000 # ms
# Stop to run old binary when build errors occur.
stop_on_error = true
# This log file places in your tmp_dir.
log = "air_errors.log"

[log]
# Show log time
time = false

[color]
# Customize each part's color. If no color found, use the raw app log.
main = "magenta"
watcher = "cyan"
build = "yellow"
runner = "green"

[misc]
# Delete tmp directory on exit
clean_on_exit = true
14 changes: 14 additions & 0 deletions .github/workflows/coveritup.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,20 @@ jobs:
go install github.com/golangci/golangci-lint/cmd/golangci-lint@latest
curl -sLk https://raw.githubusercontent.com/kevincobain2000/cover-totalizer/master/install.sh | sh
- name: NPM Install
uses: kevincobain2000/action-coveritup@v2
with:
type: npm-install-time
command: cd frontend; npm install
record: runtime

- name: NPM Build
uses: kevincobain2000/action-coveritup@v2
with:
type: npm-build-time
command: cd frontend; npm run build
record: runtime

- name: Go Mod Tidy
run: go mod tidy
- name: Go Build
Expand Down
25 changes: 24 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -5,4 +5,27 @@ disqus_jnlp.html.rst
build
dist
*egg*
emails.txt
emails.txt

# Binaries for programs and plugins
*.dll
*.so
*.dylib

# Test binary, build with `go test -c`
*.test

# Output of the go coverage tool, specifically when used with LiteIDE
*.out
# diskcache directory
cache/*
vendor/*

tmp/

.env
*.pid
dist/
bin/
main
.DS_Store
53 changes: 41 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,11 @@ A free utility to extract email address by crawling a given url upto a given dep
Web email extractor by url crawl using command line interface. Email addresses can be extracted from any url.
First it extracts all the number of urls provided by the user and at the same time extracts emails using simple Go routines. This simple application allows to crawl through a website with depth options.


<p align="center">
<a href="https://email-extractor.coveritup.app">
<img alt="free email extractor online app url" src="https://imgur.com/z9nS7xm.png" width="460">
</a>
</p>

**Quick Setup:** One command to install lighweight binary.

Expand All @@ -26,17 +30,21 @@ First it extracts all the number of urls provided by the user and at the same ti

# C.I

![go-build-time](https://coveritup.app/badge?org=kevincobain2000&repo=email_extractor&type=go-build-time&branch=master)
![go-test-runtime](https://coveritup.app/badge?org=kevincobain2000&repo=email_extractor&type=go-test-runtime&branch=master)
![coverage](https://coveritup.app/badge?org=kevincobain2000&repo=email_extractor&type=coverage&branch=master)
![go-binary-size](https://coveritup.app/badge?org=kevincobain2000&repo=email_extractor&type=go-binary-size&branch=master)
![go-mod-dependencies](https://coveritup.app/badge?org=kevincobain2000&repo=email_extractor&type=go-mod-dependencies&branch=master)
![go-build-time](https://coveritup.app/badge?org=kevincobain2000&repo=email_extractor&branch=master&output=svg&type=go-build-time)
![go-test-runtime](https://coveritup.app/badge?org=kevincobain2000&repo=email_extractor&branch=master&output=svg&type=go-test-runtime)
![coverage](https://coveritup.app/badge?org=kevincobain2000&repo=email_extractor&branch=master&output=svg&type=coverage)
![go-binary-size](https://coveritup.app/badge?org=kevincobain2000&repo=email_extractor&branch=master&output=svg&type=go-binary-size)
![go-mod-dependencies](https://coveritup.app/badge?org=kevincobain2000&repo=email_extractor&branch=master&output=svg&type=go-mod-dependencies)
![npm-install-time](https://coveritup.app/badge?org=kevincobain2000&repo=email_extractor&branch=master&output=svg&type=npm-install-time)
![npm-build-time](https://coveritup.app/badge?org=kevincobain2000&repo=email_extractor&branch=master&output=svg&type=npm-build-time)

![go-build-time](https://coveritup.app/chart?org=kevincobain2000&repo=email_extractor&type=go-build-time&output=svg&width=160&height=160&branch=master&line=fill)
![go-test-runtime](https://coveritup.app/chart?org=kevincobain2000&repo=email_extractor&type=go-test-runtime&output=svg&width=160&height=160&branch=master&line=fill)
![coverage](https://coveritup.app/chart?org=kevincobain2000&repo=email_extractor&type=coverage&output=svg&width=160&height=160&branch=master)
![go-binary-size](https://coveritup.app/chart?org=kevincobain2000&repo=email_extractor&type=go-binary-size&output=svg&width=160&height=160&branch=master)
![go-mod-dependencies](https://coveritup.app/chart?org=kevincobain2000&repo=email_extractor&type=go-mod-dependencies&output=svg&width=160&height=160&branch=master&line=fill)
![go-build-time](https://coveritup.app/chart?org=kevincobain2000&repo=email_extractor&branch=master&output=svg&type=go-build-time&theme=light&line=fill&width=175&height=160)
![go-test-runtime](https://coveritup.app/chart?org=kevincobain2000&repo=email_extractor&branch=master&output=svg&type=go-test-runtime&theme=light&line=fill&width=175&height=160)
![coverage](https://coveritup.app/chart?org=kevincobain2000&repo=email_extractor&branch=master&output=svg&type=coverage&theme=light&line=fill&width=175&height=160)
![go-binary-size](https://coveritup.app/chart?org=kevincobain2000&repo=email_extractor&branch=master&output=svg&type=go-binary-size&theme=light&line=fill&width=175&height=160)
![go-mod-dependencies](https://coveritup.app/chart?org=kevincobain2000&repo=email_extractor&branch=master&output=svg&type=go-mod-dependencies&theme=light&line=fill&width=175&height=160)
![npm-install-time](https://coveritup.app/chart?org=kevincobain2000&repo=email_extractor&branch=master&output=svg&type=npm-install-time&theme=light&line=fill&width=175&height=160)
![npm-build-time](https://coveritup.app/chart?org=kevincobain2000&repo=email_extractor&branch=master&output=svg&type=npm-build-time&theme=light&line=fill&width=175&height=160)


# Install
Expand All @@ -52,11 +60,16 @@ mv email_extractor /usr/local/bin/
**Simple usage**

```sh
# using web GUI
email_extractor

# or using CLI
email_extractor -url=kevincobain2000.github.io
```

**Advanced usages**

Cli application

```sh
# Do not crawl urls
Expand All @@ -67,6 +80,9 @@ email_extractor -out=marketing.txt -url=kevincobain2000.github.io

#extract from 100 urls
email_extractor -limit-urls=100 -url=kevincobain2000.github.io

# run on a different port (for web GUI)
email_extractor --port=8080
```

**All Options**
Expand Down Expand Up @@ -110,9 +126,21 @@ email_extractor -limit-urls=100 -url=kevincobain2000.github.io
It crawled `1000 urls`, and found `300 email addresses` in about `11 seconds`.
<p align="center">
<img alt="bar chart" src='https://instachart.coveritup.app/bar?title=Performance&subtitle=Email+Extractor&output=svg&metric=sec&theme=dark&data={%20%22x%22:%20[%22100%20URLS%22,%20%22500%20URLS%22,%20%221000%20URLS%22],%20%22y%22:%20[[1,6,11]],%20%22names%22:%20[%22Time%20to%20Extract%22]%20}'>
<img alt="bar chart" src='https://instachart.coveritup.app/bar?title=Performance&subtitle=Email+Extractor&output=svg&metric=sec&theme=light&data={%20%22x%22:%20[%22100%20URLS%22,%20%22500%20URLS%22,%20%221000%20URLS%22],%20%22y%22:%20[[1,6,11]],%20%22names%22:%20[%22Time%20to%20Extract%22]%20}'>
</p>
# Development notes
```sh
# for backend
air # or go run main.go

# for frontend
cd fronend
npm install
npm run dev
```
# CHANGE LOG
- v1.0 - Python implementation to extract email addresses by crawling URLS. Installation using pip.
Expand All @@ -122,3 +150,4 @@ It crawled `1000 urls`, and found `300 email addresses` in about `11 seconds`.
- v2.8 - Limit emails addresses, and possible fix on relative urls
- v2.10 - Adds hint for status code
- v2.12 - Option to do in parallel and better messaging
- v3.0 - Web UI
2 changes: 2 additions & 0 deletions frontend/.env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
PUBLIC_GOOGLE_ANALYTICS_ID=
PUBLIC_BASE_URL=http://localhost:3003
19 changes: 19 additions & 0 deletions frontend/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# generated types
.astro/

# dependencies
node_modules/

# logs
npm-debug.log*
yarn-debug.log*
yarn-error.log*
pnpm-debug.log*

# environment variables
.env
.env.production

# macOS-specific files
.DS_Store
dist/
4 changes: 4 additions & 0 deletions frontend/.vscode/extensions.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
{
"recommendations": ["astro-build.astro-vscode"],
"unwantedRecommendations": []
}
11 changes: 11 additions & 0 deletions frontend/.vscode/launch.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
{
"version": "0.2.0",
"configurations": [
{
"command": "./node_modules/.bin/astro dev",
"name": "Development server",
"request": "launch",
"type": "node-terminal"
}
]
}
8 changes: 8 additions & 0 deletions frontend/astro.config.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
import astroSingleFile from 'astro-single-file'
import { defineConfig } from 'astro/config'
import tailwind from '@astrojs/tailwind'

// https://astro.build/config
export default defineConfig({
integrations: [astroSingleFile(), tailwind()]
})
Loading

0 comments on commit bc01ec9

Please sign in to comment.