4.6.0

Package Maintainer Changes

We have dropped using pyppeteer and moved to using PlayWright https://playwright.dev/ for the screenshot module

Dev Changes:

Moved away from isort,black and flake8 for ruff for linting and formatting as ruff can do all that and more, not to mention is crazy fast.

What's Changed

• chore(deps): bump fastapi from 0.109.0 to 0.109.1 in /requirements by @dependabot in #1649
• chore(deps): bump slowapi from 0.1.8 to 0.1.9 by @dependabot in #1651
• chore(deps): bump fastapi from 0.109.1 to 0.109.2 by @dependabot in #1652
• chore(deps-dev): bump pydantic from 2.5.3 to 2.6.1 by @dependabot in #1650
• chore(deps-dev): bump pytest-asyncio from 0.23.4 to 0.23.5 by @dependabot in #1654
• chore(deps): bump netaddr from 0.10.1 to 1.1.0 by @dependabot in #1659
• chore(deps-dev): bump black from 24.1.1 to 24.2.0 by @dependabot in #1656
• chore(deps): bump uvicorn from 0.27.0.post1 to 0.27.1 by @dependabot in #1658
• chore(deps): bump setuptools from 69.0.3 to 69.1.0 by @dependabot in #1655
• chore(deps): bump dnspython from 2.5.0 to 2.6.0 by @dependabot in #1661
• chore(deps): bump aiosqlite from 0.19.0 to 0.20.0 by @dependabot in #1667
• Fix ip-address parsing for IPv6 addresses by @jeffguy in #1670
• chore(deps): bump netaddr from 1.1.0 to 1.2.1 by @dependabot in #1665
• chore(deps): bump dnspython from 2.6.0 to 2.6.1 by @dependabot in #1663
• chore(deps): bump setuptools from 69.1.0 to 69.1.1 by @dependabot in #1668
• chore(deps): bump fastapi from 0.109.2 to 0.110.0 by @dependabot in #1671
• chore(deps-dev): bump pytest from 7.4.4 to 8.0.2 by @dependabot in #1672
• chore(deps): bump python-dateutil from 2.8.2 to 2.9.0.post0 by @dependabot in #1674
• chore(deps): bump playwright from 1.41.2 to 1.42.0 by @dependabot in #1676
• chore(deps-dev): bump ruff from 0.3.0 to 0.3.1 by @dependabot in #1677
• chore(deps-dev): bump pytest from 8.0.2 to 8.1.1 by @dependabot in #1687
• chore(deps-dev): bump types-requests from 2.31.0.6 to 2.31.0.20240311 by @dependabot in #1686
• chore(deps-dev): bump ruff from 0.3.1 to 0.3.2 by @dependabot in #1682
• chore(deps-dev): bump mypy from 1.8.0 to 1.9.0 by @dependabot in #1680
• chore(deps-dev): bump types-python-dateutil from 2.8.19.20240106 to 2.8.19.20240311 by @dependabot in #1685
• chore(deps): bump uvicorn from 0.27.1 to 0.28.0 by @dependabot in #1681
• chore(deps-dev): bump types-pyyaml from 6.0.12.12 to 6.0.12.20240311 by @dependabot in #1684
• chore(deps-dev): bump wheel from 0.42.0 to 0.43.0 by @dependabot in #1683
• chore(deps-dev): bump pytest-asyncio from 0.23.5 to 0.23.5.post1 by @dependabot in #1679
• chore(deps-dev): bump ruff from 0.3.2 to 0.3.3 by @dependabot in #1689
• chore(deps): bump setuptools from 69.1.1 to 69.2.0 by @dependabot in #1688
• chore(deps-dev): bump types-python-dateutil from 2.8.19.20240311 to 2.9.0.20240315 by @dependabot in #1690

New Contributors

• @jeffguy made their first contribution in #1670

Full Changelog: 4.5.1...4.6.0

4.5.1

What's Changed

• chore(deps): bump fastapi from 0.105.0 to 0.108.0 by @dependabot in #1618
• chore(deps-dev): bump black from 23.12.0 to 23.12.1 by @dependabot in #1615
• chore(deps-dev): bump mypy from 1.7.1 to 1.8.0 by @dependabot in #1613
• chore(deps): bump lxml from 4.9.3 to 4.9.4 by @dependabot in #1611
• chore(deps): bump uvicorn from 0.24.0.post1 to 0.25.0 by @dependabot in #1612
• chore(deps): bump shodan from 1.30.1 to 1.31.0 by @dependabot in #1610
• chore(deps-dev): bump pydantic from 2.5.2 to 2.5.3 by @dependabot in #1614
• chore(deps): bump setuptools from 69.0.2 to 69.0.3 by @dependabot in #1617
• chore(deps): bump lxml from 4.9.4 to 5.0.0 by @dependabot in #1619
• chore(deps-dev): bump pytest from 7.4.3 to 7.4.4 by @dependabot in #1621
• chore(deps): bump netaddr from 0.9.0 to 0.10.0 by @dependabot in #1622
• chore(deps-dev): bump pytest-asyncio from 0.23.2 to 0.23.3 by @dependabot in #1620
• chore(deps): bump netaddr from 0.10.0 to 0.10.1 by @dependabot in #1624
• chore(deps-dev): bump pyflakes from 3.1.0 to 3.2.0 by @dependabot in #1626
• chore(deps-dev): bump flake8 from 6.1.0 to 7.0.0 by @dependabot in #1627
• chore(deps): bump lxml from 5.0.0 to 5.0.1 by @dependabot in #1625
• Fix #1601 by @L1ghtn1ng in #1628
• chore(deps): bump fastapi from 0.108.0 to 0.109.0 by @dependabot in #1632
• chore(deps): bump lxml from 5.0.1 to 5.1.0 by @dependabot in #1631
• chore(deps): bump uvicorn from 0.25.0 to 0.27.0 by @dependabot in #1636
• chore(deps): bump beautifulsoup4 from 4.12.2 to 4.12.3 by @dependabot in #1635
• chore(deps): bump dnspython from 2.4.2 to 2.5.0 by @dependabot in #1637
• chore(deps-dev): bump types-python-dateutil from 2.8.19.14 to 2.8.19.20240106 by @dependabot in #1630
• chore(deps): bump aiohttp from 3.9.1 to 3.9.3 by @dependabot in #1646
• chore(deps): bump certifi from 2023.11.17 to 2024.2.2 by @dependabot in #1648
• chore(deps-dev): bump pytest-asyncio from 0.23.3 to 0.23.4 by @dependabot in #1644
• chore(deps): bump censys from 2.2.10 to 2.2.11 by @dependabot in #1647
• chore(deps-dev): bump black from 23.12.1 to 24.1.1 by @dependabot in #1642
• chore(deps): bump uvicorn from 0.27.0 to 0.27.0.post1 by @dependabot in #1645

Full Changelog: 4.5.0...4.5.1

4.5.0

Package Maintainer Notes

We have now migrated to pep517

Pipx

We now support pipx installs thanks to @branchvincent in #1505 and pipx is a great way to easy fix the change with how python and pip works as it creates the virtual environment for you and no more worrying about breaking your system packages. More information about pipx can be found here

What's Changed

• Fix typo in README by @LelouBil in #1545
• build: migrate to pep517 backend by @branchvincent in #1505
• Brave search module has been fixed
• fix(censys): update module by @thehappydinoa in #1556
• fixing issue with types in FastAPI by @dbfreem in #1586
• fix typing for core fetch_all by @dbfreem in #1589
• Dockerfile and compose fixes

Misc

• Other minor assortment of tidy ups and fixes
• chore(deps): bump netaddr from 0.8.0 to 0.9.0 by @dependabot in #1542
• chore(deps): bump censys from 2.2.5 to 2.2.6 by @dependabot in #1540
• chore(deps): bump setuptools from 68.2.0 to 68.2.2 by @dependabot in #1539
• chore(deps-dev): bump types-requests from 2.31.0.2 to 2.31.0.3 by @dependabot in #1544
• chore(deps): bump fastapi from 0.103.1 to 0.103.2 by @dependabot in #1552
• chore(deps-dev): bump types-pyyaml from 6.0.12.11 to 6.0.12.12 by @dependabot in #1548
• chore(deps-dev): bump pydantic from 2.3.0 to 2.4.2 by @dependabot in #1553
• chore(deps-dev): bump types-requests from 2.31.0.3 to 2.31.0.7 by @dependabot in #1555
• chore(deps): bump censys from 2.2.6 to 2.2.7 by @dependabot in #1557
• chore(deps): bump aiohttp from 3.8.5 to 3.8.6 by @dependabot in #1563
• chore(deps): bump aiodns from 3.0.0 to 3.1.0 by @dependabot in #1562
• chore(deps): bump uvloop from 0.17.0 to 0.18.0 by @dependabot in #1566
• chore(deps-dev): bump mypy from 1.5.1 to 1.6.0 by @dependabot in #1564
• chore(deps-dev): bump black from 23.9.1 to 23.10.0 by @dependabot in #1569
• chore(deps): bump censys from 2.2.7 to 2.2.8 by @dependabot in #1571
• chore(deps): bump fastapi from 0.103.2 to 0.104.0 by @dependabot in #1573
• chore(deps): bump shodan from 1.30.0 to 1.30.1 by @dependabot in #1567
• chore(deps): bump aiodns from 3.1.0 to 3.1.1 by @dependabot in #1568
• chore(deps-dev): bump mypy from 1.6.0 to 1.6.1 by @dependabot in #1572
• chore(deps-dev): bump pyre-check from 0.9.18 to 0.9.19 by @dependabot in #1574
• chore(deps): bump fastapi from 0.104.0 to 0.104.1 by @dependabot in #1580
• chore(deps-dev): bump pytest from 7.4.2 to 7.4.3 by @dependabot in #1577
• chore(deps-dev): bump black from 23.10.0 to 23.10.1 by @dependabot in #1575
• chore(deps): bump uvloop from 0.18.0 to 0.19.0 by @dependabot in #1576
• chore(deps-dev): bump wheel from 0.41.2 to 0.41.3 by @dependabot in #1579
• chore(deps): bump uvicorn from 0.23.2 to 0.24.0.post1 by @dependabot in #1581
• chore(deps): bump censys from 2.2.8 to 2.2.9 by @dependabot in #1583
• chore(deps-dev): bump black from 23.10.1 to 23.11.0 by @dependabot in #1582
• chore(deps-dev): bump mypy from 1.6.1 to 1.7.0 by @dependabot in #1584
• chore(deps): bump aiohttp from 3.8.6 to 3.9.1 by @dependabot in #1598
• chore(deps-dev): bump isort from 5.12.0 to 5.13.1 by @dependabot in #1602
• chore(deps): bump actions/setup-python from 4 to 5 by @dependabot in #1600
• chore(deps): bump certifi from 2023.7.22 to 2023.11.17 by @dependabot in #1591
• chore(deps): bump ujson from 5.8.0 to 5.9.0 by @dependabot in #1603
• chore(deps-dev): bump pytest-asyncio from 0.21.1 to 0.23.2 by @dependabot in #1599
• chore(deps-dev): bump mypy from 1.7.0 to 1.7.1 by @dependabot in #1596
• chore(deps): bump setuptools from 68.2.2 to 69.0.2 by @dependabot in #1594
• chore(deps-dev): bump wheel from 0.41.3 to 0.42.0 by @dependabot in #1597
• chore(deps-dev): bump pydantic from 2.5.1 to 2.5.2 by @dependabot in #1595
• chore(deps): bump github/codeql-action from 2 to 3 by @dependabot in #1608
• chore(deps-dev): bump types-ujson from 5.8.0.1 to 5.9.0.0 by @dependabot in #1606
• chore(deps-dev): bump black from 23.11.0 to 23.12.0 by @dependabot in #1604
• chore(deps-dev): bump isort from 5.13.1 to 5.13.2 by @dependabot in #1607
• chore(deps): bump fastapi from 0.104.1 to 0.105.0 by @dependabot in #1605
• chore(deps): bump censys from 2.2.9 to 2.2.10 by @dependabot in #1609
• chore(deps-dev): bump pydantic from 2.4.2 to 2.5.1 by @dependabot in #1587

New Contributors

• @LelouBil made their first contribution in #1545
• @branchvincent made their first contribution in #1505
• @dbfreem made their first contribution in #1586

Full Changelog: 4.4.4...4.5.0

4.4.4

What's Changed

• Fix typo so hunterhow api key works
• chore(deps): bump fastapi from 0.101.0 to 0.101.1 by @dependabot in #1517
• chore(deps): bump shodan from 1.29.1 to 1.30.0 by @dependabot in #1518
• chore(deps): bump fastapi from 0.101.1 to 0.102.0 by @dependabot in #1526
• chore(deps-dev): bump pydantic from 2.1.1 to 2.3.0 by @dependabot in #1525
• chore(deps-dev): bump mypy from 1.5.0 to 1.5.1 by @dependabot in #1520
• bump wheel from 0.41.1 to 0.41.2 by @dependabot in #1524
• chore(deps): bump setuptools from 68.0.0 to 68.1.2 by @dependabot in #1523
• Replace an inappropriate test expression that is always evaluating to true by @munahaf in #1529
• chore(deps): bump actions/checkout from 3 to 4 by @dependabot in #1531
• chore(deps): bump censys from 2.2.4 to 2.2.5 by @dependabot in #1530
• chore(deps): bump fastapi from 0.102.0 to 0.103.1 by @dependabot in #1532
• chore(deps-dev): bump pytest from 7.4.0 to 7.4.2 by @dependabot in #1535
• chore(deps): bump setuptools from 68.1.2 to 68.2.0 by @dependabot in #1534
• chore(deps): bump black from 23.7.0 to 23.9.1 by @dependabot in #1537

New Contributors

• @munahaf made their first contribution in #1529

Full Changelog: 4.4.3...4.4.4

4.4.3

Small bugifx release for the screenshot tool

Full Changelog: 4.4.2...4.4.3

4.4.2

What's Changed

• chore(deps): bump dnspython from 2.4.0 to 2.4.1 by @dependabot in #1495
• chore(deps-dev): bump pydantic from 2.0.3 to 2.1.1 by @dependabot in #1496
• Add black and isort formatting to repo by @yoonthegoon in #1494
• chore(deps): bump fastapi from 0.100.0 to 0.100.1 by @dependabot in #1498
• Corrected module capitalization and added fuel URLs by @leebaird in #1499
• chore(deps): bump uvicorn from 0.23.1 to 0.23.2 by @dependabot in #1502
• chore(deps-dev): bump pyflakes from 3.0.1 to 3.1.0 by @dependabot in #1503
• chore(deps-dev): bump flake8 from 6.0.0 to 6.1.0 by @dependabot in #1504
• chore(deps): bump fastapi from 0.100.1 to 0.101.0 by @dependabot in #1506
• Replace import star with explicit imports by @yoonthegoon in #1507
• hot fix host_name type signature by @yoonthegoon in #1509
• chore(deps): bump aiofiles from 23.1.0 to 23.2.1 by @dependabot in #1514
• chore(deps-dev): bump mypy from 1.4.1 to 1.5.0 by @dependabot in #1515
• chore(deps): bump dnspython from 2.4.1 to 2.4.2 by @dependabot in #1513
• chore(deps-dev): bump wheel from 0.41.0 to 0.41.1 by @dependabot in #1511

Full Changelog: 4.4.1...4.4.2

4.4.1

Shout out to @yoonthegoon for fixing the mypy stuff, should help in reducing any bugs that was in there

What's Changed

• chore(deps-dev): bump wheel from 0.40.0 to 0.41.0 by @dependabot in #1490
• chore(deps): bump certifi from 2023.5.7 to 2023.7.22 by @dependabot in #1489
• MyPy CI failures by @yoonthegoon in #1488

New Contributors

• @yoonthegoon made their first contribution in #1488

Full Changelog: 4.4.0...4.4.1

4.4.0

What's Changed

• chore(deps): bump setuptools from 67.7.2 to 67.8.0 by @dependabot in #1429
• Update rocketreach api endpoint by @L1ghtn1ng in #1432
• chore(deps): bump requests from 2.30.0 to 2.31.0 by @dependabot in #1434
• chore(deps-dev): bump types-pyyaml from 6.0.12.9 to 6.0.12.10 by @dependabot in #1433
• chore(deps-dev): bump types-requests from 2.30.0.0 to 2.31.0.1 by @dependabot in #1438
• chore(deps-dev): bump pydantic from 1.10.7 to 1.10.8 by @dependabot in #1436
• Doc updates by @L1ghtn1ng in #1440
• add netlas module by @L1ghtn1ng in #1439
• chore(deps-dev): bump pydantic from 1.10.8 to 1.10.9 by @dependabot in #1444
• chore(deps): bump fastapi from 0.95.2 to 0.96.0 by @dependabot in #1443
• chore(deps): bump fastapi from 0.96.0 to 0.97.0 by @dependabot in #1448
• chore(deps-dev): bump pytest from 7.3.1 to 7.3.2 by @dependabot in #1446
• chore(deps): bump ujson from 5.7.0 to 5.8.0 by @dependabot in #1449
• chore(deps-dev): bump types-ujson from 5.7.0.5 to 5.8.0.0 by @dependabot in #1447
• chore(deps-dev): bump pytest from 7.3.2 to 7.4.0 by @dependabot in #1456
• chore(deps): bump fastapi from 0.97.0 to 0.99.0 by @dependabot in #1459
• chore(deps-dev): bump pydantic from 1.10.9 to 2.0 by @dependabot in #1458
• Fix a bug about string splicing by @captain686 in #1460
• chore(deps): bump fastapi from 0.99.0 to 0.99.1 by @dependabot in #1461
• Added two new sources, updated to use ujson instead of json, Optimized Brave and Sitedossier Search and other misc changes by @NotoriousRebel in #1463
• chore(deps-dev): bump mypy from 1.3.0 to 1.4.1 by @dependabot in #1457
• chore(deps): bump setuptools from 67.8.0 to 68.0.0 by @dependabot in #1452
• chore(deps): bump censys from 2.2.2 to 2.2.3 by @dependabot in #1450
• chore(deps): bump lxml from 4.9.2 to 4.9.3 by @dependabot in #1465
• chore(deps-dev): bump pydantic from 2.0 to 2.0.2 by @dependabot in #1464
• chore(deps): bump censys from 2.2.3 to 2.2.4 by @dependabot in #1466
• chore(deps): bump fastapi from 0.99.1 to 0.100.0 by @dependabot in #1467
• add missing hunterhow api key entry by @L1ghtn1ng in #1469
• chore(deps-dev): bump pytest-asyncio from 0.21.0 to 0.21.1 by @dependabot in #1471
• Updated takeover checks to be more thorough and show in output file and fixed bug when using resolvers and output file by @NotoriousRebel in #1470
• chore(deps-dev): bump pydantic from 2.0.2 to 2.0.3 by @dependabot in #1472
• chore(deps): bump uvicorn from 0.22.0 to 0.23.1 by @dependabot in #1475
• chore(deps): bump dnspython from 2.3.0 to 2.4.0 by @dependabot in #1474
• chore(deps): bump pyyaml from 6.0 to 6.0.1 by @dependabot in #1476
• chore(deps): bump aiohttp from 3.8.4 to 3.8.5 by @dependabot in #1478
• Add tomba module by @benemohamed in #1479
• chore(deps-dev): bump types-requests from 2.31.0.1 to 2.31.0.2 by @dependabot in #1483
• chore(deps-dev): bump types-ujson from 5.8.0.0 to 5.8.0.1 by @dependabot in #1482
• chore(deps-dev): bump types-python-dateutil from 2.8.19.13 to 2.8.19.14 by @dependabot in #1481
• chore(deps-dev): bump types-pyyaml from 6.0.12.10 to 6.0.12.11 by @dependabot in #1480
• flake8 fixes by @L1ghtn1ng in #1484
• fix security issue with base image by @L1ghtn1ng in #1485

New Contributors

• @captain686 made their first contribution in #1460
• @benemohamed made their first contribution in #1479

Full Changelog: 4.3.0...4.4.0

4.3.0

https://cybermon.uk/posts/theharvester4.3.0/

4.2.0

Removed

• All modules that use Google as there source to get the data due to Google blocking with capcha's has been removed
• Spyse module has been removed due to the Spyse team not able to maintain the company due to them being based in Ukraine and current situation happening over there

What's Changed

• Fix bug from stopping theHarvester.py from running and update the bin… by @L1ghtn1ng in #938
• Fix #931 by @L1ghtn1ng in #939
• Added ~/ expansion by @D3vil0per in #957
• use context by @cesarhub in #974
• fix: fix issue with twitter source search and proxy variable not updat… by @brokoder in #977
• point to my fork to fix install issues with spyse dep by @L1ghtn1ng in #1011
• Update starlette version due to dependency conflict by @corysolovewicz in #1096
• Binaryedge Placeholder Value Fix in README.md by @fridy1307 in #1120
• A lot of changes and fixes by @L1ghtn1ng in #1168
• Updated README and misc. by @leebaird in #1171
• Fixed VirusTotal module by @NotoriousRebel in #1172
• BeVigil Source Added by @alt-glitch in #1180
• Flake8 fixes by @L1ghtn1ng in #1181
• A bunch of dependency updates

New Contributors(Thank You)

• @D3vil0per made their first contribution in #957
• @cesarhub made their first contribution in #974
• @brokoder made their first contribution in #977
• @fridy1307 made their first contribution in #1120
• @alt-glitch made their first contribution in #1180

Full Changelog: 4.0.3...4.2.0