fidb

fidb is a Python tool that enables you to maintain a file inclusion path database. You grow your database by harvesting paths from different systems (e.g. in 24/7 CTFs like HackTheBox or while penetration testing). Then, you query for the paths you need right now (e.g. Windows only, file paths, paths containing "password" or common paths found in multiple systems). You may also specify the current directory of your file inclusion to make all queried paths relative.

Getting Started

Installing fidb

# Clone the github repository
git clone https://github.com/kijube/fidb.git

# Install requirements
cd fidb
pip install -r requirements.txt

Harvesting paths

First, you need to upload harvest.py or harvest.sh to your target system. Then, simply run it:

python3 harvest.py
# or
./harvest.sh / paths.txt

Download the resulting zip (default: paths.txt.zip) to your local machine and add the paths to the database:

python3 fidb.py unzip paths.txt.zip

Querying for paths

After you have collected paths from at least one system, you can query for them when needed:

# linux paths that contain "password" that were found in at least three systems
python3 fidb.py query -t lin -sp "password" -mo 3

You can find more examples below.

Usage

usage: fidb.py [-h] [--db DB] {unzip,read,query} ...

positional arguments:
  {unzip,read,query}
    unzip             Read paths from zip
    read              Read paths from stdin
    query             Query paths

options:
  -h, --help          show this help message and exit
  --db DB             Database connection string (default: 'sqlite:///db.sqlite')

To get further information on the unzip, read and query modules, simply use the python3 fidb.py <cmd> -h syntax.

Examples

Adding paths to the database from stdin using read command

cat paths.txt | python3 fidb.py read --db sqlite:///mydb.sqlite

This will add all the paths in the paths.txt file to the database specified by --db argument (note that the db argument is optional and fidb uses a sqlite database by default).

Querying for linux paths containing a specific string

python3 fidb.py query --type lin --search-plain "password"

This will return all linux paths in the database that contain the string "password".

Querying for windows paths with a minimum number of occurrences

python3 fidb.py query --type win --min-occurences 5

This will return all windows paths in the database that have occurred at least 5 times.

Querying for linux paths matching a regular expression

python3 fidb.py query --type lin --search-regex ".*\.php"

This will return all linux paths in the database that match the regular expression ".*.php".

Querying for windows directories only:

python3 fidb.py query --type win --only dirs

This will return only the windows directories in the database.

Querying for linux paths as relative to a specified directory:

python3 fidb.py query --type lin --relative-to /home/user/project

This will return all linux paths in the database as relative to the directory "/home/user/project".

Harvesting paths from a system

python3 harvest.py

This will add all paths in the system to the harvest.txt.zip file.

Using unzip command to read paths from a zip file

python3 fidb.py unzip paths.txt.zip

This will read the first file in the zip file "paths.txt.zip" and add its contents (which should be a list of paths, one per line) to the database. Note that paths.txt.zip is the default name of the zip file generated by harvest.py