This project is a client-server file management system utilizing a custom-implemented Merkle tree to ensure the integrity of files during upload and retrieval. The system allows secure uploading and verification of files using Merkle root hashes. It supports multiple upload batches with each batch treated as a distinct Merkle tree, ensuring robust file integrity checks.
- NestJs: Backend framework
- ExpressJs: Backend routing
- ReactJs: Frontend framework
- Postgres with Prisma: Database management (development was performed using Neon Serverless Postgres)
- Docker/Docker-compose: Used for deployment and service orchestration
The system architecture includes a client and a server with both frontend and backend components. The backend handles file uploads, constructing Merkle trees, and generating unique identifiers for each file batch. The server stores the files and updates tracking information in its database. The client frontend is designed to display and retrieve files directly from the server.
- REST API Endpoints: POST /files, GET /files, and GET /fileId.
- Upload Module: Manages file uploads into the server's 'upload' directory.
- Database Module: Interacts with the database for file tracking.
- Helper Files Module: Provides simplified APIs for database operations.
- Manages REST API endpoints POST /files and GET /fileId.
- Handles file processing, including Merkle root computation for each upload batch and storing details in
merkleRoot.txt
. - Verifies proofs of file integrity received from the server.
- Provides a basic UI for visualizing the application flow and interaction.
- Code Clarity and Type Safety: Improvements in code readability and error reduction.
- Error Handling: Systematic error management and logging.
- Data Transfer Objects (DTOs): Refinement in API interactions for robustness.
- Compression: Introduction of data compression for better storage efficiency.
- API Enhancements: Improvement of the API for enhanced performance.
Use docker-compose up --build
to start the services.
Initialize the database with:
docker exec -it server npx prisma migrate dev --name init
Check the database table creation:
docker exec -it postgres-db psql -U postgres
One inside the container run:
\dt
The application was stress-tested by uploading 1,000 small files simultaneously and performing multiple multi-file uploads and verifications. The system effectively managed high volumes of data and multiple upload batches.
Feel free to contribute, Use & improve the project.