Skip to content

kisscool/tiny-garbage

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

== Tiny-Garbage

Note : You should check the successors of Tiny-Garbage : Tiny-Garbage2 for MongoDB (http://github.com/kisscool/tiny-garbage2) and Tiny-Garbage3 for Redis (http://github.com/kisscool/tiny-garbage3).

Just a little FTP crawler, with a little Sinatra based search interface in order to browse the FTP index.
Nothing else, nothing more.

== Dependencies

Required gems are :
- dm-core
- dm-migrations
- dm-is-tree
- dm-aggregates
- dm-sqlite-adapter (or dm-mysql-adapter, depending on your need)
- sinatra
- haml
- sass
- net-ftp-list
- Rack

If you have Bundler installed on your system, you can track down all those dependencies by launching the following command from inside the project directory :
$ bundle install --path vendor

If you want only MySQL dependencies, you can use :
* bundle install --path vendor --without sqlite
(idem if you do want only sqlite dependencies)

== Install

= The crawler part

Do a clone of the project git repository, install the missing dependencies then create the configuration file by copying config.rb.sample as config.rb and edit it to suit your taste.

Configure your crontab to launch periodically the following commands :
* "$path_to_project/scripts/crawler.rb ping" (with a recommenced frequency of every 10 minutes)
* "$path_to_project/scripts/crawler.rb index" (with a recommended frequency of once a day)


= The Web UI part
For a first try you can just launch "rackup config.ru" in order to test the web UI with the pure ruby Webrick server.
If you want to deploy it in production, you will want to check Unicorn, Thin or Phusion Passenger documentations for more solids options.

== Thanks

Thanks to Loic Gomez for the first Garbage Collector, which was so useful back in the days, and thanks to Zouchaoqun's ezFtpSearch (http://github.com/zouchaoqun/ezftpsearch) whose model bootstrapped our work.

About

A tiny FTP crawler and indexer with a web UI

Resources

License

Stars

Watchers

Forks

Packages

No packages published