Batteries-included cluster computing for Go
Explore the docs »
This project is not yet production-ready. Breaking changes could be made.
Beekeeper is a batteries-included cluster computing solution written in Go. Make a homemade cluster with your old computers or an enterprise-grade deployment quickly and painless.
The idea behind Beekeeper is simple: Take a bunch of computers and make them work together for a faster and more reliable system (yay, teamwork!).
Beekeeper is split into two installs. The library is used to create new jobs and distribute them among the nodes while the CLI tools will help you get your nodes up and running.
To install the Go library run the following command in the command line:
go get github.com/CamiloHernandez/beekeeper/lib The command-line tool can be installed using go get as follows:
go get github.com/CamiloHernandez/beekeeper/bee Drop by our Quickstart page for an in-depth onboarding!
To run a node typebee start into the command prompt. The node will now listen for incoming tasks and execute them as they come in.
To create a new Job we need to define a function implementing the func (*beekeeper.Task) prototype. The Task struct will hold the information needed to run the Job, such as additional data and returns.
As an example we'll cluster a job that finds a bunch of primes numbers and returns them to the primary node:
func RandomPrime(t *beekeeper.Task) {
var primes []int64
for len(primes) < 10000 { n := rand.Int63n(10000000000000000)
if isPrime(n) {
primes = append(primes, n)
}
}
t.Returns["primes"] = primes
}To run a task we first need a server to handle it. A server can be created with the NewServer function, and it can then be started with the Start method. This is blocking, so we'll run it inside a goroutine.
go func() {
sv := beekeeper.NewServer()
err := sv.Start()
if err != nil{
panic(err)
}
}()
defer sv.Stop()Now we need a list of the available nodes in our network. To do this we use Scan. Optionally we can specify an address to connect to using the Connect method.
nodes, err := beekeeper.Scan(beekeeper.DefaultScanTime)
if err != nil{
panic(err)
} nodes will be a slice containing the workers available in the local network. Before we can run the task we need to distribute it among the nodes.
err = sv.DistributeJob("github.com/user/myFirstCluster", "RandomPrime", nodes...)
if err != nil{
panic(err)
} All is left to do is create a new Task and send it to the workers. Since we are not going to send any arguments we'll use an empty Task. We can call Execute over every node, but for convenience, we have the ExecuteMany method at hand.
task := beekeeper.NewTask()
results, err := sv.ExecuteMany(nodes, task)
if err != nil{
panic(err)
} The execution will now block until all nodes have finished their task. Afterwards we iterate over the results and print the.
var primes []int64 for _, result := range results{
fmt.Println(result.Task.Returns["primes"].[]int64)
} You can check out the official documentation to learn more about the advanced features of Beekeeper.
Contributions are always welcome! If you want to help Beekeeper please create a fork of this repository, make your changes to your fork and do a Pull Request. Keep in mind that suggestions must have a strong case for their addition, and must keep to the structure and quality of the code.
Beekeeper is distributed under the MIT License as free and open-source. See the LICENSE file for more information.