Recently all I talk about is mesos and mesos on mesos ;)
Apache Mesos abstracts CPU, memory, storage, and other compute
resources away from machines (physical or virtual), enabling
fault-tolerant and elastic distributed systems to easily be built and
Sounds fantastic and because I’m true fan of docker I went with
which is a cluster-wide init and control system for
services in cgroups or Docker containers
In combination wiht some effort mesos can replace existing
infrastructure making it more:
easier to access
After some quick test on my computer creating mesos cluster with
docker-compose i gave it a big go and installed it
First use case!
TouK is developing an application ctrl-pkw to collect protocols from poll stations and check if election results were not tampered.
So beside my plan to move most of the virtual machines from KVM to
mesos I got a task to create cassandra cluster which will be easily scalable. Let’s get to it!
Cassandra on docker
No magic here, run docker run -it 3h4x/cassandra to start playing with
It works but we want more!
Cassandra use gossip protocol to communicate. Each node need to connect to seed node and from there it will get all nodes of cassandra in cluster eventually.
So I have configured two cassandra type of nodes, seed - just few of them
to bootstrap cluster and normal node which I can scale up or scale down.
To get what we need I used service discovery via mesos-dns so cassandra nodes can see cassandra seed never mind on which mesos slave they will be. To do it I just used simple bash script
This will extract ip’s of cassandra seed from mesos-dns.
Most important stuff in this cluster config is constraint on mesos to use unique hosts, otherwise it will try to deploy on hosts where ports are already occupied and it will fail.
Cassandra needs ports open for it’s own gossip configuration and they must be avaliable on mesos-slave ip. There is no chance right now to reconfigure cassandra.yaml in a way to use different ports for internode communication.
This is why additional configuration of docker is needed
It allows to bind ports on mesos-slave instead of ip that was given by docker.
Last but not least there is SEED environment which will be given to docker container
If your rename the cluster be sure to change this dns address too, moreover every mesos slave needs to have mesos-dns configured.
This is the moment that I’ve been waiting on. We can modify cassandra app configuration on marathon and add additional nodes using scale or instances parameter
I have tried to use parameter “scaleBy” which is in marathon API. Well, better luck next time.
Full cluster with application
We are just one step away from deploying our application with cassandra database on mesos using docker containers! Awesome :D
And here is json to deploy cluster with cassandra and app on marathon