Thread

hey everyone,

ive been looking into Amazon EC2 more and more lately as my team and i are getting ready to launch our application. we will be dealing with alot (many many many gigs) of legislative data (bills, rollcalls, sponsors, etc) for legislative bodies across the US. i know that its pretty simple to set up a DB server on EC2 (havnt tried it yet though), and they are pretty easy to scale across multiple instances.

however i was wondering how you make sure if an instance goes down that your not losing that data? ive read several posts saying if an instance crashes you loose all data associated with it. how do you combat this? i am already backingup the db’s every night with S3 and a ruby script but if it goes down thats a pretty long process to load all that data back into an instance…

also, is it practical to run just the db server off of EC2 or does it do more harm than good? what i mean is does it do more harm having your app server hosted say at the planet and your db servers at amazon?

another thing we will be implementing is multiple memcached servers. again more harm than good?

As you can tell im not a sys admin by far. Neither is anyone on my team (we will be hiring someone for this in the future) but until we get off the ground its just us 4.

Thanks,

Tom

trs21219,

I’ve no experience with EC2 specifically, but have dealt with making a couple of MySQL databases tolerant of a single-point server failure. Are you using MySQL? If so, have a look at Master/Slave replication configurations. Other DBMS have the same type of replication support.

A good reference: http://oreilly.com/catalog/9780596003067/

The MySQL manual: http://dev.mysql.com/doc/refman/5.0/en/replication.html

-Jeff

yes we are using mysql. im pretty confident i can set up the servers its just the reliability. now if the master goes down do the remaining slaves then take the role of the master?

so if server 1 goes down and has two slaves (2 and 3) does 2 take the place as the master to slave 3?

I’m no sys admin either, but I was looking into databases on EC2 as well.

You might want to look into Amazon Elastic Block Storage (EBS). You can host the mysql daemon on EC2, but all the files and logs can be persisted on EBS. Here’s an article.

Also, using replication, you can have a slave do the backups to S3 so you don’t have to lock down the master. I didn’t find any automatic recovery solutions if the master goes down. It seems you’d have to be notified and then manually make a slave the master and switch any other slaves to the new master.

I’m interested to know what you decide and how it works out.

thats why im skeptical. because if myself or someone who is not fluent in ssh is not available if the database master goes down then the site is down for a while. i know that they went down frequently before but does anyone know the current situation? is it pretty stable? is it as reliable as any other virtual server?

we are very interested in this because of the options to scale automatically only when we need it then it will scale down when the rush is over (with a little help from scalr), a perfect way to handle the digg effect.

There are automatic failover methods. A good overview can be found here: http://dev.mysql.com/tech-resources/articles/failover-strategy-part1.html
-Jeff

muttlogic, thanks for the excellent link.

ya thanks guys. you were a big help

I was starting a hosting companie up with the plan mainly to create scaleable and HA applications using EC2, i would have used CI for the controll pannel. The reason i did not pursue this any further was the slight lack in interest, but mainly i have moved more down teh stage lighting and sound route, thas another reason why i haven’t been as acctive on here.