[repost]Heroku versus GAE & GAE/J


8 October 2009

At the Geek Night last night there was a lot of questions about how Heroku compared with Google App Engine (and I guess implicit in that how it compares to GAE/J with the JRuby gem). Since it came up several times I thought it would be interesting to capture some of the answers.

Firstly there are the similarities, both deal with the application as the base unit of cloud scaling. Heroku offers several services that are similar to GAE, e.g. mailing, scheduled and background jobs. Some aspects like Memcache are in beta with Heroku but fundamentally both services seem to concur.

They are both free to start developing and then the pricing for the system varies a bit between the two, GAE might be cheaper but also has a more complex set of variables that go into the pricing, Heroku prices on: data size, web processes required and then options, so it’s easier to project your costs.

Differences then: well a big one is that Heroku uses a standard SQL database and also has excellent import/export facilities. GAE uses Google BigTable which is great for scaling and availability but data export and interactions are very limited. Heroku has an amazing deployment mechanism that involves simply pushing to a remote branch. GAE is slightly more involved but it is still relatively easy compared with deploying your own applications.

GAE provides access to a variety of Google services that are handy such as Google Account integration and image manipulation. GAE also has XMPP integration which is pretty unique if you have the requirement. However these do tie you to the platform and if you wanted to migrate you’d have to avoid using the Google webapp framework and the API. If flexibility is an issue you are probably going to prefer Heroku’s use of standard Ruby libraries and frameworks.

Heroku has few restrictions on what can and can’t be done in a web process, you also have quite powerful access to the user space your application runs in. GAE and particularly GAE/J has a lot of restrictions which can be quite painful particularly as there can be a discrepency between development and deployment environments.

However running a JVM in the cloud provides a lot of flexibility in terms of languages that can be used and deployed. Heroku is focussing on Ruby for now.

From my point of view, choose Heroku if you need: uncomplicated Rails/Rack application deployment, a conventional datastore that you want to have good access to, a simple pricing model. Choose GAE if you want to have out of the box access to Google’s services and a complete scaling solution.

[repost]New Facebook Chat Feature Scales to 70 Million Users Using Erlang


New Facebook Chat Feature Scales to 70 Million Users Using Erlang

UpdateErlang at Facebook by Eugene Letuchy. How Facebook uses Erlang to implement Chat, AIM Presence, and Chat Jabber support.

I’ve done some XMPP development so when I read Facebook was making a Jabber chat client I was really curious how they would make it work. While core XMPP is straightforward, a number of protocol extensions like discovery, forms, chat states, pubsub, multi user chat, and privacy lists really up the implementation complexity. Some real engineering challenges were involved to make this puppy scale and perform. It’s not clear what extensions they’ve implemented, but a blog entry by Facebook’s Eugene Letuchy hits some of the architectural challenges they faced and how they overcame them.

A web based Jabber client poses a few problems because XMPP, like most IM protocols, is an asynchronous event driven system that pretty much assumes you have a full time open connection. After logging in the server sends a client roster information and presence information. Your client has to be present to receive the information. If your client wants to discover the capabilities of another client then a request is sent over the wire and some time later the response comes back. An ID is used to map the reply to the request. All responses are intermingled. IM messages can come in at any time. Subscription requests can come in at any time.

Facebook has the client open a persistent connection to the IM server and uses long polling to send requests and continually get data from the server. Long polling is a mixture of client pull and server push. It works by having the client make a request to the server. The client connection blocks until the server has data to return. When it does data is returned, the client processes it, and then is in position to make another request of the server and get any more data that has queued up in the mean time. Obviously there are all sorts of latency, overhead, and resource issues with this approach. The previous link discusses them in more detail and for performance information take a look at Performance Testing of Data Delivery Techniques for AJAX Applications by Engin Bozdag, Ali Mesbah and Arie van Deursen.

From a client perspective I think this approach is workable, but obviously not ideal. Your client’s IMs, presence changes, subscription requests, and chat states etc are all blocked on the polling loop, which wouldn’t have a predictable latency. Predictable latency can be as important as raw performance.

The real scaling challenge is on the server side. With 70 million people how do you keep all those persistent connections open? Well, when you read another $100 million was invested in Facebook for hardware you know why. That’s one hella lot of connections. And consider all the data those IM servers must store up in between polling intervals. Looking at the memory consumption for their servers would be like watching someone breath. Breath in- streams of data come in and must be stored waiting for the polling loop. Breath out- the polling loops hit and all the data is written to the client and released from the server. A ceaseless cycle. In a stream based system data comes in and is pushed immediately out the connection. Only socket queue is used and that’s usually quite sufficient. Now add network bandwidth for all the XMPP and TCP protocol overhead and CPU to process it all and you are talking some serious scalability issues.

So, how do you handle all those concurrent connections? They chose Erlang. When you first hear Erlang and Jabber you think ejabberd, an open source Erlang based XMPP server. But since the blog doesn’t mention ejabberd it seems they haven’t used it .

Why Erlang? First, the famous Yaws vs Apache shootout where “Apache dies at about 4,000 parallel sessions. Yaws is still functioning at over 80,000 parallel connections.” Erlang is naturally good at solving high concurrency problems. Yet following the rule that no benchmark can go unchallenged, Erik Onnen calls this the Worst Measurement Ever and has some good reasoning behind it.

In any case, Erlang does nicely match the problem space. Erlang’s approach to a concurrency problem is to throw a very light weight Erlang process at each state machine you want to be concurrent. Code-wise that’s more natural than thread pools, async IO, or thread per connection systems. Until Linux 2.6 it wasn’t even possible to schedule large numbers of threads on a single machine. And you are still devoting a lot of unnecessary stack space to each thread. Erlang will make excellent use of machine resources to handle all those connections. Something anyone with a VPS knows is hard to do with Apache. Apache sucks up memory with joyous VPS killing abandon.

The blog says C++ is used to log IM messages. Erlang is famously excellent for its concurrency prowess and equally famous for being poor at IO, so I imagine C++ was needed for efficiency.

One of the downsides of multi-language development is reusing code across languages. Facebook created Thrift to tie together the Babeling Tower of all their different implementation languages. Thrift is a software framework for scalable cross-language services development. It combines a powerful software stack with a code generation engine to build services that work efficiently and seamlessly between C++, Java, Python, PHP, and Ruby. Another approach might be to cross language barriers using REST based services.

A problem Facebook probably doesn’t have to worry about scaling is the XMPP roster (contact list). Handling that many user accounts would challenge most XMPP server vendors, but Facebook has that part already solved. They could concentrate on scaling the protocol across a bunch of shiny new servers without getting bogged down in database issues. Wouldn’t that be nice :-) They can just load balance users across servers and scalability is solved horizontally, simply by adding more servers. Nice work.