Google App Engine

Tuesday, October 28th, 2008

After Amazon, Google App Engine is the next big contender for mainstream cloud computing. The power of Google’s infrastructure will spin up as many instances of your app as needed, and they’ve provided a simple Python framework with Django templates for development, with additional languages to come. Basic accounts will always be free, and after the current preview ends, a la carte pricing wil be similar to Amazon’s services.


Listen First

Monday, October 27th, 2008

Peacefully waitingI consider myself a good listener, but really missed the mark last week. One of the cloud computing talks was being given by Joe Gregorio from Google, who I chatted with beforehand. He mentioned working on the Atom protocol, which I immediately took as an opportunity to ramble off about using it as the API for a previous project.

I looked him up afterward, and found out that he actually wrote the spec! For as many times as I read the thing, you’d think I would’ve noticed the name, or at least picked up what he was telling me before launching into my own story. Of course, being a good listener himself, he indulged me before getting into his talk on Google’s App Engine.

Facebook’s Hadoop and Hive Data Mining

Friday, October 24th, 2008

HitchcockThe second cloud computing track was on massive data processing in clusters and 15clouds, including a presentation by Facebook on their use of Hadoop and custom development of Hive to facilitate their own operations.

Data mining the 180 TB of Facebook data, which grows at 2 TB a day is no small task, so the team uses a cluster of 350 8-core machines to crunch data and figure out which popular features deserve further investment, demographics, and whatever else they can fish out of the sea of personal information users provide.


4 Cool Google Spreadsheet Tricks

Thursday, October 23rd, 2008

Rising aboveAnother cloud computing talk introduced Google Docs and showed a few cool ways to collaboratively use spreadsheets “in the cloud”:

  1. Data management - Google Checkout pulls merchant information live from spreadsheets, allowing non-technical folks to do updates without needing their own administration screen.
  2. Form entry - Generate a form from a spreadsheet and embed it in an email for higher survey response rates and easy tallying of results. Bonus: as long as you don’t change the POST URL or the names of the input fields, you can customize the format the form or split it into multiple pages.
  3. Map your data - Many “gadgets” are available to act on spreadsheet data, including displaying it in Google Maps with another column as the tooltip.
  4. MagicFill - Use the power of Google Sets to generate related words.

Cloud Computing at Amazon

Wednesday, October 22nd, 2008

Blending into the skyI attended a few talks on cloud computing last week, including an overview of Amazon Web Services. My previous view of Amazon’s mission was that they’re out to commoditize everything, from books to products to computing. In fact, they actually do have 3 lines of business:

The extent of their web services is testimony to it’s standing as a main pier of the company: it includes a variety of inexpensive a la carte services for data storage, messaging, and raw computing power. Web service traffic actually surpasses that of the retail site. Of course, the real proof comes in some of their success stories:

  • SmugMug stores and serves 700 TB of data through Amazon (case study)
  • The New York Times TimesMachine split and completed optical character recognition on 130 years work of newspaper scans in less than 24 hours for a few hundred dollars
  • Animoto launched their viral video application on Facebook, and quickly scaled from 50 virtual servers in Amazon’s cloud to 3500


Cloud Computing vs Platforms

Tuesday, October 21st, 2008

I’m going to be recapping some recent talks I participated in on cloud computing, so by way of introduction, here’s a post from Marc Andreesen detailing the three kinds of platforms you meet on the Internet (cloud computing being Level 3):

A Level 1 platform’s apps run elsewhere, and call into the platform via a web services API to draw on data and services — this is how Flickr does it.

A Level 2 platform’s apps run elsewhere, but inject functionality into the platform via a plug-in API — this is how Facebook does it. Most likely, a Level 2 platform’s apps also call into the platform via a web services API to draw on data and services.

A Level 3 platform’s apps run inside the platform itself — the platform provides the “runtime environment” within which the app’s code runs.