Life is grand

Amazon.com started 10 years ago as a monolithic application, running on a Web server, talking to a database on the back end. This application, dubbed Obidos, evolved to hold all the business logic, all the display logic, and all the functionality that Amazon eventually became famous for…

and concluded that a service-oriented architecture would give us the level of isolation that would allow us to build many software components rapidly and independently… …For us service orientation means encapsulating the data with the business logic that operates on the data, with the only access through a published service interface. No direct database access is allowed from outside the service, and there’s no data sharing among the services.

Over time, this grew into hundreds of services and a number of application servers that aggregate the information from the services. The application that renders the Amazon.com Web pages is one such application server, but so are the applications that serve the Web-services interface, the customer service application, the seller interface, and the many third-party Web sites that run on our platform.

If you hit the Amazon.com gateway page, the application calls more than 100 services to collect data and construct the page for you.

So SOA is not just an external face but an internal structure.

Giving developers operational responsibilities has greatly enhanced the quality of the services, both from a customer and a technology point of view. The traditional model is that you take your software to the wall that separates development and operations, and throw it over and then forget about it. Not at Amazon. You build it, you run it.

Great to read about this in such a big company. We have seen the success of “build it, run it” in smaller companies (Flickr, 37Signals, YouTube, MySpace etc.) but not in bigger ones where hand-over to operations is too often seen as a positive organisation choice.

If we look at how developers use these interfaces, in general the REST version is used by small libraries in Perl or PHP as part of a LAMP stack, and the SOAP calls are mainly done by applications that have been built on Java or .NET platforms by consuming our WSDL files and generating proxy objects.

Do we see that customers who develop applications using AWS care about REST or SOAP? Absolutely not! …developers really just want to build their applications using the easiest toolkit they can find. They are not interested in what goes on the wire or how request URLs get constructed…

Interesting find that. We have been discussing it on the FeedHenry project, to go SOAP or REST, but here we see it hardly matters.

there are many tools available, and we try not to impose too many constraints on our engineers. We provide incentives for some things, such as integration with the monitoring system and other infrastructure tools. But for the rest, we allow teams to function as independently as possible.

So there isn’t a monoculture in Amazon. Developers use the tools that suit the task and their skills.

One of the biggest challenges is developing at this large scale. How do you make sure that developers are productive in this large distributed service-oriented architecture? …Testing in a very large-scale distributed setting is a major challenge.

Take heart everyone, the big boys struggle with the same daily challenges.

If an idea is deemed worthy of investigation, we exploit our service development approach to scope and prototype the idea quickly. With a new radical service, you try to go into prototype mode pretty quickly, and then you start iterating on that until you feel that you understand your business problem.

Agile.

The stored data formats [for catalogues] are decoupled from the format in which you communicate data items. If there is no need for sharing schemas of the actual storage layout, you can focus on making sure that the service interfaces can evolve in a way that allows you to handle variations of data formats.

A Conversation with Werner Vogels (CTO at Amazon) is a recommended read.

This entry was posted on Thursday, May 25th, 2006 at 9:57 am and is filed under Uncategorized. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.