dcinput

Playing with Amazon Web Services

Spent several hours this evening reading tutorials about and playing around with some of the Amazon web services: S3, EC2 and SQS. I found a great tutorial by Mitch Garnaat explaining how to use all three to build a massively scalable solution to video conversion. At very short notice you can deploy hundreds maybe even thousands of machines to number crunch, pay for what you use, then give them all back. No maintenance headache, no need for massive up front investment.

Mitch has written set of Python libraries called Boto that wrap up some of he Amazon functionality. The tutorials have some good examples and seem pretty straight forward to use. I came across some authentication issues which I hope to resolve in the next few days.

In around a year or two mobile devices that can take great video will be standard. What are all these millions of people going to want to do with their video? That’s the million dollar question.

The Beat Box Fame Game. Superb video.

Marshall Kirkpatrick on the Four Eyed Monsters: “The promotional model is one that many future independent film makers will likely study closely”.

An interesting IT Conversations podcast with Doug Kaye and Jeff Bezos about how Amazon Web Services can be used to create a massively scalable solution for digital audio and video manipulation. Outsource your infrastucture.

Amazon offers it’s own queueing service but I’m wondering whether it would be possible to setup a render farm on EC2 and S3 controled by a specialist render queue manager such as Rush. A pay as you go render farm, yes please.

Update: Greg Ercolano from Rush emailed me to suggest RenderRocket which is a virtual render farm. Very interesting, I’ll have to check it out.

Dave Winer: “There once was a time when we entertained each other. If you wanted music on a Saturday night you’d have to perform it yourself or listen to a neighbor. Before there was broadcast radio, music was personal. It will be personal again, and it wouldn’t be such a bad thing, because the joy of creating is something we should all share.”

Mashable: “As of tomorrow, you can register your own .tv domain on ChannelMe, but soon they’ll add tools for users to develop that site into a personal TV channel”.

Copyright issues aside, media mashups are just funny: The Big Lebowski - the F#$%@*G Short Version [Language not work safe].

TechCrunch: “AppStore will be the one-stop shopping place for on-demand software, starting in Q1 of 2007. Developers and partners will be able to sell their programs directly through the AppStore and Salesforce will make a referral fee, based on the performance of the product”. Ooh an iTunes for web services. Nice.

Jason Calacanis links to a report looking at a new breed of people turning up in organisations: the out there people. Jason adds some of his thoughts to the discussion. Interesting reading and good food for thought.

I’ve just listening to Richard Bacon on XFM. He’s asking his listeners to upload interesing video they make to YouTube and put the word ‘BacoFilm‘ somewhere in the title so that with a simple search they will all show up. At some stage in the future he will choose the best one who will win some award. Interesting to see MSM using these new technologies in their programming. If you’re a videoblogger might be a good way to get your name out to the wider population.

Steve Rubel: “The rapid pace of change will not only turn TV into an open content platform, but it will radically shift how advertising dollars are allocated and how the entire ad industry operates.”

Susan Buice: “I just want to make something that resonates because when I see something that resonates with me I feel connected. Feeling connected to the world and to other human experiences is what makes me feel alive and happy. When I don’t feel connected I feel like a meaningless blob of flesh and I want to die.”

I picked this quote from Susan Buice because it made my laugh. It’s part of an interview she did with Arin Crumbley with whom she runs Four Eyed Monsters. Go read the interview and pay attention to these guys because they are doing interesting things. Arin is responsible for the Open Source Documentary about Net Neutrality that I wrote about a while back. That piece of work and a few other things that were happening to me at the time all converged together helping me to form new ideas about film making, documentary making and more generally how the web is turning things on their head. Ultimately it lead to my mini-essay on the disintermediation of film making. Now while I think and talk about these things Arin and Susan are out there actually doing cool stuff. I think you should pay them some attention and others do to.

Andrew Baron: “I am very optimistic for TiVo’s future as far as his vision for seamlessly merging i.p. and broadcast content”.

The ProgrammableWeb is a site that keps track of all the web APIs that are sprouting up all over the place. There’s a blog where they review them as they appear, a mashup section for the best use of them, a mashup matrix, a place to share stuff and a place to learn about how the web is becoming a platform. Really Great Resource.

David Tames: “The qualitative difference in today’s media technology landscape is that innovation is becoming the domain of end-users and is being guided by human needs, creative expression, social activities, and intellectual pursuits; rather than sales goals, quarterly profits, corporate research agendas, and marketing initiatives.”

David Tames’ essay from which the above quote is taken is a good read. He touches on many of the most interestring topics of the day such as walled-gardens, the changing role of the audience, the web’s effect on community and the theory of the long tail all from the perspective of a film maker.

What I’m finding most interesting in all my readings and podcast listening recently is that people of all sorts of different backgrounds and proffesions from all over the world are starting to form ideas and views about the web that are very similar. How interesting the world becomes when people start to have conversations, and to think that only 6 years ago I thought that the web was a glorified sales catalogue.

The 1K project II is a short generated entirely from digitally combining over 1000 car races in the driving simulator Trackmania. Sounds majorly geeky but it’s actually pretty cool.

Techcrunch reports that Amazon may have inadvertantly let slip that they are releasing a new web service called SDS - Simple Data Service…or maybe not.

One of the most frustrating things for that I find with itunes and the ipod is the whole way the syncing process works. The whole idea of syncing is appealing to me since like most people on the planet I’m a pretty busy person. The ability to be able to wake up in the morning plug the ipod into the laptop for a minute while your latest podcasts load up and then skip off to work listening to your favorite mind snack is a killer feature.

Unfortunately the apple implementation of this feature is pretty lousy. Out of all the settings, what I want is to only have on my ipod at any one time all the podcasts that I haven’t listened to yet. Now iTunes does have such a feature but according to iTunes listening to 1 second of a podcast equates to listening to the whole thing. Setting out on a journey expecting to continue listening to a podcast that you have partially listened to, only to find that it has been marked incorrectly as played and therefor has been removed from the player can do only one thing: it makes me mad.

Anyone thinking of making a podcast device needs to fix this. Please.

Something I realised today about the disintermediation caused by the web: if your profession has the word intermediate in it, start getting worried. For some reason this wasn’t obvious to me yesterday.

Read/WriteWeb: Amazon Web Services Success Stories.

Scoble wants to teach Mark Cuban how to hook up his PC to his HDTV. People are hungry for HD and there just aren’t enough people offering it yet. I hear that Rocketboom’s downloads have gone through the roof since they started offering a seperate HD feed. It might not be trivial to hook your PC to your HD but people are doing it.

Linux Devices: “Motorola is shipping the first model in its Scpl (”scalpel”) line of Linux-based phones set to replace the ubiquitous Razr”. That’s my phone! As long as the interface for texting isn’t so rubbish then this is pretty cool. Love the phone’s physical design, hate everything else about it. I want to throw it against the wall on a daily basis.