Amazon Elastic MapReduce/Hadoop

Not sure why but Amazon seems to be killing the whole innovation cycles & keep coming out with services integrated into AWS which otherwise a startup could have handled, nevertheless maybe we need to double our speed & innovate better & faster, sure so here comes the Amazon Utility service which basically will help run robust computational processes on demand & cost effective, we at attribo had started looking at integrating & making multiprocess algorithms easily deployable & manageable via cloud vendors, obviously Hadoop/mapreduce is something everyone would be interested in, though great I like the idea that Amazon has taken all that complexity involved in setting up hadoop/mapreduce environment & now it would be easy for us to integrate Amazon Elastic MapReduce directly into our product & build a wrapper layer to abstract out domain specific data churning algorithms....sad to see CloudEra's Hadoop initiatives go royally wasted ....more later

CloudCamp Bangalore 2009

Fantastic! so after almost 6+months from rest of the world where cloud brainstorming is on a rapid pace we in India got a chance to gather & meetup via CloudCamp @ Bangalore. Wonderful job done by Dave Neilsen(CloudCamp), prem(Novell), vinayak(Akamai), NARCEL, ACM, IIM & the volunteers who arranged the event & proceedings. After almost 6 month on look out for experts who are interested in cloud I had a sigh of relief to see a good amount of curious participants who introduced, argued, learnt & discussed about cloud. Overall a satisfactory event though I guess everyone went back with their own interpretation of Cloud, so confusion prevails as to what is cloud & what exactly represents cloud ex: SaaS, PaaS, IaaS etc were as confusing as ever. Though I firmly believe Cloud is all about making "the computational intensive, configurable & on demand infrastructure" accessible & available at a dramatically placed pricing model to an end user, what 1 can build on top of it is left to one's own creativity.

So my intention was to meet people & see what they are doing & what they want to know about cloud & how to jump start towards a cloud based business/service/product model, got introduced to few startups who are aggressively using the cloud (AMZ specifically) but for sure my focus towards making the end user experience while adopting to any cloud vendors seems to have gained a wide acceptance. also there are already talk to consolidate & publish a common cloud based API which interpets rest of the available Cloud vendor API's seems realistic. This is what I had aimed for since start & now I see good acceptance to a "Multi Cloud Vendor Management Console" which is what we at http://www.attribo.com/ are working on & about to deliver.

Utility Computing is something what I am also looking at & trying to understand how the cloud based architecture can be put in place for the same, bounced off at cloudEra who are doing a great job via bundling & making the Hadoop/Mapreduce process easily deployable & manageable by end users, also got to learn more about Intel/Yahoo/Hp's initiatives via OpenCirrus & Tashi (apache open source), Pig (Yahoo) so eventually most are trying to deliver a solution to allow end user adaptability of cloud.

so sure things are getting heated up & I am more inclined toward bulding or solving problems using cloud & not fall into arguing about what cloud is, as Dave Neilsen(CloudCamp) put's rightly
" Lets' learn & use Cloud for what is offers"

AMZ Console ...does it affect startups who solved the AMZ service usability issues?

yikes! honestly its heart breaking to see AMZ console, we at www.attribo.com had focused to make the AMZ services more usable via our point & click solution though we have already integrated with AMZ & GoGrid & all its services i.e EC@, S3, SDB etc , we dont see any revenue streams for our product henceforth given that AMZ is FREE! & they would be supporting all the services via that console. So looking forward we are changing our strategy & consolidating our approach towards multi vendor cloud management. Personally I think it was a great time spent digging into various API's which sure would help us build & innovate to suffice other issues while managing these cloud instances,

random thoughts

ok so its been almost few weeks that I am involved with cloud computing domain or at least looking into Amazon as my first cloud infrastructure & I hate to see there are so many arguments about what cloud should or should not be. Instead I think if there is a collective focus as to how to make the new concept acceptable by end users would make the time spent worth it.
Also I have already integrated my product with GoGrid API & seems to be a useful features when it comes to managing multiple vendor cloud environments. So cheers for all who have been involved in cloud zone, lets make the best out of it. Will be releasing my product for beta phase soon at www.attribo.com hope it solves at least some issues when it comes to actually managing the day to day admin tasks.

automating the cloud management

ok its been 2 weeks I have been exhaustively looking into offering a business solution based off cloud solution providers, though most of the conversation about cloud have been in terms of good/bad kind of debate & after going through all frustrating threads I finally myself jumped into coding frenzy once again, for various reasons I choose to dig into Amazon API's & build a management application to manage Amazon, keep an eye for www.attribo.com which I will be releasing soon for public usage, my intention is to make Amazon easy to understand for any NON Administrator or a general audience because thats where the main user base exist.

Infrastructure as a Service

just few hours back I wrote this article & already we have a best usecase where in IaaS providers will gain , http://edition.cnn.com/2008/POLITICS/09/30/congress.website/index.html
this is exactly the LIVE unpresedented, un anticipated user bombardment which could happen to any application available online/internet, I feel if they had a IaaS solution they would have easily scaled up instaed of taking the site down , so in this case both the parties are in loss i.e. the visitiors who should have been able to view the legislative document which they could not & the gov in itself was not able to server its own stakeholders (citizens) who have the rights towards the service offered.

Honestly even though I have mastered the programming side of the IT world I get sleepless when it comes to deployments & managing the applications designed, be it the 24x7 up time, faster response, or scalability issues it always feel being challenged by the unknowns of this ONLINE WORLD of inter connected applications. Good news though now that services like Hadoop, Amazon EC2, SQS, SimpleDB or evn FPS it has shifted the power back to the developers in terms of getting to design & deploy scalable & expandable infrastructure on demand which I feel will make my job much easier when it comes to deploy on EC2/S3 as a file system while run the applications as an AMI (amazon instance), essentially that is helping a common developer by abstracting the low/system level challenges into fragmented API's those are easily understandable & usable within shorter span of time & that too without having to be the master of all the low level transactions. even though amazon has been building these systems since last few years I think we are on verge of seeing an exponential growth for companies & end users (programmers) who can build & be successful with very high end solutions, & in case they even remotely FAIL (obviously for the solution use case they tackle) they can retry/realign their product without having any impact on the infrastructure investments (which are base don how much you use ) & is bare minimum even in terms of $100 & NOT Millions what we have seen getting evaporated because of above issues.

Only things that concerns me are that while accessing these on "demand business objects" off the cloud we end up paying PER FETCH that could mean that the objects are expensive till they are ALIVE, so I am giving it a try to understand how much of cost advantage it is to manage a BUSINESS OBJECT off & on cloud based platform. Though initial cost seems to be minuscule though over time the archived data would becomes more expensive (I guess) which could be a major issue for large enterprises who have to archive huge chunks of data for Sarbens Oxley or any other corporate obligations....

Cloud Computing

having seen the traditional software development efforts over the last 12+yrs & having seen the way we always have common development issues in terms of supporting the product on the server side & the object data store which is of utmost importance to scale the application & also offering the backend support for the business data that is being transacted day in & day out.This demands a greater team of resources who have expertise handling the server side & transaction level details(OS, Server Hardware, Databases, Front/GUI API etc).
Flash forward to 2008 & we are already seeing great support to Cloud or "Platform as a Service" models giving breather to groups of innovators who have been delaying the efforts due to failed teams or expensive resources, personally I have been involved with metadata based applications & support the cloud model by heart, have always tried to project that companies do not need a team of IT developers to design/develope/maintain day to day business applications, Amazon EC2/S3 & other upcoming data storage as well as PaaS Services will make it easy & cost effective for all to build/deploy/ & maintain applications in cloud with minimal IT resources/teams.
contact me on vshintre@definee.com if you are looking for a cloud/PaaS application developer