RE: Reliability in of the cloud

I don't think real-time transfer is needed, but your framework should be intelligent enough to continue in the light of failure Although its possible to have 2 trackers that work in a High Availibilty mode (constantly pinging each other to detect failure), hooking in cloud events can make life easier.

The question remains - is broadcasting failures necessary ?

Regards,
Alan Ho

---

From: Khazret Sapenov <sapenov@gmail.com>
Sent: June 05, 2008 6:00 PM
To: cloud-computing@googlegroups.com
Subject: Re: Reliability in of the cloud

Alan,

If you are talking about Hadoop, then high availability is not inherent in it yet (but maybe it changed recently).
As far as I know, while there is Secondary Name Node provided (that resides in another data center) there's no guarantee of real time switch of Job Tracker/Name Node/Task Tracker/Data Nodes of DC A to Job Tracker/Name Node/Task Tracker/Data Nodes of DC B.

 

cheers

--
Khaz Sapenov,
Director of Research & Development
Enomaly Labs

US Phone: 212-461-4988 x5
Canada Phone: 416-848-6036 x5
E-mail: khaz@enomaly.net
Get Linked in> http://www.linkedin.com/in/sapenov

On Thu, Jun 5, 2008 at 8:41 PM, Alan Ho <karlunho@yahoo.ca> wrote:

I guessed that about google app engine too.

Things get really interesting when you need to do election leader decisions across data centers. E.g. If you are doing a big map-reduce task in one data center, it goes down, so you want to finish the task in another data center.

How does one transfer the task ? Is it even worth solving ?

Alan Ho

---

From: Reuven Cohen <ruv@enomaly.com>
Sent: June 05, 2008 10:03 AM
To: cloud-computing@googlegroups.com
Subject: Re: The Business of Building Clouds

From what I've seen of Google App Engine, they distribute your python code to dozens of servers and then use some kind of round robin to spread the load. Nothing ground breaking.

r/c

On Thu, Jun 5, 2008 at 12:59 PM, wyim wyim <wingmanyim@hotmail.com> wrote:

In regards to failover, does Google App Engine have some sort of a LoadBalancer API?
 
thanks
Wayne Yim

---

From: stuartcharlton@gmail.com

To: cloud-computing@googlegroups.com

Subject: Re: The Business of Building Clouds

Date: Thu, 5 Jun 2008 09:07:12 -0700

On 5-Jun-08, at 8:35 AM, Alan Ho wrote:

Picking a provider that has data center failover is critical - but it does mean that you write your application in a way that can failover gracefully. Cloud providers need to provide the base infrastructure to do so OR constrain the user to a particular programming paradigm (like the limitations of google app engine)

That's a very astute observation, Alan.    Constraining an architecture to induce certain properties (guarantees?) is likely the right approach.  

 Though I wonder if AppEngine is a bit too "Nanny-ish" that limit its audience in ways that don't really impact the big picture qualities.   

For example, the choice of Python was easy because it was a standard Google language, but that doesn't seem to be inherently a more applicable language than say C#, Java or Ruby.

I expect in the future that cloud computing systems will provide the concept of "cloud events" in case of major datacenter failures. I just don't see any way round it.

I wonder if Google actually provides this sort of failover for AppEngine today.   Certainly, they could, though they provide no such guarantees at the moment.

As for "cloud events" - yup.   In the traditional data centre, it's likely SNMP or JMX traps.   On the cloud, it's not entirely clear if/where SNMP would play.   Or WS-Man.   Or something newer (?).

Cheers

Stu

--
--

Reuven Cohen
Founder & Chief Technologist, Enomaly Inc.
www.enomaly.com :: 416 848 6036 x 1
skype: ruv.net // aol: ruv6

blog > www.elasticvapor.com
-
Get Linked in> http://linkedin.com/pub/0/b72/7b4

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Cloud Computing" group.
To post to this group, send email to cloud-computing@googlegroups.com
To unsubscribe from this group, send email to cloud-computing-unsubscribe@googlegroups.com
For more options, visit this group at http://groups.google.ca/group/cloud-computing?hl=en
-~----------~----~----~----~------~----~------~--~---