Re: Reliability in of the cloud

The problem of delivering availability and reliability, across two
clouds, or across one cloud and your local network, is one that we ran
into at CohesiveFT. We ran into it because some of our own systems
are hosted in a 'multi-sourced' or 'multi-provider' way.

To solve this problem one of the team, Dmitriy Samovskiy, created a
cross-VPN routing layer which we then open sourced as 'VCUBEV'. It's
a really simple and neat idea, and anyone else is welcome to use it.
There are a one or two gotchas to watch out for, such as latency
between sites, but we've found that for a capable administrator these
are manageable.

A good place to start is:

http://highscalability.com/manage-downtime-risk-connecting-multiple-data-centers-secure-virtual-lan

More details and downloadable code are here:

http://www.cohesiveft.com/Developer/VcubeV/VcubeV_-_Usecases/

Enjoy :-)

alexis

On Fri, Jun 6, 2008 at 2:32 AM, Alan Ho <karlunho@yahoo.ca> wrote:
> I don't think real-time transfer is needed, but your framework should be
> intelligent enough to continue in the light of failure Although its possible
> to have 2 trackers that work in a High Availibilty mode (constantly pinging
> each other to detect failure), hooking in cloud events can make life easier.
>
> The question remains - is broadcasting failures necessary ?
>
> Regards,
> Alan Ho
>
>
>
>
> ________________________________
> From: Khazret Sapenov <sapenov@gmail.com>
> Sent: June 05, 2008 6:00 PM
> To: cloud-computing@googlegroups.com
> Subject: Re: Reliability in of the cloud
>
> Alan,
> If you are talking about Hadoop, then high availability is not inherent in
> it yet (but maybe it changed recently).
> As far as I know, while there is Secondary Name Node provided (that resides
> in another data center) there's no guarantee of real time switch of Job
> Tracker/Name Node/Task Tracker/Data Nodes of DC A to Job Tracker/Name
> Node/Task Tracker/Data Nodes of DC B.
>
> cheers
> --
> Khaz Sapenov,
> Director of Research & Development
> Enomaly Labs
>
> US Phone: 212-461-4988 x5
> Canada Phone: 416-848-6036 x5
> E-mail: khaz@enomaly.net
> Get Linked in> http://www.linkedin.com/in/sapenov
> On Thu, Jun 5, 2008 at 8:41 PM, Alan Ho <karlunho@yahoo.ca> wrote:
>>
>> I guessed that about google app engine too.
>>
>> Things get really interesting when you need to do election leader
>> decisions across data centers. E.g. If you are doing a big map-reduce task
>> in one data center, it goes down, so you want to finish the task in another
>> data center.
>>
>> How does one transfer the task ? Is it even worth solving ?
>>
>> Alan Ho
>>
>>
>> ________________________________
>> From: Reuven Cohen <ruv@enomaly.com>
>> Sent: June 05, 2008 10:03 AM
>> To: cloud-computing@googlegroups.com
>> Subject: Re: The Business of Building Clouds
>>
>> From what I've seen of Google App Engine, they distribute your python code
>> to dozens of servers and then use some kind of round robin to spread the
>> load. Nothing ground breaking.
>>
>> r/c
>>
>> On Thu, Jun 5, 2008 at 12:59 PM, wyim wyim <wingmanyim@hotmail.com> wrote:
>>>
>>>
>>> In regards to failover, does Google App Engine have some sort of a
>>> LoadBalancer API?
>>>
>>> thanks
>>> Wayne Yim
>>>
>>>
>>> ________________________________
>>> From: stuartcharlton@gmail.com
>>> To: cloud-computing@googlegroups.com
>>> Subject: Re: The Business of Building Clouds
>>> Date: Thu, 5 Jun 2008 09:07:12 -0700
>>>
>>>
>>>
>>> On 5-Jun-08, at 8:35 AM, Alan Ho wrote:
>>>
>>> Picking a provider that has data center failover is critical - but it
>>> does mean that you write your application in a way that can failover
>>> gracefully. Cloud providers need to provide the base infrastructure to do so
>>> OR constrain the user to a particular programming paradigm (like the
>>> limitations of google app engine)
>>>
>>> That's a very astute observation, Alan. Constraining an architecture
>>> to induce certain properties (guarantees?) is likely the right approach.
>>> Though I wonder if AppEngine is a bit too "Nanny-ish" that limit its
>>> audience in ways that don't really impact the big picture qualities.
>>> For example, the choice of Python was easy because it was a standard
>>> Google language, but that doesn't seem to be inherently a more applicable
>>> language than say C#, Java or Ruby.
>>>
>>> I expect in the future that cloud computing systems will provide the
>>> concept of "cloud events" in case of major datacenter failures. I just don't
>>> see any way round it.
>>>
>>> I wonder if Google actually provides this sort of failover for AppEngine
>>> today. Certainly, they could, though they provide no such guarantees at
>>> the moment.
>>> As for "cloud events" - yup. In the traditional data centre, it's
>>> likely SNMP or JMX traps. On the cloud, it's not entirely clear if/where
>>> SNMP would play. Or WS-Man. Or something newer (?).
>>> Cheers
>>> Stu
>>>
>>>
>>>
>>
>>
>>
>> --
>> --
>>
>> Reuven Cohen
>> Founder & Chief Technologist, Enomaly Inc.
>> www.enomaly.com :: 416 848 6036 x 1
>> skype: ruv.net // aol: ruv6
>>
>> blog > www.elasticvapor.com
>> -
>> Get Linked in> http://linkedin.com/pub/0/b72/7b4
>>
>>
>>
>>
>>
>
> >
>

--
Alexis Richardson
+44 20 7617 7339 (UK)
+44 77 9865 2911 (cell)
+1 650 206 2517 (US)

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Cloud Computing" group.
To post to this group, send email to cloud-computing@googlegroups.com
To unsubscribe from this group, send email to cloud-computing-unsubscribe@googlegroups.com
For more options, visit this group at http://groups.google.ca/group/cloud-computing?hl=en
-~----------~----~----~----~------~----~------~--~---