Please start any new threads on our new site at https://forums.sqlteam.com. We've got lots of great SQL Server experts to answer whatever question you can come up with.

 All Forums
 SQL Server 2005 Forums
 High Availability (2005)
 Mirroring Results In Split-Brain Scenario

Author  Topic 

ddamico
Yak Posting Veteran

76 Posts

Posted - 2008-06-28 : 07:45:17
Good day. I was at a customer site and saw a very interesting issue that the customer had faced with mirroring. It appears that an automatic failover to the mirror on multiple database had occured the odd thing was that the databases all came back in different modes

3 actual resulting scenarios occrurd
Server A Principal
Server B Mirror
Server C Witness

4 databases came back online as
A mirror synchronized/restoring and B principal synchronized

3 databases came back online as
A principal synchronized and B principal sychronizing

3 database came back online as
A principal disconnected/in recovery and B principal disconnected


The second scenario resulted in transactions being able to take place to both the principal and mirror. This seems like the "split brain" scenario which is not supposed to happen

Microsoft case formed they said they could reproduce the second scenario by causing a network issue. It also appears that the issue at the customer could be a network issue. The customer is doing the backups from the principal to the actual Witness machine over the network and the failover happened 10 minutes after the backups kicked off. Funny thing is that they have been running in this same layout for a while with no issues.


Principal and Mirror are at Version 3054 SQL Server Standard Edition 64 Bit and the Witness is at Version 3042 SQL Server Express Edition

1. Case 1 and 3 are easily explained but case 2 is puzzling. I am trying to understand the order of how a failover occurs and how the database could come back in different states?

2. Could there be a perfect sequence of events that could trigger the "split brain" result that hasn't been intended for by sql server software.


Has anyone heard of this or having information regarding this.
Thanks

ddamico@fittechnologies.net

sodeep
Master Smack Fu Yak Hacker

7174 Posts

Posted - 2008-06-28 : 12:30:38
quote:
Originally posted by ddamico

Good day. I was at a customer site and saw a very interesting issue that the customer had faced with mirroring. It appears that an automatic failover to the mirror on multiple database had occured the odd thing was that the databases all came back in different modes

3 actual resulting scenarios occrurd
Server A Principal
Server B Mirror
Server C Witness

4 databases came back online as
A mirror synchronized/restoring and B principal synchronized

3 databases came back online as
A principal synchronized and B principal sychronizing

3 database came back online as
A principal disconnected/in recovery and B principal disconnected


The second scenario resulted in transactions being able to take place to both the principal and mirror. This seems like the "split brain" scenario which is not supposed to happen

Microsoft case formed they said they could reproduce the second scenario by causing a network issue. It also appears that the issue at the customer could be a network issue. The customer is doing the backups from the principal to the actual Witness machine over the network and the failover happened 10 minutes after the backups kicked off. Funny thing is that they have been running in this same layout for a while with no issues.


Principal and Mirror are at Version 3054 SQL Server Standard Edition 64 Bit and the Witness is at Version 3042 SQL Server Express Edition

1. Case 1 and 3 are easily explained but case 2 is puzzling. I am trying to understand the order of how a failover occurs and how the database could come back in different states?

2. Could there be a perfect sequence of events that could trigger the "split brain" result that hasn't been intended for by sql server software.


Has anyone heard of this or having information regarding this.
Thanks

ddamico@fittechnologies.net





Dup Post: http://www.sqlteam.com/forums/topic.asp?TOPIC_ID=105675
Go to Top of Page
   

- Advertisement -