Please start any new threads on our new
site at https://forums.sqlteam.com. We've got lots of great SQL Server
experts to answer whatever question you can come up with.
Author |
Topic |
Kristen
Test
22859 Posts |
Posted - 2007-03-07 : 06:10:54
|
Edit: Some changes addedThis is my procedure for "rescuing" data from a corrupted database. Obviously restoring from backup is a lot easier!0) Set the damaged database to Read-Only. if you don't have a backup make one now.1) Script the database2a) Create a new, TEMP database - preferably on a different machine in case of hardware problems on the original machine2b) Size the Data for the TEMP database same size as the original (to avoid dynamic extensions). Size the Log something large-ish!3) Run the Script on the TEMP database. Do NOT create any FK etc. yet4a) Attempt to transfer all tables:-- Prepare script of: INSERT INTO ... SELECT * FROM ...SET NOCOUNT ONSELECT 'PRINT ''' + name + '''' + CHAR(13) + CHAR(10) + 'GO' + CHAR(13) + CHAR(10) + CASE WHEN C.id IS NULL THEN '' ELSE 'SET IDENTITY_INSERT dbo.[' + name + '] ON' + CHAR(13) + CHAR(10) END + 'INSERT INTO MyTempDatabase.dbo.[' + name + ']' + CHAR(13) + CHAR(10) + 'SELECT * FROM dbo.[' + name + ']' + CHAR(13) + CHAR(10) + CASE WHEN C.id IS NULL THEN '' ELSE 'SET IDENTITY_INSERT dbo.[' + name + '] OFF' + CHAR(13) + CHAR(10) END + 'GO' FROM dbo.sysobjects AS O LEFT OUTER JOIN ( SELECT DISTINCT C.id FROM dbo.syscolumns AS C WHERE C.colstat = 1 -- Identity column ) AS C ON C.id = O.idWHERE type = 'U' AND name NOT IN ('dtproperties')ORDER BY nameSET NOCOUNT OFF this generates statements like this:PRINT 'MyTable'GOSET IDENTITY_INSERT dbo.[MyTable] ONINSERT INTO RESTORE_XFER_TEMP.dbo.[MyTable]SELECT * FROM dbo.[MyTable]SET IDENTITY_INSERT dbo.[MyTable] OFFGO 4b) This will give some sort of error on the tables which cannot be copied, and they will need to be rescued by some other means.5a) Each "broken" table needs to be rescued using an index. Ideally you will have a clustered index on the PK and that will be undamaged, so you can "rescue" all the PKs into a temp table:-- Copy PK fields to a temporary table-- DROP TABLE MyRestoreDatabase.dbo.TEMP_RESCUE_PK-- TRUNCATE TABLE MyRestoreDatabase.dbo.MyBrokenTableSELECT [ID]=IDENTITY(int, 1, 1), [IsCopied]=CONVERT(tinyint, 0), MyPKINTO MyRestoreDatabase.dbo.TEMP_RESCUE_PK FROM MyBrokenDatabase.dbo.MyBrokenTable ORDER BY MyPK 5b) If that is successful you have a list of all the PKs, so can can try to copy data matching those PKs, in batches:-- If OK then selectively copy data across-- First Prep. a temp Batch table-- DROP TABLE MyRestoreDatabase.dbo.TEMP_RESCUE_BATCHSELECT TOP 1 [ID]=CONVERT(int, NULL), [IsCopied]=CONVERT(bit, 0), MyPKINTO MyRestoreDatabase.dbo.TEMP_RESCUE_BATCHFROM MyRestoreDatabase.dbo.TEMP_RESCUE_PKGO--DECLARE @intStart int, @intStop int, @intBatchSize int-- NOTE: After the first run set these to any "gaps" in the table that you want to fillSELECT @intStart = 1, @intBatchSize = 10000, @intStop = (SELECT MAX([ID]) FROM MyRestoreDatabase.dbo.TEMP_RESCUE_PK)SELECT @intStart = MIN([ID])FROM MyRestoreDatabase.dbo.TEMP_RESCUE_PKWHERE IsCopied = 0 AND [ID] >= @intStartWHILE @intStart < @intStopBEGIN SET ROWCOUNT @intBatchSize -- Isolate batch of Keys into separate table TRUNCATE TABLE MyRestoreDatabase.dbo.TEMP_RESCUE_BATCH INSERT INTO MyRestoreDatabase.dbo.TEMP_RESCUE_BATCH SELECT T.* FROM MyRestoreDatabase.dbo.TEMP_RESCUE_PK AS T WHERE IsCopied = 0 AND [ID] >= @intStart AND [ID] < @intStart + @intBatchSize -- Attempt to copy matching records, for this batch PRINT CONVERT(varchar(20), @intStart) INSERT INTO MyRestoreDatabase.dbo.MyBrokenTable SELECT S.* FROM MyRestoreDatabase.dbo.TEMP_RESCUE_BATCH AS T LEFT OUTER JOIN MyRestoreDatabase.dbo.MyBrokenTable AS D ON D.MyPK = T.MyPK -- This will try to get the data from the broken table, it may fail! JOIN MyBrokenDatabase.dbo.MyBrokenTable AS S ON S.MyPK = T.MyPK WHERE D.MyPK IS NULL -- Belt and braces so as not to copy existing rows -- Flag the rows just "Copied" UPDATE U SET IsCopied = 1 FROM MyRestoreDatabase.dbo.TEMP_RESCUE_PK AS U WHERE IsCopied = 0 AND [ID] >= @intStart AND [ID] < @intStart + @intBatchSize -- Loop round, until done SELECT @intStart = @intStart + @intBatchSizeENDGOSET ROWCOUNT 0 -- Turn OFF!!GO 5c) This will copy in batches of 10,000 [you can adjust @intbatchSize depending on table size] until it gets to a damaged part of the table, then it will abort.Change the @intStart to the last ID number displayed, and reduce @intBatchSize (by an order of magnitude each time) until you have rescued as many records as possible in the first "part" of the table.5d) Reset the batch size @intBatchSize to 10,000 [or whatever size is appropriate], and increase the @intStart repeatedly until you are past the damaged section - copying will start again, and will abort if there are further damaged sections5e) Repeat that process until you have rescued as much of the data as is possible6) Check what is left to be rescued-- Check amount NOT done:SELECT COUNT(*), MIN([ID]), MAX([ID])FROM MyRestoreDatabase.dbo.TEMP_RESCUE_PKWHERE IsCopied = 0-- AND [ID] > 123456 -- Optionally count items after a "gap"---- Double check that IsCopied set correctly, and the number of records "lost"SELECT COUNT(*), [IsCopied] = SUM(CONVERT(int, IsCopied)), [IsCopied+Record] = SUM(CASE WHEN IsCopied = 1 AND C.MyPK IS NOT NULL THEN 1 ELSE 0 END), [IsCopiedNoRecord] = SUM(CASE WHEN IsCopied = 1 AND C.MyPK IS NULL THEN 1 ELSE 0 END), [IsNOTCopied] = SUM(CASE WHEN IsCopied = 0THEN 1 ELSE 0 END), [IsNOTCopied+Record] = SUM(CASE WHEN IsCopied = 0 AND C.MyPK IS NOT NULL THEN 1 ELSE 0 END), [IsNOTCopiedNoRecord] = SUM(CASE WHEN IsCopied = 0 AND C.MyPK IS NULL THEN 1 ELSE 0 END)FROM MyRestoreDatabase.dbo.TEMP_RESCUE_PK AS T LEFT OUTER JOIN MyRestoreDatabase.dbo.MyBrokenTable AS C ON C.MyPK = T.MyPK---- List of "Lost" recordsSELECT MyPKFROM MyRestoreDatabase.dbo.TEMP_RESCUE_PKWHERE IsCopied = 0ORDER BY [ID] You will then have to "find" and rescue the lost records somewhere.I have a further process using OPENQUERY() to rescue records to fill the gaps in the event that they are available on a remote system - a straight JOIN to get them is going to be far to slow on anything other than tiny tables!7a) Create the FKs etc. Update the statistics, rebuild indexes, and possibly shrink the Log if it is unrealistically big7b) Backup and Restore over the original database7c) DBCC CHECKDB ('MyDatabaseName') WITH ALL_ERRORMSGS, NO_INFOMSGSGood luck!Kristen |
|
Andraax
Aged Yak Warrior
790 Posts |
Posted - 2007-03-08 : 03:23:13
|
Very nice!How often do you experience corrupted data? I've yet to experience it, and I've been working with SQL Server since 1997 (not on a very large scale though)./Andraax |
|
|
Kristen
Test
22859 Posts |
Posted - 2007-03-08 : 03:38:53
|
We've had three corruptions on one server in the last year. All caused by failure of a drive in RAID5. That database is in FULL recovery mode and on all three occasions we were able to fix it by:Take a "final" TLog backupRestore FULL backupRestore all TLog backupsthus there was ZERO data loss. We keep the Backups on separate drives to the data - so the chance of them both failing at the same time is very low. Plus that server should be using RAID10 ... but that's another story!All the corruptions I have come across were caused by hardware failure.Kristen |
|
|
Kristen
Test
22859 Posts |
Posted - 2007-06-20 : 04:44:49
|
There are some useful steps to Diagnose and Recover a corruption here:http://www.sqlteam.com/forums/topic.asp?TOPIC_ID=84958#314526 |
|
|
frank.svs
Constraint Violating Yak Guru
368 Posts |
Posted - 2012-08-03 : 01:20:51
|
Hi Frank,This is a lovely post.I have little requirement. In your script, it is able to handle both non-identity tables as well as identity tables.The problem here is , if we do not specify the explicit column list for identity tables, the insertion fails with below error. Msg 8101, Level 16, State 1, Line 1An explicit value for the identity column in table 'Target.dbo.Emp' can only be specified when a column list is used and IDENTITY_INSERT is ON.Do you have any logic or script to modify the same above script to handle this. Here is what i have tested. Src =====use mastergoCREATE DATABASE Demo go use demo go create table Dept(Deptno int not null primary key,Dname varchar(100))go insert into dept values(10,'ACCOUNTS');insert into dept values(20,'SALES');insert into dept values(30,'MARKETTING');insert into dept values(40,'FINANCE');gocreate table Emp(Empno int not null primary key identity(1,1), Ename varchar(10), Esal int, Deptno int references dept(deptno))go insert into Emp(Ename,Esal,Deptno) values('Manu',9000,10)insert into Emp(Ename,Esal,Deptno) values('Anu',9000,20)insert into Emp(Ename,Esal,Deptno) values('Ajay',9000,30)insert into Emp(Ename,Esal,Deptno) values('Sidhu',9000,40)insert into Emp(Ename,Esal,Deptno) values('Brahma',9000,20)insert into Emp(Ename,Esal,Deptno) values('Mayank',9000,10)create table Student (sid int, sname varchar(10))go insert into student values(1001,'Mark')insert into student values(1002,'Anthony')insert into student values(1003,'Smith')goTarget ======use mastergoCREATE DATABASE target go use target go create table Dept(Deptno int not null primary key,Dname varchar(100))go create table Emp(Empno int not null primary key identity(1,1), Ename varchar(10), Esal int, Deptno int references dept(deptno))go create table Student (sid int, sname varchar(10))go use Demogo -- Prepare script of: INSERT INTO ... SELECT * FROM ...SET NOCOUNT ONSELECT 'PRINT ''' + name + '''' + CHAR(13) + CHAR(10) + 'GO' + CHAR(13) + CHAR(10) + CASE WHEN C.id IS NULL THEN '' ELSE 'SET IDENTITY_INSERT dbo.[' + name + '] ON' + CHAR(13) + CHAR(10) END + 'INSERT INTO Target.dbo.[' + name + ']' + CHAR(13) + CHAR(10) + 'SELECT * FROM dbo.[' + name + ']' + CHAR(13) + CHAR(10) + CASE WHEN C.id IS NULL THEN '' ELSE 'SET IDENTITY_INSERT dbo.[' + name + '] OFF' + CHAR(13) + CHAR(10) END + 'GO' FROM dbo.sysobjects AS O LEFT OUTER JOIN ( SELECT DISTINCT C.id FROM dbo.syscolumns AS C WHERE C.colstat = 1 -- Identity column ) AS C ON C.id = O.idWHERE type = 'U' AND name NOT IN ('dtproperties')ORDER BY nameSET NOCOUNT OFF-- output -- This will take care of identity tables as well and normal tablesPRINT 'Dept'GOINSERT INTO Target.dbo.[Dept]SELECT * FROM dbo.[Dept]GOPRINT 'Emp'GOSET IDENTITY_INSERT Target.dbo.[Emp] ONgoINSERT INTO Target.dbo.[Emp]SELECT * FROM dbo.[Emp]SET IDENTITY_INSERT Target.dbo.[Emp] OFFGOPRINT 'Student'GOINSERT INTO Target.dbo.[Student]SELECT * FROM dbo.[Student]GOAppreciate if someone can provide inputs on this.Thank you. |
|
|
|
|
|
|
|