Please start any new threads on our new site at https://forums.sqlteam.com. We've got lots of great SQL Server experts to answer whatever question you can come up with.

 All Forums
 SQL Server 2005 Forums
 Transact-SQL (2005)
 What is the best way to update 100+ columns?

Author  Topic 

boogiezy
Starting Member

12 Posts

Posted - 2011-12-21 : 17:10:39
I have a table with 200 columns and need to update 100 columns of them. There are only 800-1000 rows in the tables. Don't know if it is important, but 95% of the columns needing update are float.
A basic update statement takes 1.5+ seconds to run. 800 rows take over 20 minutes.

UPDATE table_abc SET
col_101 = 1.23456789,
col_102 = 2.23456789,
col_103 = 3.23456789,
...
col_200 = 0.23456789
WHERE ID = 123;

Any suggestion?


Thanks,
ZY

tkizer
Almighty SQL Goddess

38200 Posts

Posted - 2011-12-21 : 17:40:49
Is the query being blocked? Have you looked at the wait types?

Tara Kizer
Microsoft MVP for Windows Server System - SQL Server
http://weblogs.sqlteam.com/tarad/

Subscribe to my blog
Go to Top of Page

X002548
Not Just a Number

15586 Posts

Posted - 2011-12-21 : 17:59:20
I'm guessing this is not in a sproc, nor is it part of an application and it's batch

AND

I'm guessing, you inherited this denormalized piece of garbage

(Correct me when I'm wrong)

AND I assume that there is NO WAY 800 rows should take 20 minutes

AND I know that's this thing is doing a scan (did you look at the plan?)

And What is the EXACT SQL..for example...WHERE do you get all of the values to SET in the first place?

Post the Code

and this miraculous table DDL


Brett

8-)

Hint: Want your questions answered fast? Follow the direction in this link
http://weblogs.sqlteam.com/brettk/archive/2005/05/25/5276.aspx


Want to help yourself?

http://msdn.microsoft.com/en-us/library/ms130214.aspx

http://weblogs.sqlteam.com/brettk/

http://brettkaiser.blogspot.com/


Go to Top of Page

boogiezy
Starting Member

12 Posts

Posted - 2011-12-22 : 11:14:04
Sorry, I tried to replied, but forgot to put in the password. It errored out, and lost my long reply. Then I had to go.

tkizer: Sorry, I don't know what you mean "query being blocked" and don't know how to look up wait type either.

X002548: Yes, it's inherited... Worse yet, tables were converted/migrated from MS Access without remodeling.
There’s a VBA module in the original Access that process some imported data (the first 100 columns), then compute values for the rest of the 100 columns. It only takes about 1 to 5 minutes to process and update 800-1000 rows.

I created a stored procedure and a view that pulls together all the necessary data from different tables for processing. The SP uses a cursor to go thru the view, processes each rows to get the 100 new data points, generate the UPDATE query string, then execute the update query. Without the UPDATE, the SP ran for a second; with the UPDATE, became 20-25 minutes... just to process 800 rows!
I ran one of these generated UPDATE query on SQL Server Management Studio. Just one update takes 1.5+ seconds…
Here is an example:
UPDATE [dbo].[BillingDetail] set measdata_key = 'Processed' , nFWV = 0.0039202269852283, nAsDelMMBTU = 2618.6937232558348000, nAsDelMCF = 2369.0000000000000000, nDryToAsDelMMBTU = 10.3062767441651890, nAsDelBTU = 1105.7500000000000000, nNetGthMMBTU = 2475.7130459660661000, nNetGthMCF = 2239.6525999999999000, nNatGasPrice = 4.2300000000000004, nC2Price = 0.4166605000000000, nC3Price = 1.3899336000000000, niC4Price = 1.9380025999999999, nnc4Price = 1.5607557999999999, niC5Price = 1.9776110000000000, nnC5Price = 1.9776110000000000, nC6Price = 1.9776110000000000, nGatheringFee = 0.1000000000000000, nGatheringValue = 261.8693723255834700, nCompressionFee = 0.0650000000000000, nCompressionValue = 510.6452760348877900, nCompressorFuelMCF = 124.3725000000000000, nCompressorFuelMMBtu = 137.4814204709313400, nSystemFuelMCF = 4.9748999999999999, nSystemFuelMMBTU = 5.4992568188372521, nDeliveredToPlantMCF = 1797.32, nDeliveredToPlantMMBTU = 1986.76, nBypassPct = 0.1975000000000000, nBypassMMBtu = 488.9533265782980600, nBypassMCF = 442.3313885000000000, nProcessingFee = 0.3000000000000000, nProcessingValue = 596.0279158163303900, nOtherFee = 0.5000000000000000, nOtherValue = 993.3798596938841000, nTotalGnPValue = 2361.9224238706856000, nNetNGLSettlementValue = 972.9815727964542000, nRecMode = 'Recovery', nC1MP = 90.8165772248488000, nC2MP = 4.6198179872425111, nC3MP = 1.5050765370253198, niC4MP = 0.3038043307695054, nnC4MP = 0.3466357610091405, niC5MP = 0.1842747580077328, nnC5MP = 0.1215217323078021, nC6MP = 0.5926674649437891, nN2MP = 0.3516161598742144, nCO2MP = 0.7659853454483594, nH2OMP = 0.3920226985228344, nTotalMP = 100.0000000000000000, nC2GPM = 1.2352437134157657, nC2TheoGals = 2220.1297274941826000, nC2RecPct = 0.5900000000000000, nC2RecGals = 1309.8765392215676000, nC2SettlementPrice = 0.4166605000000000, nC2Value = 545.7738137703279300, nC3GPM = 0.4145434653848236, nC3TheoGals = 745.0677634248594400, nC3RecPct = 0.9000000000000000, nC3RecGals = 670.5609870823734600, nC3SettlementPrice = 1.3899336000000000, nC3Value = 932.0352467949568300, niC4GPM = 0.0993904405014131, niC4TheoGals = 178.6365469335185300, niC4RecPct = 0.9400000000000000, niC4RecGals = 167.9183541175074200, niC4SettlementPrice = 1.9380025999999999, niC4Value = 325.4262068674500500, nnC4GPM = 0.1092583457145203, nnC4TheoGals = 196.3723422861074000, nnC4RecPct = 0.9500000000000000, nnC4RecGals = 186.5537251718020300, nnC4SettlementPrice = 1.5607557999999999, nnC4Value = 291.1648085734959700, niC5GPM = 0.0673751442081149, niC5TheoGals = 121.0947758131162700, niC5RecPct = 0.9500000000000000, niC5RecGals = 115.0400370224604500, niC5SettlementPrice = 1.9776110000000000, niC5Value = 227.5044426560250500, nnC5GPM = 0.0440399749996238, nnC5TheoGals = 79.1539812207535790, nnC5RecPct = 0.9500000000000000, nnC5RecGals = 75.1962821597158920, nnC5SettlementPrice = 1.9776110000000000, nnC5Value = 148.7089947581579000, nC6GPM = 0.2559585002675873, nC6TheoGals = 460.0396417946630000, nC6RecPct = 0.9500000000000000, nC6RecGals = 437.0376597049298100, nC6SettlementPrice = 1.9776110000000000, nC6Value = 864.2904832467259000, nTtlGPM = 2.2258095844918486, nTtlTheoGals = 4000.4947789672015000, nTtlRecGals = 2962.1835844803568000, nTtlNGLValue = 3334.9039966671398000, nZFactor = 0.9973935646887231, nC2ShrinkMMBtu = 87.1242934468687620, nC3ShrinkMMBtu = 61.5590252774342590, niC4ShrinkMMBtu = 16.7734244665479080, nnC4ShrinkMMBtu = 19.4036578282542390, niC5ShrinkMMBtu = 12.6505641376995360, nnC5ShrinkMMBtu = 8.3587984705411653, nC6ShrinkMMBtu = 51.4019897458092740, nTtlShrinkMMBtu = 257.2717533731551500, nPlantFuelPct = 0.0250000000000000, nPlantFuelMMBtu = 49.6689929846942060, nPTRMMBtu = 306.9407463578493800, nResidueMMBtu = 2168.7722996082166000 WHERE ID = 104
~~~~~~~~~~~~~~~~~~~~~~~~~
“Script Table as CREATE” gives me this:
CREATE TABLE [dbo].[BillingDetail](
[ID] [int] IDENTITY(1,1) NOT NULL,
[MeasData_Key] [varchar](255) NULL,
[Accounting_Month] [datetime] NULL,
[Production_Month] [datetime] NULL,
[Owner_ID] [varchar](10) NULL,
[Owner_Name] [varchar](60) NULL,

… (another 180 columns in between)
[niC4ShrinkMMBtu] [float] NULL,
[nnC4ShrinkMMBtu] [float] NULL,
[niC5ShrinkMMBtu] [float] NULL,
[nnC5ShrinkMMBtu] [float] NULL,
[nC6ShrinkMMBtu] [float] NULL,
[nTtlShrinkMMBtu] [float] NULL,
[nPlantFuelPct] [float] NULL,
[nPlantFuelMMBtu] [float] NULL,
[nPTRMMBtu] [float] NULL,
[nResidueMMBtu] [float] NULL,
[nGenerateNGLStatement] [bit] NULL,
[nUpdateDate] [datetime] NULL,
[nUpdateBy] [varchar](30) NULL,
[Update_User] [dbo].[Update_User] NULL,
[Update_Date] [dbo].[Update_Date] NULL,
CONSTRAINT [PK_BillingDetail] PRIMARY KEY CLUSTERED
(
[ID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]


~~~~~~~~~~~~~~~~~~~~~~~~~
BTW, I tried both EXECUTE [query string] and EXECUTE sp_executesql […]. Resulting performance difference was insignificant.
There were 16 indexes generated by SQL Server. I disabled all by the PK index. No difference in performance either.

Seems to me SQL Server is taking too long to search for ONE column out of 200, and do this 100 times for just updating 1 row.

My last resort is to leave thing in MS Access!


Thanks,
ZY
Go to Top of Page

X002548
Not Just a Number

15586 Posts

Posted - 2011-12-22 : 11:54:34
it would help if you posted the sproc

Brett

8-)

Hint: Want your questions answered fast? Follow the direction in this link
http://weblogs.sqlteam.com/brettk/archive/2005/05/25/5276.aspx


Want to help yourself?

http://msdn.microsoft.com/en-us/library/ms130214.aspx

http://weblogs.sqlteam.com/brettk/

http://brettkaiser.blogspot.com/


Go to Top of Page

boogiezy
Starting Member

12 Posts

Posted - 2011-12-22 : 12:52:39
I'm sending the version with simple "EXEC (@QueryString)". Like I said the "EXECUTE sp_executesql" did not help with the performance much if at all. It is way longer because of another set of parm variables.


USE [EXXXXXHE]
GO

/****** Object: StoredProcedure [dbo].[exsp_Process_Exxxxxhe] Script Date: 12/22/2011 11:21:44 ******/
IF EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[dbo].[exsp_Process_Exxxxxhe]') AND type in (N'P', N'PC'))
DROP PROCEDURE [dbo].[exsp_Process_Exxxxxhe]
GO

USE [EXXXXXHE]
GO

/****** Object: StoredProcedure [dbo].[exsp_Process_Exxxxxhe] Script Date: 12/22/2011 11:21:44 ******/
SET ANSI_NULLS ON
GO

SET QUOTED_IDENTIFIER ON
GO


-- =============================================
-- Author: ZY
-- Create date: Dec 1, 2011
-- Description: Converted from Will's Access VBA
-- =============================================
CREATE PROCEDURE [dbo].[exsp_Process_Exxxxxhe]
-- Add the parameters for the stored procedure here
@inContract_Number varchar(12) = NULL
--,@inProd_Month Datetime
--,@inAcct_Month Datetime
--,@Cursor_Count Int Output
AS
BEGIN
-- SET NOCOUNT ON added to prevent extra result sets from
-- interfering with SELECT statements.
SET NOCOUNT ON;

print 'start exsp_Process_Exxxxxhe for ' + @inContract_Number

-- declare variables for current rows
declare
@c_Owner_Contract_Number Varchar(12),
@c_Accounting_Month datetime,
@c_Production_Month datetime,
@c_StationId Varchar(10),
@c_ContractNumber Varchar(12),
@c_AcctMonth datetime,
@c_RecoveryMode bit,
@c_ProcessFlag bit,
@c_PlantFuel Float,
@c_CustomerName Varchar(100),
@c_NGPriceIndex int,
@c_NGLPriceIndex int,
@c_ContractH2OBasis Varchar(1),
@c_PriceNG Float,
@c_PriceNGL_C2Purity Float,
@c_PriceNGL_C2 Float,
@c_PriceNGL_C3 Float,
@c_PriceNGL_IC4 Float,
@c_PriceNGL_NC4 Float,
@c_PriceNGL_IC5 Float,
@c_PriceNGL_NC5 Float,
@c_PriceNGL_C6Plus Float,
@c_PlantRecTerm Varchar(24),
@c_BeginDate datetime,
@c_EndDate datetime,
@c_Description Varchar(255),
@c_ProcessGas bit,
@c_BypassPct Float,
@c_RecC2Pct Float,
@c_RecC3Pct Float,
@c_ReciC4Pct Float,
@c_RecnC4Pct Float,
@c_ReciC5Pct Float,
@c_RecnC5Pct Float,
@c_RecC6Pct Float,
@c_RejC2Pct Float,
@c_RejC3Pct Float,
@c_RejiC4Pct Float,
@c_RejnC4Pct Float,
@c_RejiC5Pct Float,
@c_RejnC5Pct Float,
@c_RejC6Pct Float,
@c_GatheringFee Float,
@c_CompFee Float,
@c_ProcessingFee Float,
@c_OtherFee Float,
@c_C2TnFCharge Float,
@c_C3PlusTnFCharge Float,
@c_FnECharge Float,
@c_BTURequired int,
@c_Stages_Of_Compression numeric(10,0),
@c_MeasurementBasis Varchar(3),
@c_Dry_Contract_Btu numeric(10,3),
@c_GrossMmbtu numeric(22,10),
@c_GrossVolume numeric(22,10),
@c_Compression_Fuel_Percent numeric(10,5),
@c_Compression_Rate numeric(10,5),
@c_System_Fuel_Percent numeric(10,5),
@c_Gathering_Transmission_Rate numeric(10,5),
@c_Btu_Dry numeric(10,3),
@c_Nitrogen_MP numeric(8,4),
@c_Co2_MP numeric(8,4),
@c_Methane_MP numeric(8,4),
@c_Ethane_MP numeric(8,4),
@c_Propane_MP numeric(8,4),
@c_Isobutane_MP numeric(8,4),
@c_NButane_MP numeric(8,4),
@c_Isopentane_MP numeric(8,4),
@c_NPentane_MP numeric(8,4),
@c_HexanePlus_MP numeric(8,4),
@c_FlowingTemp Float,
@c_FlowingPress Float,
@c_DataProcessStatus Varchar(255),
@c_nProcessError bit,
@c_nGenerateNGLStatement bit,
@c_BillingDetail_ID int,

@ErrorMsg Varchar(255),
@UpdateBDSQL1 VARCHAR(MAX),
@UpdateBDSQL2 VARCHAR(MAX),
@Update_count int,
-- variables for debugging only
@debug_var_int int,
@debug_var_varchar varchar(255),
@debug_var_float float
-- end variables for debugging only
SET @Update_count =0

if UPPER(@inContract_Number) = 'ALL' or @inContract_Number is NULL
begin
Declare Column_Cursor Cursor SCROLL--FORWARD_ONLY --
FOR SELECT * from [dbo].[vBillingDetail_b4DataProcess] order by stationid
end
else
begin
Declare Column_Cursor Cursor SCROLL--FORWARD_ONLY --
FOR SELECT * from [dbo].[vBillingDetail_b4DataProcess] where Owner_Contract_Number = @inContract_Number order by stationid
end

Open Column_Cursor
Fetch Next From Column_Cursor into
@c_Owner_Contract_Number ,
@c_Accounting_Month ,
@c_Production_Month ,
@c_StationId ,
@c_ContractNumber ,
@c_AcctMonth ,
@c_RecoveryMode ,
@c_ProcessFlag ,
@c_PlantFuel ,
@c_CustomerName ,
@c_NGPriceIndex ,
@c_NGLPriceIndex ,
@c_ContractH2OBasis ,
@c_PriceNG ,
@c_PriceNGL_C2Purity ,
@c_PriceNGL_C2 ,
@c_PriceNGL_C3 ,
@c_PriceNGL_IC4 ,
@c_PriceNGL_NC4 ,
@c_PriceNGL_IC5 ,
@c_PriceNGL_NC5 ,
@c_PriceNGL_C6Plus ,
@c_PlantRecTerm ,
@c_BeginDate ,
@c_EndDate ,
@c_Description ,
@c_ProcessGas ,
@c_BypassPct ,
@c_RecC2Pct ,
@c_RecC3Pct ,
@c_ReciC4Pct ,
@c_RecnC4Pct ,
@c_ReciC5Pct ,
@c_RecnC5Pct ,
@c_RecC6Pct ,
@c_RejC2Pct ,
@c_RejC3Pct ,
@c_RejiC4Pct ,
@c_RejnC4Pct ,
@c_RejiC5Pct ,
@c_RejnC5Pct ,
@c_RejC6Pct ,
@c_GatheringFee ,
@c_CompFee ,
@c_ProcessingFee ,
@c_OtherFee ,
@c_C2TnFCharge ,
@c_C3PlusTnFCharge ,
@c_FnECharge ,
@c_BTURequired ,
@c_Stages_Of_Compression ,
@c_MeasurementBasis ,
@c_Dry_Contract_Btu ,
@c_GrossMmbtu ,
@c_GrossVolume ,
@c_Compression_Fuel_Percent ,
@c_Compression_Rate ,
@c_System_Fuel_Percent ,
@c_Gathering_Transmission_Rate ,
@c_Btu_Dry ,
@c_Nitrogen_MP ,
@c_Co2_MP ,
@c_Methane_MP ,
@c_Ethane_MP ,
@c_Propane_MP ,
@c_Isobutane_MP ,
@c_NButane_MP ,
@c_Isopentane_MP ,
@c_NPentane_MP ,
@c_HexanePlus_MP ,
@c_FlowingTemp ,
@c_FlowingPress ,
@c_DataProcessStatus ,
@c_nProcessError ,
@c_nGenerateNGLStatement ,
@c_BillingDetail_ID
While @@FETCH_STATUS=0
BEGIN
-- ** declare variables for Exxxxxhe calculations
declare @zStatusMsg varchar(255)
,@bErrFlag bit
,@dFWV float
,@dAsDelMMBTU float
,@dAsDelMCF float
,@dDryToAsDelMMBTU float
,@dAsDelBTU float
,@dNetGatheredAsDelMMBTU float
,@dNetGatheredMCF float
,@dPriceNG float
,@dPriceNGLs_C2 float
,@dPriceNGLs_C3 float
,@dPriceNGLs_iC4 float
,@dPriceNGLs_nC4 float
,@dPriceNGLs_iC5 float
,@dPriceNGLs_nC5 float
,@dPriceNGLs_C6 float
,@dGathFee float
,@dGatheringValue float
,@dCompFee float
,@dCompressionValue float
,@dCompressionFuelMCF float
,@dCompressionFuelMMBtu float
,@dSystemFuelMCF float
,@dSystemFuelMMBtu float
,@dDelToPlantMCF float --result will be saved as INT, but calculations need to be float
,@dDelToPlantMMBtu float --result will be saved as INT, but calculations need to be float
,@dBypassPct float
,@dBypassMMBtu float
,@dBypassMCF float
,@dProcFee float
,@dProcessFeeValue float
,@dOtherFee float
,@dOtherFeeValue float
,@dTtlGnPValue float
,@dNetNGLValue float
,@bRecMode varchar(12)
,@dC1MP_AsDel float
,@dC2MP_AsDel float
,@dC3MP_AsDel float
,@diC4MP_AsDel float
,@dnC4MP_AsDel float
,@diC5MP_AsDel float
,@dnC5MP_AsDel float
,@dC6MP_AsDel float
,@dN2MP_AsDel float
,@dCO2MP_AsDel float
,@dH2OMP float
,@dTotalMP float
,@dGPM_C2 float
,@dTheoGals_C2 float
,@dRecoveries_C2 float
,@dRecoveredGals_C2 float
,@dNGLValue_C2 float
,@dGPM_C3 float
,@dTheoGals_C3 float
,@dRecoveries_C3 float
,@dRecoveredGals_C3 float
,@dNGLValue_C3 float
,@dGPM_iC4 float
,@dTheoGals_iC4 float
,@dRecoveries_iC4 float
,@dRecoveredGals_iC4 float
,@dNGLValue_iC4 float
,@dGPM_nC4 float
,@dTheoGals_nC4 float
,@dRecoveries_nC4 float
,@dRecoveredGals_nC4 float
,@dNGLValue_nC4 float
,@dGPM_iC5 float
,@dTheoGals_iC5 float
,@dRecoveries_iC5 float
,@dRecoveredGals_iC5 float
,@dNGLValue_iC5 float
,@dGPM_nC5 float
,@dTheoGals_nC5 float
,@dRecoveries_nC5 float
,@dRecoveredGals_nC5 float
,@dNGLValue_nC5 float
,@dGPM_C6 float
,@dTheoGals_C6 float
,@dRecoveries_C6 float
,@dRecoveredGals_C6 float
,@dNGLValue_C6 float
,@dTtlGPMs float
,@dTtlTheoGals float
,@dTtlRecoveredGals float
,@dTtlNGLValue float
,@dzFactor float
,@dShrinkMMBtu_C2 float
,@dShrinkMMBtu_C3 float
,@dShrinkMMBtu_iC4 float
,@dShrinkMMBtu_nC4 float
,@dShrinkMMBtu_iC5 float
,@dShrinkMMBtu_nC5 float
,@dShrinkMMBtu_C6 float
,@dTtlShrinkMMBtu float
,@dPlantFuelPct float
,@dPlantFuelMMBtu float
,@dPTRMMBtu float
,@dPlantResidueMMBtu float
,@bGenNGLStatement bit
-- ** end variables declaration for Exxxxxhe calculations
Set @ErrorMsg = @c_StationId
if @c_PriceNG IS NULL
begin
Set @ErrorMsg = @ErrorMsg + ': No NG Price'
end
if @c_PriceNGL_C2 IS NULL
begin
Set @ErrorMsg = @ErrorMsg + ': No NGL Price'
end
if @c_PlantRecTerm IS NULL
begin
Set @ErrorMsg = @ErrorMsg + ': No PlantRecTerm'
end
if @c_Methane_MP IS NULL
begin
Set @ErrorMsg = @ErrorMsg + ': No Mole Percent Data'
end

if @c_PriceNG IS NOT NULL and @c_PriceNGL_C2 IS NOT NULL and
@c_PlantRecTerm IS NOT NULL and @c_Methane_MP IS NOT NULL
begin
-- for debugging using 1 station only
if ltrim(@c_StationId) = '03560'
begin
SET @UpdateBDSQL1 = ''
SET @UpdateBDSQL2 = ''


-- Exxxxxhe Calculations
SET @dFWV = [...calcuation code...]
SET @dAsDelMMBTU = [...calcuation code...]
SET @dAsDelMCF = [...calcuation code...]
SET @dDryToAsDelMMBTU = [...calcuation code...]
SET @dAsDelBTU = [...calcuation code...]
SET @dPriceNG = [...calcuation code...]
SET @dPriceNGLs_C2 = [...calcuation code...]

[[...calcuation code... for the rest of the @dxxxxx variables]

SET @dPlantFuelMMBtu = [...calcuation code...]
SET @dPTRMMBtu = [...calcuation code...]
SET @dPlantResidueMMBtu = [...calcuation code...]

-- end Exxxxxhe Calculations
SET @Update_count = @Update_count + 1
SET @UpdateBDSQL1 = 'UPDATE [dbo].[BillingDetail] set measdata_key = ''Processed'' ' +
', nFWV = ' + str(@dFWV, 25,17) +
', nAsDelMMBTU = ' + str(@dAsDelMMBTU, 25,17) +
', nAsDelMCF = ' + str(@dAsDelMCF, 25,17) +
', nDryToAsDelMMBTU = ' + str(@dDryToAsDelMMBTU, 25,17) +
', nAsDelBTU = ' + str(@dAsDelBTU, 25,17) +
', nNetGthMMBTU = ' + str(@dNetGatheredAsDelMMBTU, 25,17) +
', nNetGthMCF = ' + str(@dNetGatheredMCF, 25,17) +
', nNatGasPrice = ' + str(@dPriceNG, 25,17) +
', nC2Price = ' + str(@dPriceNGLs_C2, 25,17) +
', nC3Price = ' + str(@dPriceNGLs_C3, 25,17) +
', niC4Price = ' + str(@dPriceNGLs_iC4, 25,17) +
', nnc4Price = ' + str(@dPriceNGLs_nC4, 25,17) +
', niC5Price = ' + str(@dPriceNGLs_iC5, 25,17) +
', nnC5Price = ' + str(@dPriceNGLs_nC5, 25,17) +
', nC6Price = ' + str(@dPriceNGLs_C6, 25,17) +
', nGatheringFee = ' + str(@dGathFee, 25,17) +
', nGatheringValue = ' + str(@dGatheringValue, 25,17) +
', nCompressionFee = ' + str(@dCompFee, 25,17) +
', nCompressionValue = ' + str(@dCompressionValue, 25,17) +
', nCompressorFuelMCF = ' + str(@dCompressionFuelMCF, 25,17) +
', nCompressorFuelMMBtu = ' + str(@dCompressionFuelMMBtu, 25,17) +
', nSystemFuelMCF = ' + str(@dSystemFuelMCF, 25,17) +
', nSystemFuelMMBTU = ' + str(@dSystemFuelMMBtu, 25,17) +
', nDeliveredToPlantMCF = ' + str(@dDelToPlantMCF, 10,2) +
', nDeliveredToPlantMMBTU = ' + str(@dDelToPlantMMBtu, 10,2) +
', nBypassPct = ' + str(@dBypassPct, 25,17) +
', nBypassMMBtu = ' + str(@dBypassMMBtu, 25,17) +
', nBypassMCF = ' + str(@dBypassMCF, 25,17) +
', nProcessingFee = ' + str(@dProcFee, 25,17) +
', nProcessingValue = ' + str(@dProcessFeeValue, 25,17) +
', nOtherFee = ' + str(@dOtherFee, 25,17) +
', nOtherValue = ' + str(@dOtherFeeValue, 25,17) +
', nTotalGnPValue = ' + str(@dTtlGnPValue, 25,17) +
', nNetNGLSettlementValue = ' + str(@dNetNGLValue, 25,17) --+
SET @UpdateBDSQL2 =
', nRecMode = ''' + @bRecMode + '''' +
', nC1MP = ' + str(@dC1MP_AsDel, 25,17) +
', nC2MP = ' + str(@dC2MP_AsDel, 25,17) +
', nC3MP = ' + str(@dC3MP_AsDel, 25,17) +
', niC4MP = ' + str(@diC4MP_AsDel, 25,17) +
', nnC4MP = ' + str(@dnC4MP_AsDel, 25,17) +
', niC5MP = ' + str(@diC5MP_AsDel, 25,17) +
', nnC5MP = ' + str(@dnC5MP_AsDel, 25,17) +
', nC6MP = ' + str(@dC6MP_AsDel, 25,17) +
', nN2MP = ' + str(@dN2MP_AsDel, 25,17) +
', nCO2MP = ' + str(@dCO2MP_AsDel, 25,17) +
', nH2OMP = ' + str(@dH2OMP, 25,17) +
', nTotalMP = ' + str(@dTotalMP, 25,17) +
', nC2GPM = ' + str(@dGPM_C2, 25,17) +
', nC2TheoGals = ' + str(@dTheoGals_C2, 25,17) +
', nC2RecPct = ' + str(@dRecoveries_C2, 25,17) +
', nC2RecGals = ' + str(@dRecoveredGals_C2, 25,17) +
', nC2SettlementPrice = ' + str(@dPriceNGLs_C2, 25,17) +
', nC2Value = ' + str(@dNGLValue_C2, 25,17) +
', nC3GPM = ' + str(@dGPM_C3, 25,17) +
', nC3TheoGals = ' + str(@dTheoGals_C3, 25,17) +
', nC3RecPct = ' + str(@dRecoveries_C3, 25,17) +
', nC3RecGals = ' + str(@dRecoveredGals_C3, 25,17) +
', nC3SettlementPrice = ' + str(@dPriceNGLs_C3, 25,17) +
', nC3Value = ' + str(@dNGLValue_C3, 25,17) +
', niC4GPM = ' + str(@dGPM_iC4, 25,17) +
', niC4TheoGals = ' + str(@dTheoGals_iC4, 25,17) +
', niC4RecPct = ' + str(@dRecoveries_iC4, 25,17) +
', niC4RecGals = ' + str(@dRecoveredGals_iC4, 25,17) +
', niC4SettlementPrice = ' + str(@dPriceNGLs_iC4, 25,17) +
', niC4Value = ' + str(@dNGLValue_iC4, 25,17) +
', nnC4GPM = ' + str(@dGPM_nC4, 25,17) +
', nnC4TheoGals = ' + str(@dTheoGals_nC4, 25,17) +
', nnC4RecPct = ' + str(@dRecoveries_nC4, 25,17) +
', nnC4RecGals = ' + str(@dRecoveredGals_nC4, 25,17) +
', nnC4SettlementPrice = ' + str(@dPriceNGLs_nC4, 25,17) +
', nnC4Value = ' + str(@dNGLValue_nC4, 25,17) +
', niC5GPM = ' + str(@dGPM_iC5, 25,17) +
', niC5TheoGals = ' + str(@dTheoGals_iC5, 25,17) +
', niC5RecPct = ' + str(@dRecoveries_iC5, 25,17) +
', niC5RecGals = ' + str(@dRecoveredGals_iC5, 25,17) +
', niC5SettlementPrice = ' + str(@dPriceNGLs_iC5, 25,17) +
', niC5Value = ' + str(@dNGLValue_iC5, 25,17) +
', nnC5GPM = ' + str(@dGPM_nC5, 25,17) +
', nnC5TheoGals = ' + str(@dTheoGals_nC5, 25,17) +
', nnC5RecPct = ' + str(@dRecoveries_nC5, 25,17) +
', nnC5RecGals = ' + str(@dRecoveredGals_nC5, 25,17) +
', nnC5SettlementPrice = ' + str(@dPriceNGLs_nC5, 25,17) +
', nnC5Value = ' + str(@dNGLValue_nC5, 25,17) +
', nC6GPM = ' + str(@dGPM_C6, 25,17) +
', nC6TheoGals = ' + str(@dTheoGals_C6, 25,17) +
', nC6RecPct = ' + str(@dRecoveries_C6, 25,17) +
', nC6RecGals = ' + str(@dRecoveredGals_C6, 25,17) +
', nC6SettlementPrice = ' + str(@dPriceNGLs_C6, 25,17) +
', nC6Value = ' + str(@dNGLValue_C6, 25,17) +
', nTtlGPM = ' + str(@dTtlGPMs, 25,17) +
', nTtlTheoGals = ' + str(@dTtlTheoGals, 25,17) +
', nTtlRecGals = ' + str(@dTtlRecoveredGals, 25,17) +
', nTtlNGLValue = ' + str(@dTtlNGLValue, 25,17) +
', nZFactor = ' + str(@dzFactor, 25,17) +
', nC2ShrinkMMBtu = ' + str(@dShrinkMMBtu_C2, 25,17) +
', nC3ShrinkMMBtu = ' + str(@dShrinkMMBtu_C3, 25,17) +
', niC4ShrinkMMBtu = ' + str(@dShrinkMMBtu_iC4, 25,17) +
', nnC4ShrinkMMBtu = ' + str(@dShrinkMMBtu_nC4, 25,17) +
', niC5ShrinkMMBtu = ' + str(@dShrinkMMBtu_iC5, 25,17) +
', nnC5ShrinkMMBtu = ' + str(@dShrinkMMBtu_nC5, 25,17) +
', nC6ShrinkMMBtu = ' + str(@dShrinkMMBtu_C6, 25,17) +
', nTtlShrinkMMBtu = ' + str(@dTtlShrinkMMBtu, 25,17) +
', nPlantFuelPct = ' + str(@dPlantFuelPct, 25,17) +
', nPlantFuelMMBtu = ' + str(@dPlantFuelMMBtu, 25,17) +
', nPTRMMBtu = ' + str(@dPTRMMBtu, 25,17) +
', nResidueMMBtu = ' + str(@dPlantResidueMMBtu, 25,17) +
' WHERE ID = ' + str(@c_BillingDetail_ID,10,0)
print str(@Update_count) +'-'+ @c_StationId + ':- '
--PRINT @UpdateBDSQL1
--PRINT @UpdateBDSQL2
if @Update_count <= 20 Exec(@UpdateBDSQL1 + @UpdateBDSQL2) --limit to 20 for debugging
end
-- end debugging

end
--else
--Print @ErrorMsg

--get next row from view
Fetch Next From Column_Cursor into
@c_Owner_Contract_Number ,
@c_Accounting_Month ,
@c_Production_Month ,
@c_StationId ,
@c_ContractNumber ,
@c_AcctMonth ,
@c_RecoveryMode ,
@c_ProcessFlag ,
@c_PlantFuel ,
@c_CustomerName ,
@c_NGPriceIndex ,
@c_NGLPriceIndex ,
@c_ContractH2OBasis ,
@c_PriceNG ,
@c_PriceNGL_C2Purity ,
@c_PriceNGL_C2 ,
@c_PriceNGL_C3 ,
@c_PriceNGL_IC4 ,
@c_PriceNGL_NC4 ,
@c_PriceNGL_IC5 ,
@c_PriceNGL_NC5 ,
@c_PriceNGL_C6Plus ,
@c_PlantRecTerm ,
@c_BeginDate ,
@c_EndDate ,
@c_Description ,
@c_ProcessGas ,
@c_BypassPct ,
@c_RecC2Pct ,
@c_RecC3Pct ,
@c_ReciC4Pct ,
@c_RecnC4Pct ,
@c_ReciC5Pct ,
@c_RecnC5Pct ,
@c_RecC6Pct ,
@c_RejC2Pct ,
@c_RejC3Pct ,
@c_RejiC4Pct ,
@c_RejnC4Pct ,
@c_RejiC5Pct ,
@c_RejnC5Pct ,
@c_RejC6Pct ,
@c_GatheringFee ,
@c_CompFee ,
@c_ProcessingFee ,
@c_OtherFee ,
@c_C2TnFCharge ,
@c_C3PlusTnFCharge ,
@c_FnECharge ,
@c_BTURequired ,
@c_Stages_Of_Compression ,
@c_MeasurementBasis ,
@c_Dry_Contract_Btu ,
@c_GrossMmbtu ,
@c_GrossVolume ,
@c_Compression_Fuel_Percent ,
@c_Compression_Rate ,
@c_System_Fuel_Percent ,
@c_Gathering_Transmission_Rate ,
@c_Btu_Dry ,
@c_Nitrogen_MP ,
@c_Co2_MP ,
@c_Methane_MP ,
@c_Ethane_MP ,
@c_Propane_MP ,
@c_Isobutane_MP ,
@c_NButane_MP ,
@c_Isopentane_MP ,
@c_NPentane_MP ,
@c_HexanePlus_MP ,
@c_FlowingTemp ,
@c_FlowingPress ,
@c_DataProcessStatus ,
@c_nProcessError ,
@c_nGenerateNGLStatement ,
@c_BillingDetail_ID
END


CLOSE Column_Cursor
DEALLOCATE Column_Cursor

print 'end exsp_Process_Exxxxxhe'
print str(@Update_count) + ' record Processed'

END
GO




Thanks,
ZY
Go to Top of Page

X002548
Not Just a Number

15586 Posts

Posted - 2011-12-22 : 12:56:38
There is no need for a cursor

why don't you just collapse it into 1 update with a join?

and select * is just plain bad form

and how can you have a set of declares INSIDE a loop?

Brett

8-)

Hint: Want your questions answered fast? Follow the direction in this link
http://weblogs.sqlteam.com/brettk/archive/2005/05/25/5276.aspx


Want to help yourself?

http://msdn.microsoft.com/en-us/library/ms130214.aspx

http://weblogs.sqlteam.com/brettk/

http://brettkaiser.blogspot.com/


Go to Top of Page

X002548
Not Just a Number

15586 Posts

Posted - 2011-12-22 : 13:05:15
I think you only need this



UPDATE b
SET measdata_key = 'Processed'
, billingdetail COLUMNS = d.columns and calculations
FROM [dbo].[BillingDetail] b
JOIN [dbo].[vBillingDetail_b4DataProcess] d
ON b.key = d.key
WHERE Owner_Contract_Number = CASE WHEN @inContract_Number = 'ALL' THEN Owner_Contract_Number
WHEN @inContract_Number IS NULL THEN Owner_Contract_Number
ELSE @inContract_Number






Brett

8-)

Hint: Want your questions answered fast? Follow the direction in this link
http://weblogs.sqlteam.com/brettk/archive/2005/05/25/5276.aspx


Want to help yourself?

http://msdn.microsoft.com/en-us/library/ms130214.aspx

http://weblogs.sqlteam.com/brettk/

http://brettkaiser.blogspot.com/


Go to Top of Page

boogiezy
Starting Member

12 Posts

Posted - 2011-12-22 : 13:14:36
hmm how do I do the calcuations when some of the values depends on others? for example the calcuated dFWV feeds many other fields, and some are SUM of 6 new values...

Thanks,
ZY
Go to Top of Page

X002548
Not Just a Number

15586 Posts

Posted - 2011-12-22 : 13:19:58
I Assume the "Values" are coming from the Cursor..correct?

Can you post an example?

But it's gonna be

UPDATE
SET Col = d.Col1 + d. Col2 / d. Col3

Or whatever the formula is

I've written some doozies

As a matter of fact, I actually create a view with those calculations to obfuscate the complexity and just select from the view

THEN, anywhere those calcs are required, they just use the view

THEN, let's say something is wrong with one calc..all you do is change the view and nothing else




Brett

8-)

Hint: Want your questions answered fast? Follow the direction in this link
http://weblogs.sqlteam.com/brettk/archive/2005/05/25/5276.aspx


Want to help yourself?

http://msdn.microsoft.com/en-us/library/ms130214.aspx

http://weblogs.sqlteam.com/brettk/

http://brettkaiser.blogspot.com/


Go to Top of Page

boogiezy
Starting Member

12 Posts

Posted - 2011-12-22 : 14:11:09
example of those caculations:

SET @dFWV = [dbo].f_Exxxxhe_AsDel_CF_v1(@c_Methane_MP , @c_Ethane_MP , @c_Propane_MP , @c_Isobutane_MP , @c_NButane_MP , @c_Isopentane_MP , @c_NPentane_MP , @c_HexanePlus_MP , @c_Co2_MP , @c_Nitrogen_MP , 0 , @c_FlowingTemp , @c_FlowingPress , 14.73)
SET @dAsDelMMBTU = (1 - @dFWV) * @c_GrossMmbtu
SET @dDryToAsDelMMBTU = @c_GrossMmbtu - @dAsDelMMBTUSET @dAsDelBTU = ROUND(@c_Btu_Dry * (1 - @dFWV),2)
SET @dGatheringValue = @c_GatheringFee * @dAsDelMMBTUSET @dSystemFuelMMBtu = @c_System_Fuel_Percent * @dAsDelMMBTU / 100
SET @dNetGatheredAsDelMMBTU = @dAsDelMMBTU - @dSystemFuelMMBtu - @dCompressionFuelMMBtu
SET @dTtlGnPValue = @dProcessFeeValue + @dOtherFeeValue + @dGatheringValue + @dCompressionValue


~~~~~~~~~~~~~
but like I said, the SProc runs very fast if I comment out the update query. Even if I turn it into a single update, it will still has 100 columns to set values.
Won't you think the first example that set values to a hardcoded numbers would update fast? but it took 1.5+ SECONDS.

UPDATE [dbo].[BillingDetail] set measdata_key = 'Processed' , nFWV = 0.0039202269852283, nAsDelMMBTU = 2618.6937232558348000, nAsDelMCF = 2369.0000000000000000, nDryToAsDelMMBTU = 10.3062767441651890, nAsDelBTU = 1105.7500000000000000, nNetGthMMBTU = 2475.7130459660661000, nNetGthMCF = 2239.6525999999999000, [another 90+ columns] WHERE ID = 104


Thanks,
ZY
Go to Top of Page

X002548
Not Just a Number

15586 Posts

Posted - 2011-12-22 : 14:24:57
I assume that the input to the udf is from the cursor

and so you have here 2 levels of derived tables...to keep the calc simple...I mean you can sill do it in one...but

what type of UDF is this?

[dbo].f_Exxxxhe_AsDel_CF_v1





Brett

8-)

Hint: Want your questions answered fast? Follow the direction in this link
http://weblogs.sqlteam.com/brettk/archive/2005/05/25/5276.aspx


Want to help yourself?

http://msdn.microsoft.com/en-us/library/ms130214.aspx

http://weblogs.sqlteam.com/brettk/

http://brettkaiser.blogspot.com/


Go to Top of Page

Kristen
Test

22859 Posts

Posted - 2011-12-23 : 00:34:46
quote:
Originally posted by boogiezy

hmm how do I do the calcuations when some of the values depends on others? for example the calcuated dFWV feeds many other fields, and some are SUM of 6 new values...



Dunno if relevant, but if it helps you can do

UPDATE MyTable
SET @SomeVariable = Col1 + Col2,
@SomeOtherVariable = @SomeVariable + Col3,
Col4 = @SomeOtherVariable,
...

I use this where I need "working variables", per row (or even inter-row), or to "return" values to a process that needs them after the UPDATE completes (not very useful for a multiple-row update, but handy for single-row updates)
Go to Top of Page

X002548
Not Just a Number

15586 Posts

Posted - 2011-12-23 : 08:57:46
you can do that????

Another undocumented "Feature"

You can't do that in a select



Brett

8-)

Hint: Want your questions answered fast? Follow the direction in this link
http://weblogs.sqlteam.com/brettk/archive/2005/05/25/5276.aspx


Want to help yourself?

http://msdn.microsoft.com/en-us/library/ms130214.aspx

http://weblogs.sqlteam.com/brettk/

http://brettkaiser.blogspot.com/


Go to Top of Page

boogiezy
Starting Member

12 Posts

Posted - 2011-12-23 : 16:49:42
"UPDATE MyTable SET @SomeVariable = Col1 + Col2" doesn't make sense to me either. Don't we have to use a column name where "@SomeVariable" is?

When I get back to the office Tuesday, I'll create another 200 columns table and see if a simple update on 100 columns takes long. The table would have 100 varchar and 100 float columns, and named col_001 to col_200. If you guys can try the same and let me know how you long it takes you for 1 row, that would be nice.

Merry Christmas you guys!


Thanks,
ZY
Go to Top of Page

Kristen
Test

22859 Posts

Posted - 2011-12-25 : 14:25:46
quote:
Originally posted by X002548

You can't do that in a select



Indeed ...
Go to Top of Page

Kristen
Test

22859 Posts

Posted - 2011-12-25 : 14:31:40
quote:
Originally posted by boogiezy

"UPDATE MyTable SET @SomeVariable = Col1 + Col2"
doesn't make sense to me either. Don't we have to use a column name where "@SomeVariable" is?


No, after the SET you can do:

ColumnName = expression
or
@VariableName = Expression
or
@VariableName = ColumnName = Expression
(to save whatever value the ColumnName is set to also to @VariableName)

Its documented:

SET
{ column_name = { expression | DEFAULT | NULL }
| { udt_column_name.{ { property_name = expression
| field_name = expression }
| method_name ( argument [ ,...n ] )
}
}
| column_name { .WRITE ( expression , @Offset , @Length ) }
| @variable = expression
| @variable = column = expression
| column_name { += | -= | *= | /= | %= | &= | ^= | |= } expression
| @variable { += | -= | *= | /= | %= | &= | ^= | |= } expression
| @variable = column { += | -= | *= | /= | %= | &= | ^= | |= } expression
} [ ,...n ]
Go to Top of Page

Transact Charlie
Master Smack Fu Yak Hacker

3451 Posts

Posted - 2011-12-26 : 18:17:57
quote:
Originally posted by X002548

you can do that????
Another undocumented "Feature"



'Quirky' update pretty much depends on s**t like this.

Just remember that there is NO GUARANTEED ORDER al all (even with an ORDER BY clause) unless there is a clustered index on the table in question. (and then you get the clustered index order.... and THEN only if there is no parallelism involved.

basically. don't do this - unless you absolutely need the hyper speed. (even then if you do need to do that then you probably shouldn't be using a database for the job anyway)

in interesting (or otherwise) aside. MySQL uses on c**p like this all the time (I think).

Charlie
===============================================================
Msg 3903, Level 16, State 1, Line 1736
The ROLLBACK TRANSACTION request has no corresponding BEGIN TRANSACTION
Go to Top of Page

boogiezy
Starting Member

12 Posts

Posted - 2011-12-27 : 13:05:54
I did the test on updating a new 200 columns table, it was FAST. So that's not the problem. You all already know that...

I cleaned up all the extended properties, probably came from MS Access conversion. Then I deleted all the extra indexes and constraints that were generated by something. No improvement from that.

Finally, I saw there's a trigger created by another person for "audit trail" which recorded the old and new values of each column. That's where all the processing time went!!

HO... HO... HO...LY COW!!!

I disabled the trigger; the sp_executesql took only 1 second.
So, I'd rather not spend few more days to change from cursor to a single enormous UPDATE query.

But what advice do you have on this audit trail trigger that records every changed columns? (other than not use it, I mean...)


Thanks,
ZY
Go to Top of Page

sunitabeck
Master Smack Fu Yak Hacker

5155 Posts

Posted - 2011-12-27 : 13:48:16
Rather than auditing each column, save the entire DELETED (and INSERTED) rows into the audit table, along with other info you need (for example, timestamp, the user who made the update/insert/delete etc.) Granted, that will take up more disk space (but if you have only a few thousand rows and if the updates are fairly infrequent that should not be a major consideration).
Go to Top of Page

boogiezy
Starting Member

12 Posts

Posted - 2011-12-27 : 14:28:56
The audit table is shared by all tables with the audit trigger. Fields include [Type],[TableName],[PK],[FieldName],[OldValue],[NewValue],[UpdateDate],[UserName].
To do what you suggested, will I need to create a separate audit table for this 200 columns table?

Is it possible to disable and enable a trigger in a stored proc? if so, what's the syntax


Thanks,
ZY
Go to Top of Page
    Next Page

- Advertisement -