Please start any new threads on our new
site at https://forums.sqlteam.com. We've got lots of great SQL Server
experts to answer whatever question you can come up with.
| Author |
Topic |
|
jimf
Master Smack Fu Yak Hacker
2875 Posts |
Posted - 2010-04-12 : 10:43:47
|
Hi all,I am working on a getting a running total so as to get the incremental amount necessary to keep that running total under a certain amount - to cap the actual running total. This query produces the correct result, but it seems a bit cumbersome to me. The actual underlying data set could have as many as 30,000 records, so was wondering if somebody has a more elegant solution?Thanks,JimDECLARE @cap intSET @cap = 100000DECLARE @table TABLE (ayq int,cyq int, paid int)INSERT INTO @table SELECT 200604, 200604, 2125 UNION ALLSELECT 200604, 200701, 305 UNION ALLSELECT 200604, 200702, 0 UNION ALLSELECT 200604, 200703, 0 UNION ALLSELECT 200604, 200704, 0 UNION ALLSELECT 200604, 200802, 0 UNION ALLSELECT 200604, 200903, 0 UNION ALLSELECT 200604, 200904, 107000 SELECT t2.ayq,t2.cyq,t2.paid ,t2.totpaid ,[IncrementalPaid] = CASE WHEN @cap - t2.totpaid < 0 THEN @cap - (t2.TotPaid - t2.paid) ELSE t2.paid ENDFROM ( select t1.ayq,t1.cyq, t1.paid ,[TotPaid] = sum(t2.paid) --,@cap - sum(t2.paid) from @table t1 inner join @table t2 on t1.ayq = t2.ayq and t1.cyq >= t2.cyq group by t1.ayq,t1.cyq, t1.paid ) t2RESULT SET:ayq cyq paid totpaid IncrementalPaid200604 200604 2125 2125 2125200604 200701 305 2430 305200604 200702 0 2430 0200604 200703 0 2430 0200604 200704 0 2430 0200604 200802 0 2430 0200604 200903 0 2430 0200604 200904 107000 109430 97570 Everyday I learn something that somebody else already knew |
|
|
DBA in the making
Aged Yak Warrior
638 Posts |
Posted - 2010-04-12 : 11:38:36
|
This could be one of the few occasions where using a cursor gives better results. I think a cursor would be quicker, because you're method isn't keeping a single running total, it's, running an aggregate query for every row in the table, which is pretty expensive on long tables. Try this:DECLARE @ayq INTDECLARE @cyq INT DECLARE @paid INTDECLARE @ayqLast INT --SET @ayqLast = 0DECLARE @totpaid INTDECLARE @tableOut TABLE (ayq int,cyq int, paid int, totpaid INT) --, [IncrementalPaid] INT) DECLARE cur CURSOR FORWARD_ONLY FORSELECT ayq, cyq, paidFROM @TableORDER BY ayq, cyqOPEN curFETCH NEXT FROM cur INTO @ayq, @cyq, @paidWHILE @@Fetch_Status = 0BEGIN IF @ayqLast = @ayq BEGIN -- The ayq value is the same as the previous record. SET @totpaid = @totpaid + @paid END ELSE BEGIN -- The ayq value has changed. SET @totpaid = @paid END INSERT INTO @tableOut VALUES (@ayq, @cyq, @paid, @totpaid) SET @ayqLast = @ayq FETCH NEXT FROM cur INTO @ayq, @cyq, @paid ENDCLOSE curDEALLOCATE curSELECT ayq,cyq,paid ,totpaid ,[IncrementalPaid] = CASE WHEN @cap - totpaid < 0 THEN @cap - (TotPaid - paid) ELSE paid ENDFROM @tableOut ------------------------------------------------------------------------------------Any and all code contained within this post comes with a 100% money back guarantee. |
 |
|
|
waterduck
Aged Yak Warrior
982 Posts |
Posted - 2010-04-12 : 11:55:42
|
;with cte as( select ROW_NUMBER() over (order by ayq, cyq) num, * from @table )select a.ayq, a.cyq, a.paid, incre, a.paid-isnull(b.paid, 0)from cte a left join cte b on a.ayq=b.ayq and a.num=b.num+1cross apply(select SUM(paid)incre from cte c where c.cyq<=a.cyq and c.ayq=a.ayq)c Hope can help...but advise to wait pros with confirmation... |
 |
|
|
DBA in the making
Aged Yak Warrior
638 Posts |
Posted - 2010-04-12 : 12:11:15
|
quote: Originally posted by waterduck ;with cte as( select ROW_NUMBER() over (order by ayq, cyq) num, * from @table )select a.ayq, a.cyq, a.paid, incre, a.paid-isnull(b.paid, 0)from cte a left join cte b on a.ayq=b.ayq and a.num=b.num+1cross apply(select SUM(paid)incre from cte c where c.cyq<=a.cyq and c.ayq=a.ayq)c
That doesn't give the same results as the original (In fact it fails to take the cap into account at all), and it still has the same issue as the original, it runs an aggregate query for each row returned.------------------------------------------------------------------------------------Any and all code contained within this post comes with a 100% money back guarantee. |
 |
|
|
waterduck
Aged Yak Warrior
982 Posts |
Posted - 2010-04-12 : 12:15:39
|
so sorry...i was just trying to help Hope can help...but advise to wait pros with confirmation... |
 |
|
|
jimf
Master Smack Fu Yak Hacker
2875 Posts |
Posted - 2010-04-12 : 12:22:15
|
| Thanks for the input guys. Long time no see waterduck, I was wondering where you've been. Unfortunately your answer doesn't quite give me what I'm looking for.DBAITM, you may be right, a cursor may be what is needed here. I do need the running total row-by-row, as I need to know when the amount goes over a certain limit (suppied by the user). I also found that my query doesn't work either. If I take a @cap of 2000 it illustrates the point that I also need to take into account that after that cap is reached, every incremental thereafter is 0. I really don't want to introduce another CASE statement, but that may be the best solution. I'll take a few tests with my way vs. cursors and see which one runs best and will post back when I get an answer.Thanks,JimEveryday I learn something that somebody else already knew |
 |
|
|
visakh16
Very Important crosS Applying yaK Herder
52326 Posts |
Posted - 2010-04-12 : 13:31:20
|
| [code]DECLARE @cap intSET @cap = 100000SELECT t.ayq,t.cyq,t.paid,t.paid + COALESCE(t1.paid,0) AS totpaid,CASE WHEN @cap - (t.paid + COALESCE(t1.paid,0)) < 0 THEN @cap - COALESCE(t1.paid,0) ELSE t.paid END AS IncrementalPaidFROM YourTable tOUTER APPLY (SELECT SUM(paid) AS prevpaid FROM Table WHERE ayq=t.ayq AND cyq < t. cyq )t1[/code]------------------------------------------------------------------------------------------------------SQL Server MVPhttp://visakhm.blogspot.com/ |
 |
|
|
madhivanan
Premature Yak Congratulator
22864 Posts |
|
|
DBA in the making
Aged Yak Warrior
638 Posts |
Posted - 2010-04-13 : 10:13:43
|
| I did some testing on this. The server I used is running inside a VM on my laptop, so it's not the fastest. For data, I populates the @Table with 100,000 rows. 10,000 ayq values, each with 10 cyq values, all with random paid values. I ran a test on the method I posted, which used a cursor (Method 1), against the method posted by visakh16, which used repeated SUMs (Method 2). With the table declared as is, here were the run times. Method 1:- 7 secondsMethod 2:- 15 minutes 46 secondsThen I added a clustered index to the table declaration on ayq, cyq, and the results changed dramatically. Method 1:- 9 secondsMethod 2:- 3 secondsThen I used a temp table instead of a table variable, and with the same clustered index, I got the same results. However, when I remove the index, I expected Method 2 to slow down again, but it didn't. Here are the result for that:Method 1:- 7 secondsMethod 2:- 4 secondsI ran these tests several times, and the results were very consistent. So for method 2, the clustered index had very little effect on the temp table, but a massive effect on the table variable. And for method 1, it decreased performance.------------------------------------------------------------------------------------Any and all code contained within this post comes with a 100% money back guarantee. |
 |
|
|
jimf
Master Smack Fu Yak Hacker
2875 Posts |
Posted - 2010-04-13 : 14:19:15
|
Very interesting. My method also ran surprisingly fast, and with fewer reads. I had to tweak the heck out of it since I didn't consider the situation where the total can fall back under the cap. For payments that's never the case, once the money's spent it's spent, but I also need to apply this to incurred losses -- how much we think we will have to pay -- and this can go back under the cap, for example, we think we have to pay out 150K, bu then the case settles for 75K. Here is my convulution for a dataset of 30,000 records. It runs in 500 ms and and about 500 reads. I put a clustered index on a temp table over policynumber,claimnumber,ayq,cyq.I'm not convinced it's going to work in all case yet, there always seems to be another quirk in the data to deal with.I'm still trying to wrap my head around yours and Visakh's solutions. I'm more of a Neanderthal programmer!Thanks againJim select t2.policynumber,t2.claimnumber , t2.ayq,t2.cyq ,t2.paid,t2.totpaid,t2.incurred,t2.totincurred ,[IncrementalPaid] = CASE WHEN CASE WHEN @cap - t2.totpaid < 0 THEN @cap - (t2.TotPaid - t2.paid) ELSE t2.paid END > 0 THEN CASE WHEN @cap - t2.totpaid < 0 THEN @cap - (t2.TotPaid - t2.paid) ELSE t2.paid END ELSE 0 END ,[IncrementalIncurred] = CASE WHEN t2.totincurred < @cap and t2.incurred < 0 then t2.incurred else CASE WHEN -- when you're over the cap CASE WHEN @cap - t2.totincurred < 0 THEN @cap - (t2.totincurred - t2.incurred) ELSE t2.incurred END > 0 THEN CASE WHEN @cap - t2.totincurred < 0 THEN @cap - (t2.totincurred - t2.incurred) ELSE t2.incurred END ELSE --If you went back under the cap due to take down CASE WHEN t2.totincurred < @cap and t2.incurred < 0 THEN t2.totincurred - @cap ELSE 0 END END end from ( select t1.policynumber,t1.claimnumber ,t1.ayq,t1.cyq, t1.paid ,t1.incurred--,t1.incexp,t1.expenses ,[TotPaid] = sum(t2.paid) ,[TotIncurred] = sum(t2.incurred) from #t1 t1 inner join #t1 t2 on t1.policynumber = t2.policynumber and t1.claimnumber = t2.claimnumber and t1.ayq = t2.ayq and t1.cyq >= t2.cyq -- where t1.policynumber = 'aka000002' and t1.claimnumber = 214125 group by t1.policynumber,t1.claimnumber, t1.ayq,t1.cyq, t1.paid,t1.incurred ) t2 order by policynumber,claimnumber,cyqEND Everyday I learn something that somebody else already knew |
 |
|
|
Lamprey
Master Smack Fu Yak Hacker
4614 Posts |
|
|
SwePeso
Patron Saint of Lost Yaks
30421 Posts |
|
|
|
|
|
|
|