x

The ‘FIFO Stock Inventory’ SQL Problem

Phil Factor SQL Speed Phreak Competition: No 2

alt text

This competition is now over, but the winner, Dave, got an Amazon Voucher for $60, and the privilege of being able to display the 'Phil Factor SQL Speed Phreak' award on their own site

It was quite a struggle with some close competition from many of those who participated in this competition. However, Dave came up with a clever solution that produced an FIFO calculation from a million rows in a just a few seconds. The third Phil Factor Speed Phreak competition will soon be held on here. Watch out!


(here is the original preamble.)

This competition is to calculate current items in stock and current stock value in FIFO order.
I'll tell you the business rules for the algorithm and provide a sample cursor-based routine that generates the correct result, but slowly (about 40 minutes).

I have seen many different algorithms to do this and most of them involve a cursor. Can it be done more quickly without, or even with, a cursor? In other words, what is the fastest way in SQL Server (any version) to provide this stock inventory report?
It is a reasonable request. We have a stock transaction list (1,000,001 records, 17.6MB zipped) with 15,002 unique articles and we need to do a stock inventory report that gives the current breakdown of the ArticleID, the number of items in stock and the current stock value according to FIFO rules. The list should be in ArticleID order. The sample stock transaction list will include two running totals so you can check the results of your routine. They are NOT part of the competition, and you are not allowed to use them in your calculation.

Beware that the sample file includes the two extra columns; CurrentItems (INT) and CurrentValue (MONEY) !
If you want a smaller subset to test on, import all records and remove all articles but two or three. These are the samples for which all suggestions will be measured with.

The table is in this form (simplified from the way we'd do it in a real system of course).
In this made-up but realistic problem, you have an existing table and database design which you can’t do anything about. (we can all relate to that!) You have an existing cursor-based solution that is taking several minutes to run. (Yes, we all relate to that). The database is performing badly, and you need to take effective steps to remedy this. The only thing you can do is to come up with a better-performing routine. Redesigning the database isn’t an option very often, in the real world, because this requires team sign-in. This is a competition based on the real, sometimes imperfect, world: not an exposition of database design. The point is that we are faced with designs like this and we have to heal them. The competition is all about making stuff go faster.

CREATE TABLE    dbo.Stock
                (
                    StockID INT IDENTITY(1, 1) NOT NULL,
                    ArticleID SMALLINT NOT NULL,
                    TranDate DATETIME NOT NULL,
                    TranCode VARCHAR(3) NOT NULL,
                    Items INT NOT NULL,
                    Price MONEY NULL,
                    CONSTRAINT [PK_Stock] PRIMARY KEY CLUSTERED 
                    (
                        StockID ASC
                    )
                )  

CREATE NONCLUSTERED INDEX IX_Input ON dbo.Stock (TranCode, ArticleID)
--INCLUDE (TranDate, Items, Price) -- Remove comment for SQL Server 2005 and later
--WHERE TranCode IN ('IN', 'RET')   -- Remove comment for SQL Server 2008

CREATE NONCLUSTERED INDEX IX_Output ON dbo.Stock (TranCode, ArticleID)
-- INCLUDE (TranDate, Items) -- Remove comment for SQL Server 2005 and later
--WHERE TranCode = 'OUT'  -- Remove comment for SQL Server 2008

You are welcome to change the two nonclustered indexes to suit your solution. You can download the complete sample data here. I have an idea of my own of the way to do this but I don’t know if it is the fastest.

Explanation of FIFO rules (example, abbreviated)

StockID ArticleID TranDate TranCode Items  Price CurrentItems CurrentValue
   4567     10000 10:45:07 IN         738 245.94          738   181,503.72
  21628     10000 12:05:25 OUT        600                 138    33,939.72
  22571     10000 14:39:27 IN          62 199.95          200    46,336.62
  30263     10000 16:14:13 OUT        165                  35     6,998.25
  42090     10000 18:18:58 RET          5                  40     7,998.00
  58143     10000 20:18:54 IN         500 135.91          540    75,953.00

a) First IN add 738 items (each $245.94) to the stock, for a total of $181,503.72
b) Then we take out 600 items (each 245.94) from the stock, leaving a total of $33,939.72
c) Then we insert 62 items (each 199.95) to the stock, for a total of $46,336.62
d) Then we take out 165 items (138 each 245.94 and 27 each 199.95), leaving a total of $6,998.25
e) Then we return 5 items. We can’t track at which price we took them out; so all returns are priced at the price of the latest ones inserted before the return. Even if there should be items left for the price of 245.94, the returned items are valued for 199.95. After the return, the current stock value is $7,998.00
f) The final insert adds $67,995.00 to the stock value, for a total of $75,953.00

As mentioned before, the CurrentItems and CurrentValue columns in the sample data are only included for you to validate your routines.

Here are some guidelines for your entries:

1) Include a header in your suggestion. Make sure your name and date is present.
2) Include an edition number. First edition is 1. If you later improve your current suggestion post it again as version 2. Example: “Peso 1” and if improved, “Peso 1b”, “Peso 1c” etc.
3) If you are trying a new algorithm, change the edition to “Peso 2”. If you improve this algorithm, change the version to “Peso 2b”, “Peso 2c” etc. This will save Phil hours of work in the test harness!
4) If you create a temp table, make sure you delete it in the script.
5) Keep the order of columns in output as ArticleID, CurrentItems, CurrentValue

I will allow you to use an existing tally number table (make sure it starts with 0). You can use any kind of object for this competition, except SQLCLR. If you are using a fixed tally number table, it has to be named dbo.TallyNumbers and the column named Number.
The time taken for their creation will not be included in the timings. The time measured is the “Main” script/procedure. If you want to call sub-procedures, go ahead.

The winner will be amongst the tied fastest entrants (generally there is a group of these) and it will be the one with the highest number of votes. We'll announce the winner in three week's time on 16th November.

For a starter, here is a common cursor based solution that you will find in production in many places.

CREATE TABLE    #Work
                (
                    RowID INT IDENTITY(1, 1) PRIMARY KEY CLUSTERED,
                    Price MONEY
                )  

DECLARE @ArticleID INT = 1,
        @PrevID INT = 0,
        @TranCode VARCHAR(3),
        @Items INT,
        @Price MONEY,
        @Loop INT = 0,
        @StockID INT,
        @LatestPrice MONEY,
        @Total INT = (SELECT COUNT(*) FROM dbo.Stock)

DECLARE curYak CURSOR FORWARD_ONLY FOR
                SELECT      ArticleID,
                            TranCode,
                            Items,
                            Price,
                            StockID
                FROM        dbo.Stock
                ORDER BY    ArticleID,
                            TranDate

OPEN    curYak

FETCH   NEXT
FROM    curYak
INTO    @ArticleID,
        @TranCode,
        @Items,
        @Price,
        @StockID

WHILE @@FETCH_STATUS = 0
    BEGIN
        IF @ArticleID > @PrevID
            BEGIN
                TRUNCATE TABLE  #Work

                SET @LatestPrice = NULL
            END

        IF @TranCode = 'IN'
            BEGIN
                INSERT  #Work
                        (
                            Price
                        )
                SELECT  @Price
                FROM    dbo.TallyNumbers
                WHERE   Number < @Items

                SET     @LatestPrice = @Price
            END

        IF @TranCode = 'RET'
            INSERT  #Work
                    (
                        Price
                    )
            SELECT  @LatestPrice
            FROM    dbo.TallyNumbers
            WHERE   Number < @Items

        IF @TranCode = 'OUT'
            DELETE  w
            FROM    (
                        SELECT      TOP(@Items)
                                    RowID
                        FROM        #Work
                        ORDER BY    RowID
                    ) AS w

        UPDATE      s
        SET         s.CurrentItems = w.CurrentItems,
                    s.CurrentValue = COALESCE(w.CurrentValue, 0)
        FROM        dbo.Stock AS s
        INNER JOIN  (
                        SELECT  COUNT(*) AS CurrentItems,
                                SUM(Price) AS CurrentValue
                        FROM    #Work
                    ) AS w ON s.StockID = @StockID

        SELECT  @PrevID = @ArticleID,
                @Loop += 1

        IF @Loop % 1000 = 0
            RAISERROR('Now updating record %d of %d.', 10, 1, @Loop, @Total) WITH NOWAIT

        FETCH   NEXT
        FROM    curYak
        INTO    @ArticleID,
                @TranCode,
                @Items,
                @Price,
                @StockID
    END

DROP TABLE  #Work

CLOSE       curYak
DEALLOCATE  curYak

The above is already done. Here is the code to produce the final resultset.

SELECT      ArticleID,
            CurrentItems,
            CurrentValue
FROM        (
                SELECT  ArticleID,
                        CurrentItems,
                        CurrentValue,
                        ROW_NUMBER() OVER (PARTITION BY ArticleID ORDER BY TranDate DESC) AS recID
                FROM    dbo.Stock
            ) AS d
WHERE       recID = 1
ORDER BY    ArticleID

Good luck to you all!

Peter Larsson

more ▼

asked Oct 23, 2009 at 03:56 PM in Default

Peso gravatar image

Peso
1.6k 5 6 8

Was there any particular reason you disallowed CLR functionality?

On an initial look, this would be the ideal scenario in which to use a CLR type...
Oct 23, 2009 at 04:34 PM Peso
Two things had us made this decision: 1) We have tried CLR before and there was no performance gain. 2) To test and verify, poster need to disclode the source code for the suggestion. But if you have a CLR routine that runs on 15 seconds or less, I think Phil will be interested in testing it.
Oct 23, 2009 at 05:08 PM Peso
Questions: 1) Is the Stock Transaction List referred to the one that will be used for Final Evaluation? If not, can you describe the differences? 2) What is the configuration that entries will be (finally) evaluated on?
Oct 23, 2009 at 05:09 PM RBarryYoung
Yes, the downloadable file is the data against all suggestions will be measured. Phil will drop or rename the two last columns so that no suggestion can use them. I guess Phil will be using same test harness as last time with 'Subscription List' competition.
Oct 23, 2009 at 05:16 PM Peso
Those filtered indexes will only work on SQL Server 2008, right?
Oct 23, 2009 at 05:46 PM RBarryYoung
(comments are locked)
10|1200 characters needed characters left

28 answers: sort oldest

Dave Ballantyne ver 2.b -- Had a bit of a think and a eureka moment --- --- 2.a Had a bug with 0 valued articles not being shown

drop table tallynumbers go Create Table tallynumbers( number integer ) go insert into tallynumbers select top 100000 row_number() over(order by (select 1)) from syscolumns a cross join syscolumns b go create unique clustered index idxTally on tallynumbers(number) go

with cteStockList as ( select ArticleId, ItemsIN=case when TranCode in('In','ret') then Items else 0 end, ItemsOUT=case when TranCode in('Out') then Items else 0 end from stock -- where TranCode in('IN','RET') --where ArticleId in( 10128) ), cteStockSum as ( Select ArticleID,SUM(ItemsIn) as TotalItemsIn,SUM(ItemsOUT) as TotalItemsOut,SUM(ItemsIn) - SUM(ItemsOUT) as TotalStock from cteStockList group by ArticleID ) , CteStockLastItem as ( Select cteStockSum.ArticleID, TotalItemsIn, TotalItemsOut ,TotalStock, LastIn.StockID , SumItems from cteStockSum cross apply (Select top 1 Stock.ArticleId, Stock.StockID, SUM(InnerStock.Items) as SumItems from stock join stock innerStock on innerStock.ArticleID = stock.ArticleID and InnerStock.StockId >= stock.StockId where innerStock.TranCode in('In','Ret') and Stock.TranCode in('In','Ret') and stock.ArticleID = cteStockSum.ArticleID group by Stock.ArticleId,Stock.StockID having SUM(InnerStock.Items)>=cteStockSum.TotalStock order by Stock.ArticleId,Stock.StockID desc ) as LastIn ) , cteTallyList
as ( select CteStockLastItem.*,stock.StockId as CalcStockId,case when TotalStock = 0 then NULL else ROW_NUMBER() over (Partition By stock.ArticleId order by stock.StockId desc) end as Rown from CteStockLastItem join stock on stock.ArticleId = CteStockLastItem.ArticleID and stock.StockId >= CteStockLastItem.StockID left join TallyNumbers on TallyNumbers.Number <= Items and TotalStock > 0 where TranCode in ('In','Ret') ) , ctePreCalc as ( select ArticleID, TotalItemsIn, TotalItemsOut ,TotalStock,CalcStockId,sum(case when Rown is not null then 1 else 0 end) as CountStock From cteTallyList where Rown<=cteTallyList.TotalStock or Rown is null group by ArticleID, TotalItemsIn, TotalItemsOut ,TotalStock,CalcStockId )

Select ctePreCalc.ArticleID, ctePreCalc.TotalStock as CurrentItems ,sum(Price*CountStock) as CurrentValue from ctePreCalc cross apply ( select top 1 Price from stock where stock.StockId <= ctePreCalc.CalcStockId and stock.ArticleId = ctePreCalc.ArticleId and TranCode='IN' order by StockId desc ) as Price group by ctePreCalc.ArticleID, ctePreCalc.TotalItemsIn, ctePreCalc.TotalItemsOut ,ctePreCalc.TotalStock order by ctePreCalc.ArticleID
more ▼

answered Oct 28, 2009 at 07:48 AM

dave ballantyne gravatar image

dave ballantyne
928 1 1 4

Excellent. I just had to change "on TallyNumbers.Number <= Items" to "on TallyNumbers.Number between 1 and Items" because the TallyNumbers table starts with 0.
Oct 28, 2009 at 08:50 AM Peso
(comments are locked)
10|1200 characters needed characters left

Now the query should be correct:

-- Andriy Z. 1 // still version 1, because first one was incorrect
create nonclustered index [IX_StockIdx] on [dbo].[Stock]([ArticleID] asc ,[TranDate] asc)
include([TranCode] ,[Items]) 

select si.ArticleID
      ,CurrentItems = sum(itemNums.val)
      ,CurrentValue = sum(si.Price * itemNums.val) 
  from Stock si
       outer apply (
           select sum(I)
             from (select case when sPrev.TranCode = 'IN' and sPrev.TranDate < si.TranDate then sPrev.Items
                               when sPrev.TranCode = 'IN' and sPrev.TranDate > si.TranDate then 0
                               when sPrev.TranCode = 'RET' then sPrev.Items  
                               when sPrev.TranCode = 'OUT' then -1 * sPrev.Items 
                          end
                     from Stock sPrev
                    where sPrev.ArticleID = si.ArticleID
                      and sPrev.TranDate <> si.TranDate
                  )Ax(i)              
       )prevB(val)
       cross apply (
           select case when (si.Items + isnull(prevB.val, 0) ) > 0 
                       then (si.Items + isnull(prevB.val, 0) )
                       else 0
                  end
       )itemNums(val)
 where si.TranCode = 'IN' 
 group by si.ArticleID 
 order by si.ArticleID

Regards, Andriy Z.

more ▼

answered Oct 28, 2009 at 10:22 AM

Andriy Zabavskyy gravatar image

Andriy Zabavskyy
11 1

Andriy - still not quite... For article 10005 your query gives 1069 items @ 133818.95, whereas the reference query gives 1065 items @ 133316.75... Performance was 21.678 seconds on my box...
Oct 28, 2009 at 10:38 AM Matt Whitfield ♦♦
This is suggestion 1c?
Oct 28, 2009 at 12:25 PM Peso
(comments are locked)
10|1200 characters needed characters left

Essentially the same but added some comments , tidied it up , added an index

/* Dave Ballantyne - Phil Factor Challenge Entry 3.b*/ /* “A problem well stated is a problem half solved” - Charles F. Kettering 

Let us think about what has been asked within the challenge.

We have a warehouse which starts empty , the data contains the stock movements in (including returns) and out. The stock movements ALWAYS happen in a first in first out basis. With this we know that if the present stock level (sum(in's) - sum(out's) is 50 that will be the last 50 that have entered the warehouse. The key to this query is to efficiently find the cost of those 50 items. */ go Drop index Stock.IxStockDave go Create NonClustered index IxStockDave on stock(ArticleID,StockId,TranCode) include(Items) -- Not entirely neccesary but will shave of some time. go

with cteStockList as ( select ArticleId, StockId, ItemsIN=case when TranCode in('In','ret') then items else 0 end, ItemsOUT=case when TranCode ='Out' then Items else 0 end from stock ), cteStockSum /* Sum up the ins and outs to calculate the remaining stock level / as ( Select ArticleID, SUM(ItemsIn) as TotalItemsIn, SUM(ItemsOUT) as TotalItemsOut, SUM(ItemsIn) - SUM(ItemsOUT) as TotalStock from cteStockList group by ArticleID ) , cteReverseInSum / Perform a rolling balance ( in reverse order ) through the stock movements in / as ( Select ArticleId, StockId, (Select SUM(Items) from Stock InnerStock where InnerStock.StockId >= Stock.StockId and InnerStock.TranCode in('In','Ret') and InnerStock.ArticleID = Stock.ArticleID) as RollingStock, stock.items as ThisStock from stock where TranCode in('In','Ret') ) , / Using the rolling balance above find the first stock movement in that meets (or exceeds) our required stock level */ cteWithLastStockId as ( select cteStockSum.ArticleId, cteStockSum.TotalStock, LastPartialStock.StockId, LastPartialStock.UseThisStock from cteStockSum cross apply ( Select Top 1 cteReverseInSum.StockId, cteReverseInSum.ThisStock as StockToUse, cteReverseInSum.RollingStock as RunningTotal, TotalStock - (cteReverseInSum.RollingStock -ThisStock ) as UseThisStock from cteReverseInSum where cteStockSum.ArticleId = cteReverseInSum.ArticleId
and cteStockSum.TotalStock <= cteReverseInSum.RollingStock order by cteReverseInSum.StockId desc )
as LastPartialStock

) /* Sum up the cost of 100% of the stock movements in after the returned stockid and for that stockid we need 'UseThisStock' items' / select cteWithLastStockId.ArticleId, cteWithLastStockId.TotalStock as CurrentItems , sum(case when stock.stockId = cteWithLastStockId.StockId then cteWithLastStockId.UseThisStock else stock.Items end * Price.Price) as CurrentValue from cteWithLastStockId join stock on stock.ArticleId = cteWithLastStockId.ArticleId and stock.StockId >= cteWithLastStockId.StockID and stock.TranCode in('In','Ret') / Find the Price of the item in */ cross apply(Select Top 1 innerstock.Price from stock innerstock where innerstock.ArticleId = stock.ArticleID and innerstock.StockId <= stock.StockId and innerstock.TranCode ='In' order by innerstock.StockId Desc) as Price

group by cteWithLastStockId.ArticleId, cteWithLastStockId.TotalStock order by cteWithLastStockId.ArticleId
more ▼

answered Oct 28, 2009 at 11:12 AM

dave ballantyne gravatar image

dave ballantyne
928 1 1 4

Now, this baby is flying!
Oct 28, 2009 at 02:21 PM Peso
2.873 seconds on my box, correct results too - excellent!
Oct 28, 2009 at 03:03 PM Matt Whitfield ♦♦
+1: Wow! Got the StockID subsumes TranDate trick to work, I see.
Oct 28, 2009 at 03:26 PM RBarryYoung
A little note however. The sample system depicted in the competition DOES have StockID in same order as TranDate. If your system doesn't follow same order, this will return wrong value! If your system allows to register future planned INs this will not work. For example you know you are getting 500 items next wednesday and you register them already today with wednesday's date.
Oct 28, 2009 at 05:12 PM Peso
I'd suggest you change all "StockID" to "TranDate". It will in fact run faster. With StockID you average at 2.0 seconds and uses 438k reads. With TranDate you average at 1.8 seconds and uses 243k reads.
Oct 28, 2009 at 05:24 PM Peso
(comments are locked)
10|1200 characters needed characters left

## Herman, 30 Oct 2009 ##

Hi there, I decided to give it a try since I solved this problem for the stock market once before, using a slow cursor solution, and I always wanted to write it as a Set based solution.

This is my first attempt, and I am happy I got the answers right (I think), but I did not optimize it yet. It ran 86 secs on my laptop. I know I should how to do it different, but anyway decided to send this version.

--For my solution I need to create a function to determine the minimum of 4 values IF (SELECT OBJECT_ID('dbo.Min4')  ) IS NOT NULL  DROP FUNCTION dbo.Min4 GO

CREATE FUNCTION dbo.Min4(@V1 money, @V2 money, @V3 money, @V4 money) RETURNS Money AS BEGIN DECLARE @Min Money SET @Min = CASE WHEN @V1<@V2 THEN CASE WHEN @V1<@V3 THEN CASE WHEN @V1<@V4 THEN @V1 ELSE @V4 END ELSE CASE WHEN @V3<@V4 THEN @V3 ELSE @V4 END END ELSE CASE WHEN @V2<@V3 THEN CASE WHEN @V2<@V4 THEN @V2 ELSE @V4 END ELSE CASE WHEN @V3<@V4 THEN @V3 ELSE @V4 END END END

RETURN @Min END GO

WITH -- First generate a transaction order per aticle, the last transaction has TransOrder=1 OrderedTransactions AS (SELECT ROW_NUMBER() OVER(PARTITION BY ArticleID ORDER BY TranDate DESC) as TranOrder, StockID,ArticleID, TranDate, TranCode, Items, Price FROM dbo.Stock), --Calculate running totals and determine for every transaction the last date a price was defined for the article RunningTotals AS (SELECT Tr.*,Tot.* FROM OrderedTransactions as Tr CROSS APPLY (select SUM(case when trancode = 'OUT' then -Items else Items end) as CurrentItems, SUM(case when trancode <> 'OUT' then Items else 0 end) as CurrentInItems, MAX(case trancode when 'IN' then TranDate else CAST(0 as datetime) end) as LastPriceDate from OrderedTransactions as PrevTr where Tr.TranDate >= PrevTr.TranDate and Tr.ArticleID = PrevTr.ArticleID group by ArticleId ) as Tot), --Use LastPriceDate to determine the price of The 'RET' transactions PricedTotals AS (SELECT Tr.TranOrder, Tr.ArticleID, Tr.TranDate, Tr.TranCode, Tr.Items, Tr.CurrentItems, Tr.CurrentInItems, case Tr.TranCode when 'IN' THEN Tr.Price when 'RET' THEN Pr.Price else 0 end as Price
FROM RunningTotals as Tr LEFT JOIN OrderedTransactions Pr on Tr.ArticleID = Pr.ArticleID and Tr.LastPriceDate = Pr.TranDate), --Now match the 'OUT' transactionjs with their FIFO 'INI' and 'RET' transactions MatchedTransactions as
(SELECT Tr.TranOrder, Tr.ArticleID, Tr.TranDate, Tr.TranCode, Tr.Price, Tr.Items, Tr.CurrentItems, IT.TranOrder AS InTranOrder, IT.TranDate as InTranDate, IT.TranCode as InTranCode, IT.Price AS InPrice, IT.Items as InItems, case when Tr.TranCode = 'OUT' then dbo.Min4(Tr.CurrentINItems-IT.CurrentINItems+IT.Items-Tr.CurrentItems, Tr.CurrentItems+Tr.Items - (Tr.CurrentINItems-IT.CurrentINItems), IT.Items, Tr.Items) end as Usage FROM PricedTotals as Tr LEFT JOIN PricedTotals as IT on Tr.TranCode = 'OUT' and IT.TranCode <> 'OUT' and Tr.TranDate > IT.TranDate and Tr.ArticleID = IT.ArticleID where ((Tr.CurrentINItems-IT.CurrentINItems+IT.Items>Tr.CurrentItems and Tr.CurrentINItems-IT.CurrentINItems<=Tr.CurrentItems+Tr.Items) OR Tr.TranCode <> 'OUT')) --And finally calculate the SUM of the Value of all transactions, and get CurrentItems from the last transaction. SELECT T.ArticleID, MAX(CASE WHEN T.TranOrder=1 THEN T.CurrentItems ELSE 0 END) as CurrentItems, SUM(CASE WHEN T.TranCode='OUT' THEN (-T.Usage*T.InPrice) ELSE (T.Items*T.Price) END) as CurrentValueValue FROM MatchedTransactions T GROUP BY T.ArticleID GO

DROP FUNCTION dbo.Min4
more ▼

answered Oct 30, 2009 at 11:05 AM

Herman gravatar image

Herman
11 1 1 2

It's a good attempt Herman. However, I need you to add an ORDER BY to your final SELECT.
Oct 30, 2009 at 02:26 PM Peso
Thanks for the commenrt, I did forget it. Anyway my solution was a first try and is calculating much more than it is displaying, so I wil post a new solution.
Nov 01, 2009 at 12:40 PM Herman
(comments are locked)
10|1200 characters needed characters left

Herman 1d, 9 nov 2009

I was calculating much more than I was displaying, so I reconsidered my first algorithm to get to the requested result.
By using some new indexes, and optimizing the code so that it gets the price for RET transaction only when required, I was able to get my solution to work even faster.
The only thing to do is to somehow stop the RunningTotals to be calculated for transactions which are not needed in the final result.
I guess that is why some of you get amazing fast results.


I did some attempts to stop the RunningTotals when enough In transacrtions have been processed to cover the stock, but without success. I would never come up with Dave's top 1 (LastStockID) solution.
I did however improve my solution by replacing the SUM() OVER .. with SUM() GROUP BY. I initially expected the new SUM() OVER to be as efficient as the GROUP BY, but it turned out that it better to calculate totals per article and not per transaction. Which sound very reasonable actually...

I am using the same indexes:

CREATE NONCLUSTERED INDEX [IX_GetPrice] on [dbo].[Stock]([ArticleID] asc, [TranCode] , [TranDate] asc)
  include([Price]) 
  WHERE (TranCode='IN') 
go

CREATE NONCLUSTERED INDEX [IX_Transactions] on [dbo].[Stock]([ArticleID] asc ,[TranDate] desc)
  include([TranCode],[Items],[Price]) 
go

CREATE NONCLUSTERED INDEX IX_InItems ON dbo.Stock (TranCode, ArticleID asc , TranDate asc )
  INCLUDE (Items) -- Remove comment for SQL Server 2005 and later
  WHERE TranCode IN ('IN', 'RET')   -- Remove comment for SQL Server 2008
GO

And here is my 1d solution

WITH Transactions AS  (SELECT ArticleID, TranDate, TranCode, Items, Price--,  FROM dbo.Stock),  SummedTrans AS --Calculate the totals for In and Out transactions  (SELECT ArticleID,  SUM(CASE WHEN TranCode ='OUT' THEN 0 ELSE Items END) AS TotalsInItems,  SUM(CASE WHEN TranCode ='OUT' THEN Items ELSE 0 END) AS TotalsOutItems  FROM Transactions  GROUP BY ArticleID),  RunningTotals AS -- Determine the corresponding In tranacions to be matched to an Out transaction  (SELECT Tr.*, S.TotalsInItems, S.TotalsOutItems,  CASE WHEN TranCode IN ('IN', 'RET')  THEN (SELECT SUM(NextTr.Items) FROM Transactions AS NextTr  WHERE Tr.TranDate <= NextTr.TranDate  and Tr.ArticleID = NextTr.ArticleID  AND NextTr.TranCode IN ('IN', 'RET')  GROUP BY NextTr.ArticleID  HAVING SUM(NextTr.Items)-Tr.Items <= S.TotalsInItems-S.TotalsOutItems)  END AS CurrentInItemsRev  FROM Transactions as Tr  inner join SummedTrans s  ON Tr.ArticleID = S.ArticleID),  MatchedTotals AS --Match the In/Out transactions for current Articles on stock  (SELECT *,  --CASE WHEN CurrentInItemsRev-Items <= TotalsInItems-TotalsOutItems  -- THEN  CASE WHEN CurrentInItemsRev < TotalsInItems-TotalsOutItems THEN Items  ELSE -(CurrentInItemsRev-Items)+TotalsInItems-TotalsOutItems  -- END  --ELSE 0  END Usage  FROM RunningTotals  WHERE (CurrentInItemsRev-Items <= TotalsInItems-TotalsOutItems)),  PricedTotals AS -- Determine the value of the In transacions  (SELECT *,  CASE WHEN Price IS NOT NULL THEN Price  ELSE (SELECT TOP 1 Price  FROM RunningTotals P  WHERE P.ArticleID = T.ArticleID  and P.TranCode ='IN'  AND P.TranDate < T.TranDate  ORDER BY P.TranDate DESC)  END * Usage AS Value
FROM MatchedTotals T WHERE TranCode IN ('IN', 'RET') )
--And Sum everything up SELECT ArticleID, max(TotalsInItems-TotalsOutItems) as CurrentItems, sum(Value) as CurrentValue FROM PricedTotals GROUP BY ArticleID ORDER BY ArticleID GO
more ▼

answered Nov 01, 2009 at 01:08 PM

Herman gravatar image

Herman
11 1 1 2

Thank you. I will try it later when I get clean access to my preliminary test machine.
Nov 02, 2009 at 10:38 AM Peso
(comments are locked)
10|1200 characters needed characters left
Your answer
toggle preview:

Up to 2 attachments (including images) can be used with a maximum of 524.3 kB each and 1.0 MB total.

New code box

There's a new way to format code on the site - the red speech bubble logo will automatically format T-SQL for you. The original code box is still there for XML, etc. More details here.

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

SQL Server Central

Need long-form SQL discussion? SQLserverCentral.com is the place.

Topics:

x985
x343
x8
x7
x5

asked: Oct 23, 2009 at 03:56 PM

Seen: 28628 times

Last Updated: Nov 14, 2009 at 01:55 PM