Phil Factor SQL Speed Phreak Competition: No 2
This competition is now over, but the winner, Dave, got an Amazon Voucher for $60, and the privilege of being able to display the 'Phil Factor SQL Speed Phreak' award on their own siteIt was quite a struggle with some close competition from many of those who participated in this competition. However, Dave came up with a clever solution that produced an FIFO calculation from a million rows in a just a few seconds. The third Phil Factor Speed Phreak competition will soon be held on here. Watch out! (here is the original preamble.) This competition is to calculate current items in stock and current stock value in FIFO order. I have seen many different algorithms to do this and most of them involve a cursor.
Can it be done more quickly without, or even with, a cursor? In other words, what is the fastest way in SQL Server (any version) to provide this stock inventory report? Beware that the sample file includes the two extra columns; CurrentItems (INT) and CurrentValue (MONEY) ! The table is in this form (simplified from the way we'd do it in a real system of course).
You are welcome to change the two nonclustered indexes to suit your solution. You can download the complete sample data here. I have an idea of my own of the way to do this but I don’t know if it is the fastest. Explanation of FIFO rules (example, abbreviated)
a) First IN add 738 items (each $245.94) to the stock, for a total of $181,503.72 As mentioned before, the CurrentItems and CurrentValue columns in the sample data are only included for you to validate your routines. Here are some guidelines for your entries:1) Include a header in your suggestion. Make sure your name and date is present. I will allow you to use an existing tally number table (make sure it starts with 0). You can use any kind of object for this competition, except SQLCLR. If you are using a fixed tally number table, it has to be named dbo.TallyNumbers and the column named Number. The winner will be amongst the tied fastest entrants (generally there is a group of these) and it will be the one with the highest number of votes. We'll announce the winner in three week's time on 16th November. For a starter, here is a common cursor based solution that you will find in production in many places.
The above is already done. Here is the code to produce the final resultset.
Good luck to you all! Peter Larsson
(comments are locked)
|
|
/* Dave Ballantyne - 20091027 / / Ver 1a */ Please follow the guidelines when naming your suggestions. Is this a better edition 1? or a new edition?
Oct 27 '09 at 09:45 AM
Peso
@Peso : Slighty more streamlined than before. Dont think it'll get anywhere near Barrys best at the moment
Oct 27 '09 at 10:14 AM
dave ballantyne
(comments are locked)
|
|
For anyone having trouble loading the data file, here's the XML for a format file and a BULK INSERT statement that uses it. 1) Save this XML as fifo_format.xml:
2) Change the paths for the data file and format file as needed and execute this statement: Nice collateral benefit to following the challenge. Thanks!
Oct 27 '09 at 06:43 PM
KenJ
(comments are locked)
|
|
Dave Ballantyne ver 2.b -- Had a bit of a think and a eureka moment --- --- 2.a Had a bug with 0 valued articles not being shown Excellent. I just had to change "on TallyNumbers.Number <= Items" to "on TallyNumbers.Number between 1 and Items" because the TallyNumbers table starts with 0.
Oct 28 '09 at 08:50 AM
Peso
(comments are locked)
|
|
Essentially the same but added some comments , tidied it up , added an index Now, this baby is flying!
Oct 28 '09 at 02:21 PM
Peso
2.873 seconds on my box, correct results too - excellent!
Oct 28 '09 at 03:03 PM
Matt Whitfield ♦♦
+1: Wow! Got the StockID subsumes TranDate trick to work, I see.
Oct 28 '09 at 03:26 PM
RBarryYoung
A little note however. The sample system depicted in the competition DOES have StockID in same order as TranDate. If your system doesn't follow same order, this will return wrong value! If your system allows to register future planned INs this will not work. For example you know you are getting 500 items next wednesday and you register them already today with wednesday's date.
Oct 28 '09 at 05:12 PM
Peso
I'd suggest you change all "StockID" to "TranDate". It will in fact run faster. With StockID you average at 2.0 seconds and uses 438k reads. With TranDate you average at 1.8 seconds and uses 243k reads.
Oct 28 '09 at 05:24 PM
Peso
(comments are locked)
|
|
Here will the preliminary timings be presented. I write preliminary since Phil is (will be) away for a few days, so I will do the preliminary timings on a similar machine as Phil has. Current standingsBarry's timings on his solution varies due to a lot of tempdb space. But it is still a great suggestion.
Oct 26 '09 at 08:51 AM
Peso
What configuration are you testing on, Peso? I would think that my solution #2 would do a lot better than that on a multi-core environment. (Or is MAXDOP set down?)
Oct 26 '09 at 08:58 PM
RBarryYoung
It's probably my tempDB going through the roof. I'll try the suggestions on a better tempdb configuration. Your queries generates a lot of activity on the tempdbs.
Oct 27 '09 at 09:44 AM
Peso
(comments are locked)
|
« previous 1 2 3 4 5 6 next page »



Was there any particular reason you disallowed CLR functionality?
On an initial look, this would be the ideal scenario in which to use a CLR type...
Two things had us made this decision: 1) We have tried CLR before and there was no performance gain. 2) To test and verify, poster need to disclode the source code for the suggestion. But if you have a CLR routine that runs on 15 seconds or less, I think Phil will be interested in testing it.
Questions: 1) Is the Stock Transaction List referred to the one that will be used for Final Evaluation? If not, can you describe the differences? 2) What is the configuration that entries will be (finally) evaluated on?
Yes, the downloadable file is the data against all suggestions will be measured. Phil will drop or rename the two last columns so that no suggestion can use them. I guess Phil will be using same test harness as last time with 'Subscription List' competition.
Those filtered indexes will only work on SQL Server 2008, right?