question

Henrik Staun Poulsen avatar image
Henrik Staun Poulsen asked

Estimate time to Create index

Is it possible to estimate how long time it will take to create an index? Is it possible to see the progress on index creation, as we can see the progrees when restoring a database (in percent (of type tinyint = no decimals))?
indexingtime
10 |1200

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

TimothyAWiseman avatar image
TimothyAWiseman answered
As Grant said, there is no way to get an estimate of how long an index will take to create. If you are inexperienced (forgive me if I am assuming, I have no way to know) and you just need a very rough rule of thumb, then I will say the only time I have ever seen it take more than 10 minutes to index a table involved a blocking process. As a general rule, unless there is a blocking process or the server is very overtaxed, an average index will take a couple of minutes or less to create.
7 comments
10 |1200

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Fatherjack avatar image Fatherjack ♦♦ commented ·
+1 - having thought about this I agree, I have never seen an index take longer than 10 minutes unless there have been blocking issues either. I generally schedule index work to occur out of hours to avoid such issues.
1 Like 1 ·
Håkan Winther avatar image Håkan Winther commented ·
I promise you, I have seen indexes that takes a lot longer then 10 minutes without blocking, but the tables have been 400 Gb. I really understand your concern Henrik! :)
1 Like 1 ·
Oleg avatar image Oleg commented ·
How about using TOAD to connect to Oracle which has whatever the equivalent of Sybase's "Lock All Pages" thingy. Any DML statement implicitly begins transaction, so if one dev issues something like **update the_table set col1 = col1 where key_col = 'some_value'** and then goes home forgetting to explicitly type and execute **rollback**, all others are hosed until the next day!!! Issuing create index under this seemingly innocent scenario would take a bit of time :( When I was trying to answer this question: http://ask.sqlservercentral.com/questions/7079/advise-on-deleting-huge-amount-of-rows Henrik gave me an excellent suggestion which proved to be very helpful when trying to sniff into some complex queries to figure the time taken by different steps:
declare @d varchar(19);
set @d = convert(varchar(19), getDate(), 120);
raiserror(@d, 10, 1) with nowait;

waitfor delay '00:00:10';

set @d = convert(varchar(19), getDate(), 120);
raiserror(@d, 10, 1) with nowait;
waitfor delay '00:00:10';
select 'All Done'
go
The above prints activity in **real time**. Substitute raiserror with select or print and one is doomed to wait entire 20 seconds and then get all prints at once :(
0 Likes 0 ·
Henrik Staun Poulsen avatar image Henrik Staun Poulsen commented ·
@TimothyAWiseman; well, our 7 biggest tables are between 1 and 70 billion rows, taking up a TB each. A covering index can easily take 300 GB. This takes a bit more than 10 minutes to complete. I was thinking more along the line of something like this: SELECT * FROM sys.dm_os_tasks where session_id=68
0 Likes 0 ·
TimothyAWiseman avatar image TimothyAWiseman commented ·
@Henrik, yes as Hakan also pointed out, with very large databases it is quite possible to have a backup legitimately take more than 10 minutes, especially if you are deliberately creating a covering index which will have several columns. But, if someone relatively new to SQL were asking me how long it takes to make an index, I think it would be safe to say: "It depends on a lot of factors, but for a very general idea they will normally only take a couple of minutes and if it goes over 10 minutes you probably will want to look for blocking processes." It gives them an order of scale so they know it will rarely be instant but also should not take hours (for tables of average size.) Of course, I also would never recommend someone relatively new to SQL add indexes to a table that was several hundred GB large.
0 Likes 0 ·
Show more comments
Grant Fritchey avatar image
Grant Fritchey answered
First question, not really, no. The amount of memory, amount of data involved, contention with other resources, blocking on the table, all this and more are going to affect how long it takes to create an index. There's no way at all that I know of to estimate how long it could take. A very small index could take minutes and a very large index could take seconds, just depending on everything that is going on in the system. The second question, you can turn on STATS when running a restore. You can specify the percentage that you want to display, either 1 or 10 and it will update with those stats in a series of messages. It's not terribly accurate, but it will do what you want.
5 comments
10 |1200

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Grant Fritchey avatar image Grant Fritchey ♦♦ commented ·
I get those questions. I just look to see if there is blocking. If not, I say it'll be done when it's done.
1 Like 1 ·
Henrik Staun Poulsen avatar image Henrik Staun Poulsen commented ·
yes, I know about stats=x, but I trying to figure out if there was anything of the like when creating indexes.
0 Likes 0 ·
Grant Fritchey avatar image Grant Fritchey ♦♦ commented ·
Nope. Sorry. I suspect it's not possible for SQL Server itself to know because of what can happen with blocking & locking, resource use, etc.,etc. ad infinitum.
0 Likes 0 ·
Fatherjack avatar image Fatherjack ♦♦ commented ·
+1 - agreed. If the rebuild in online then the progress would depend on other activity in the database so it would be impossible to tell
0 Likes 0 ·
Henrik Staun Poulsen avatar image Henrik Staun Poulsen commented ·
I know that it is impossible to accurately estimate the finish time.

But it ought to be possible to see how far the SQL Server is right now. As you can see from my reply to Timothy, I've been playing with sys.dm_os_tasks.

I probably out to rephrase my question along the lines of:
what do you do when your manager asks when it will be finished?
Do you have a small notebook with notes on the last round of index creation?
0 Likes 0 ·
Henrik Staun Poulsen avatar image
Henrik Staun Poulsen answered
Grzegorz Popczyk pointed me towards a solution of this problem. Well, it solves 50% of the problem, as the solution only works for online index generation. But 50% is a lot more than 0 %. At ms-help://MS.SQLCC.v10/MS.SQLSVR.v10.en/s10de_4deptrbl/html/491616c1-f666-4b16-a5ea-1192bf156692.htm we can read about a Progress Report: Online Index Operation Event Class. So the solution to my problem:
run a trace,
include Progress Report: Online Index Operation and ...

you are able to see how far the job is. Update: I use Sp_whoIsActive to see if there is any blocking at the moment. Brilliant Sproc!
10 |1200

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Write an Answer

Hint: Notify or tag a user in this post by typing @username.

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.