Remove Duplicates

Question

I am trying to remove duplicates from a table. Not all the students have duplicates though. I will provide 2 scenarios below: Any sugestions? I am using this query but it just removes the duplicate ids. I need more logic based on status and year.

         delete
         FROM  ##temp_test
         WHERE ##temp_test.%%physloc%%
         NOT IN (SELECT MIN(##temp_test.%%physloc%%)
          FROM   ##temp_test 
          GROUP BY id)

The first scenario I would want the record that has STATUS = Final AND the most recent YEAR. (This is the 3rd record of id 1)

The second scenario since there IS NO Final I just will take the record with the most recent YEAR (This is the second record of id 2)

Scenario 1
ID NAME   CLASS Status YEAR
1   Joe   Math  Final   2010
1   Joe   Math  Initial 2010
1   Joe   Math  Final   2011
Scenario 2
2   Bob   Eng   Initial 2011
2   Bob   Eng   Initial  2010

Answer 1

This example shows how to use row_number() to rank based on various criteria. I partition each id, and then order descending by year first (as you stated it has precedence) and then order descending by a calculated value that is either 1 if the status is 'Final' or 0 otherwise.

I then choose the row number 1's from that data set and I've eliminated the dupes.

declare @temp_test table (
    id int,
    name varchar(10),
    Class varchar(10),
    status varchar(10),
    [Year] int)

insert into @temp_test select 1,'Joe','Math','Final',2010
insert into @temp_test select 1,'Joe','Math','Initial',2010
insert into @temp_test select 1,'Joe','Math','Final',2011
insert into @temp_test select 2,'Bob','Eng','Initial',2011
insert into @temp_test select 2,'Bob','Eng','Initial',2010
insert into @temp_test select 3,'Kev','Physics','Initial',2010
insert into @temp_test select 3,'Kev','Physics','Final',2010
insert into @temp_test select 3,'Kev','Physics','Initial',2011

select *
from (
    select 
     *, 
     rn=row_number()over(partition by id order by year desc, 
                   case when [status] = 'Final' then 1 else 0 end desc)
    from @temp_test
    )a
where a.rn = 1

Answer 2

0

You can also use this solution to delete the duplicates:

;WITH cte
AS(
SELECT *, ROW_NUMBER() OVER(PARTITION BY col1, col2 ORDER BY colN) AS rowcnt FROM yourtable
)
DELETE FROM cte 
/*
--remove this comment mark if you want the deleted records to end up in an archive table
OUTPUT DELETED.col1, DELETED.col2, DELETED.colN 
INTO myArchive(col1,col2,colN )
*/
WHERE rowcnt >1

Remove Duplicates

Follow this question

Related Questions