Please Note: This article is written for users of the following Microsoft Excel versions: 2007 and 2010. If you are using an earlier version (Excel 2003 or earlier), this tip may not work for you. For a version of this tip written specifically for earlier versions of Excel, click here: Checking for Duplicate Rows Based on a Range of Columns.

Checking for Duplicate Rows Based on a Range of Columns

by Allen Wyatt
(last updated September 29, 2014)

8

Jennifer has a lot of data in a worksheet, and she considers some of the rows to be duplicates. She determines whether a row is a duplicate based upon whether a range of columns in one row is identical to the same range of columns in the previous row. For instance, if all of the values in F7:AB7 are identical to the values in F6:AB6, the Jennifer would consider row 7 to be a duplicate of row 6. She wonders if there is a way that she can easily check for such duplicate rows and highlight the duplicates in some manner.

One approach to this problem is to utilize the conditional formatting capabilities of Excel. If your data is in rows A1:AZ100, then select the range You could then use the following as a formulaic test within your conditional format:

=IF(AND($F2:$AB2=$F1:$AB1),1,0)=1

If your conditional format applies a color to the cells, then you'll see the color appear anytime the values in columns F through AB are equal to the values in the same columns of the row directly above the one that is colored.

If Jennifer's data consists only of cells in the columns F:AB, then she can use the filtering capabilities of Excel to mark the duplicate rows. Here are the general steps:

  1. Select the cells containing your data. For instance, if the first row contains column headers, you should select F2:AB100.
  2. Apply a color to those cells, so that all the data is shaded some unique color.
  3. Select any cell in data table.
  4. Display the Data tab of the ribbon.
  5. Click Advanced in the Sort & Filter group. Excel displays the Advanced Filter dialog box. (See Figure 1.)
  6. Figure 1. The Advanced Filter dialog box.

  7. Make sure that Filter the List, In Place is selected. (It should be selected by default.)
  8. Click the Unique Records Only check box.
  9. Click OK. Excel collapses your data so that only unique (non-duplicate) records are shown.
  10. Select the visible rows of data.
  11. Remove the color you applied in step 2.
  12. Remove the filter you applied in steps 5 through 8.

At this point, only the duplicate records are highlighted with the color you used in step 2. These records can be safely deleted, leaving only the unique records.

Perhaps an even easier approach is to allow Excel to determine the duplicates and remove the rows. Follow these steps:

  1. Select the rows you want to check. For instance, you might select rows 1 through 100.
  2. Display the Data tab of the ribbon.
  3. Click the Remove Duplicates tool in the Data Tools group. Excel displays Remove Duplicates dialog box. (See Figure 2.)
  4. Figure 2. The Remove Duplicates dialog box.

  5. Make sure that only those column headers that represent columns F:AB are selected. The check boxes for all other columns should be cleared.
  6. Click OK. Excel shows you a confirmation message indicating how many records were checked and how many non-duplicates were kept.

The powerful feature of using the Remove Duplicates tool is that the order of the records in the table don't really matter. In other words, the tool doesn't just compare the specified columns in one row to the row above it—it compares the specified columns in one row to all the other rows and it keeps only those that are unique.

ExcelTips is your source for cost-effective Microsoft Excel training. This tip (10608) applies to Microsoft Excel 2007 and 2010. You can find a version of this tip for the older menu interface of Excel here: Checking for Duplicate Rows Based on a Range of Columns.

Author Bio

Allen Wyatt

With more than 50 non-fiction books and numerous magazine articles to his credit, Allen Wyatt is an internationally recognized author. He  is president of Sharon Parq Associates, a computer and publishing services company. ...

MORE FROM ALLEN

Basing Headers and Footers on the Previous Section

Word treats the headers and footers in a document independently, based on the section in which they appear. This means that ...

Discover More

Hiding the Taskbar when It is Not in Use

Don't like the Taskbar visible on the screen? You can easily hide it when you aren't using it by making just one small ...

Discover More

AutoFill Won't Work as Expected

AutoFill is a great tool for filling cells with data. What do you do when it doesn't work like you expect, however? Here's ...

Discover More

Program Successfully in Excel! John Walkenbach's name is synonymous with excellence in deciphering complex technical topics. With this comprehensive guide, "Mr. Spreadsheet" shows how to maximize your Excel experience using professional spreadsheet application development tips from his own personal bookshelf. Check out Excel 2013 Power Programming with VBA today!

More ExcelTips (ribbon)

Counting Cells Containing a Formula

Cells in a worksheet can contain different types of information, such as numbers, text, and formulas. If you want to ...

Discover More

Looking Backward through a Data Table

Sometimes you need to look backward, through the information above your formula, to find the data you need. This can be done ...

Discover More

Deriving High and Low Non-Zero Values

When analyzing your numeric data, you may need to figure out the largest and smallest numbers in a set of values. If you ...

Discover More
Subscribe

FREE SERVICE: Get tips like this every week in ExcelTips, a free productivity newsletter. Enter your address and click "Subscribe."

View most recent newsletter.

Comments

If you would like to add an image to your comment (not an avatar, but an image to help in making the point of your comment), include the characters [{fig}] in your comment text. You’ll be prompted to upload your image when you submit the comment. Maximum image size is 6Mpixels. Images larger than 600px wide or 1000px tall will be reduced. Up to three images may be included in a comment. All images are subject to review. Commenting privileges may be curtailed if inappropriate images are posted.

What is seven less than 7?

2015-06-17 09:44:01

Brandon

Hi, I tried this formula and cannot adjust it to do what I need it to do. Is it possible to say IF a value in this column repeats and has the same value in another column then highlight the cell or put true in third column?
=IF(($B2:$B$30000AND$F2:$F$30000),1,0)=1
I have 300,000 records so values in (F) repeat but I am only looking for the ones that repeat on the same account# (B)


2014-12-15 06:34:32

jim

I am trying to write a conditional format for duplicate values. I use 2010 Excel to create a biweekly employee roster. There are times I enter duplicate position assignments and don't catch the error immediately.

I have been trying to write a dynamic rule that as I enter the data if I enter a duplicate it will notify me with a distinct color.

Any help would be appreciated.


2014-04-04 18:21:08

Chuck Trese

@Bryan, ahhh, mia culpa! Yeah, it seems it needs that AND() to keep Excel from trying to re-interpret the equation as text. I did test it before I posted, so now I'm not sure how i got it working - or tricked myself into thinking it was working.


2014-04-04 06:54:22

Bryan

@Chuck: Actually, that's not working for me. It's not comparing each element of the first array to the respective element in the second array; instead, it only highlights the rows when the *first* elements match.


2014-04-03 10:32:13

Neil

We have found that the Remove Duplicates feature keeps the top row of a set of duplicates, so if you have something like a list of chemicals with various daily quantities and you want to remove duplicate chemical listings but keep the largest daily use, sort by use largest to smallest and then Remove Duplicates for chemical name.


2014-04-03 08:58:49

Chuck Trese

Actually, for the conditional format formula, you only need:
=$F2:$AB2=$F1:$AB1
Personally, I would add parentheses, just to make it easier to understand what is going on:
=($F2:$AB2=$F1:$AB1)


2014-04-03 08:35:38

Bryan

AND already returns TRUE/FALSE, so you can simplify your expression to =AND($F2:$AB2=$F1:$AB1). Even if it didn't, IF lets you return TRUE/FALSE, so you could have just said =IF(AND($F2:$AB2=$F1:$AB1),TRUE,FALSE). There's no reason to ever use the IF(something,1,0)=1 construct.


2013-04-02 09:20:26

Ron Mytko

This might work also:

• highlight the rows (or range) to check,

• click function key F5 (Go To),

• click the Special button at bottom left,

• select Row differences,

• any duplicate rows are highlighted.


This Site

Got a version of Excel that uses the ribbon interface (Excel 2007 or later)? This site is for you! If you use an earlier version of Excel, visit our ExcelTips site focusing on the menu interface.

Newest Tips
Subscribe

FREE SERVICE: Get tips like this every week in ExcelTips, a free productivity newsletter. Enter your address and click "Subscribe."

(Your e-mail address is not shared with anyone, ever.)

View the most recent newsletter.