Identifying Duplicates

Thank you Petyer and J. I think the Fuzzy Lookup will do the trick. Hopefully it works with Win 11 and MS365, but we'll see. Moving to a new PC right now, but I can't wait to try it. Thanks to all who replied.

@Richard Hellenbrecht
Have you tried Microsoft's Fuzzy Lookup Add-In for Excel? See
https://www.microsoft.com/en-us/download/details.aspx?id=15011

Richard Hellenbrecht
I would try to merge the files and work from there. If the fields do not agree in order and type you may be able to Get & Transform (Data tab) to order the columns but I'm not honestly sure. Then sort the data on Company names and that might be sufficient for you to identify duplicates.

I've written a couple of macros including the ISLIKE UDF that might help. You will not need the last name function
Sub Compare()
Dim r As Range, c2f As String, c As Range, _
nr As Long, counter As Long, i As Long, _
bIsLike As Boolean, s As String
Set r = Selection
nr = r.Rows.Count
For i = 1 To nr
counter = 0
c2f = "*" & LastName(r(i)) & "*"
For Each c In r
s = c
bIsLike = ISLIKE(s, c2f)
If bIsLike Then
counter = counter + 1
If counter > 1 Then c.Offset(0, 4) = "Duplicate"
End If
Next c
Next i
End Sub
Function ISLIKE(text As String, Pattern As String) As Boolean
If text Like Pattern Then ISLIKE = True _
Else ISLIKE = False
End Function

(see Figure 1 below)

Re: Identifying Duplicates

Peter Atherton. I should have been more clear. I am not working with just first and last names. I am merging three separate database downloads into one file of about 16,000 records. Neither of these files have strong entry controls. They contain company names, not persons' names. A period, or lack of period in initial; spaces between initials, or no space; ", Inc." or just "Inc." cannot be queried.

The same company is interpreted differently under these circumstances. I'm looking for a "fuzzy" duplicate finder. After finding exact matches, about 3,000, I try phone numbers, then last names to weed out more, but its grueling. Ideas?

Stephanie & Tomek
Any large number can only be shown in Excel as text, so the range must be pre-formatted as text. Writing a UDF does seem to work but if you want to have a list this macro will do it.
Sub Incre()
Dim startNumber, s1 As String, s2 As Long
Dim i As Long
startNumber = Range("a1")
s1 = Left(startNumber, 8)
s2 = Right(startNumber, Len(startNumber) - 8)
For i = 1 To 10
'Debug.Print s1 & s2 + i
Cells(i + 1, 1) = s1 & s2 + i
Next i
End Sub

@Stephanie:
It seems that Excel cannot handle numbers with more than 15 significant digits. I you enter the *number* 1234567891234567, excel will change it to 1234567891234560 truncating digits after 15th.

If you enter that number into a cell formatted as text it will keep all digits. If you use such entry in an arithmetic calculation it will be truncated, e.g. if you have 1234567891234567 in the Cell A1 and in the cell A2 enter =A1+1 you will get 1234567891234560. (see Figure 1 below)

Note that in the picture below the cell 2 is formatted explicitly as number with 1 decimal, otherwise you may see 1.23457E+15, which hides the exact content.

Similar thing happens when you use conditional formatting for duplicates: even though the content of the cell is text, Excel sees that it looks as a number and treats it as a number hence truncating it to 15 significant digits. This happens even if you enter the number with a leading apostrophe to force it to be text and keep all digits.

Allen's first helper-column approach works for the situation you described, so this may be your best option. The second helper column approach does not work, as it uses a COUNTIF function and this triggers the "looks as a number - is a number" Excel logic.

The conditional formatting tip rocks! Thank you, Allen!!!!

You may want to add an exception to this statement..."Be aware that the conditional formatting approach highlights all duplicates within your data, whereas the helper column approaches mentioned earlier flag only the second and subsequent occurrences of the data you are checking. Also, the conditional formatting approach checks only the first 255 characters of each cell."

I consistently have numerical values flagged as duplicates using the conditional formatting approach because the first 15 characters are the same.

For example, these values all flag as duplicate even though the 16th character is different
1234567891234567
1234567891234568
1234567891234569

However, I did test the above values by replacing one character with a letter and Excel removed the "duplication" formatting. I believe the 255 character statement is accurate for cells containing alphabet characters.

If you know your data set has long numeric strings only, you may want to seriously consider a helper column, even if you only use one to investigate the duplicates identified by the conditional formatting.

I work with data sets that are thousands of lines long and often have this 15-character issue in the serial number column. I definitely use a helper column on the smaller subset of formatted "duplicates" to avoid the workbook from bogging down with non-essential formula calculations.

Richard Hellenbrecht

A little more robust method than the lastName function would be to Add the first initial to the last name. But it would still fail with two brothers name William and Walter or a marrid couple named Chas and Cheryl.

Function CheckName(ByVal ref) As String
Dim p As Integer
p = InStrRev(ref, " ") + 1
CheckName = Mid(ref, 1, 1) & " " & Mid(ref, p, Len(ref))
End Function

Entered as =CheckName(A1)

(see Figure 1 below)

Richard Hellenbrecht
For poorly entered data as shown you need a helper column showing just the last names, then you can use Micky's formula. Here is a UDF for the last names.
Function LastName(ByVal ref) As String
Dim p As Integer
p = InStrRev(ref, " ") + 1
LastName = Mid(ref, p, Len(ref))
End Function

If you have not used UDFs before, right-click the sheet tab, select view code, Select Insert Module, paste in the code, & press Alt + Q to rteturn to sheet.
(see Figure 1 below)

Duplicate identifier is very helpful for exact duplicates. But what about near-duplicates, such as H.R. Johnson vs HR Johnson or H. R. Johnson? Is there a function in Excel to do that?

As for the asked question - as per my opinion - there is no nee for a Helper-Column NOR a build-in Conditional-Formatting layout.

The use of C.F. leaning on the formula =COUNTIF(C$2:C2,C2)>1 is more than enough.