How to Remove Duplicate Lines From a List (Emails, Keywords, Names)

The short answer

Put one item per line, paste the list into the Remove Duplicate Lines tool, choose whether "Apple" and "apple" count as the same, and click Remove Duplicates. The tool keeps the first copy of each line in its original position, removes the repeats and any blank lines, and tells you how many lines it removed. Copy the result or download it as a .txt file.

When this is useful

• Email lists merged from two sign-up forms, where the same address appears twice.
• Keyword lists collected from several research tools.
• Attendance or name lists pasted from different sheets.
• URLs or product codes you need to check one by one.
• Log lines where you only want each distinct message once.

The rule is always the same: one item per line. If your list is separated by commas, put each item on its own line first.

Step by step

Open Remove Duplicate Lines.

Paste your list into the box, one item per line.

Set the three options (explained below). By default, Case sensitive and Trim whitespace are ticked and Sort result is not.

Click Remove Duplicates.

Read the summary, for example "3 duplicates removed · 4 unique lines".

Click Copy result, or Download .txt to save a file named deduplicated.txt.

If you edit the input after processing, the old result disappears so you never copy an out-of-date list. Click the button again.

The three options, with examples

Here is one test list, including a line with a trailing space and an empty line:

• apple
• Banana
• apple (with a space after it)
• (empty line)
• banana
• Cherry
• apple

These results were produced by the tool's own code.

Settings	Result	Removed
Case sensitive on, Trim on (default)	apple, Banana, banana, Cherry	3
Case sensitive off, Trim on	apple, Banana, Cherry	4
Case sensitive on, Trim off	apple, Banana, apple (with space), banana, Cherry	2

Case sensitive

• On: "Banana" and "banana" are different lines, so both stay.
• Off: they count as the same line. The tool keeps the first spelling it saw ("Banana") and drops the later one.

Turn it off for email addresses, website domains and most keyword lists, where capital letters do not change the meaning. In this list:

• a@x.com
• b@y.com
• A@x.com
• c@z.com (with a space in front)
• b@y.com

with case sensitivity off and trim on, the result is a@x.com, b@y.com and c@z.com, with 2 lines removed.

Leave it on for things like passwords, product codes or file names on systems where capitals matter.

Trim whitespace

• On: spaces and tabs at the start and end of each line are removed before comparing, so "apple" and "apple " are treated as duplicates. The output lines are trimmed too, which cleans up your list.
• Off: lines are compared exactly as they are. Invisible trailing spaces from spreadsheets or PDFs will make lines look identical but still count as different.

Keep trim on unless leading spaces matter, for example in indented code.

Sort result

• Off: lines stay in the order they first appeared. Use this when order matters, such as a priority list or a sequence of steps.
• On: the unique lines are sorted A to Z after duplicates are removed.

For more sorting choices, such as Z to A, by length, numeric order or a random shuffle, use Sort Text Lines on the result.

Blank lines are always removed

Empty lines are dropped whatever the settings, and they are included in the "removed" count. In the test list above, the default result removed 3 lines: the second "apple", the "apple " with a space, and the empty line. Remember this when the numbers do not add up the way you expect.

Removing duplicates from an Excel or Google Sheets column

Select the cells in one column and copy them. Each cell becomes one line when pasted.

Paste into the tool and run it.

Copy the result and paste it back into a new column.

Spreadsheets also have built-in duplicate removal (Excel has a Remove Duplicates command on the Data tab). The online tool is handy when the data is not in a spreadsheet, when you want to check the counts quickly, or when you are on a device without the spreadsheet app.

Note: this works on one column at a time. If your rows have several columns (name, email, phone) and you want to remove rows where only the email is repeated, do that in the spreadsheet itself.

Troubleshooting

Two lines look the same but both stay

The usual causes:

• Different capital letters with Case sensitive on. Untick it.
• Extra spaces with Trim off. Tick Trim whitespace.
• Spaces in the middle, such as "New Delhi" (two spaces) vs "New Delhi". Trim only removes spaces at the ends. Clean these with Find and Replace by replacing two spaces with one.
• Look-alike characters, such as a non-breaking space copied from a web page, or a curly apostrophe vs a straight one. Retype the character or use Find and Replace.

Everything ended up on one line

Your list used commas or semicolons, not line breaks, so the whole list was one line. Put each item on its own line and run it again.

I need to see what was removed, not just the count

Compare the original and the result with the Text Diff Checker. Lines that were removed are highlighted.

Privacy

The list is processed by the page in your browser; the deduplicate function runs locally when you click the button. That makes it reasonable for internal lists, but as with any tool, avoid pasting passwords or sensitive personal data you do not need to process.

Related tools

• Count how many items are left with the Word Counter.
• Change capitalisation across the whole list with the Case Converter.