urlkit[ TIDY / [1×] ]
Duplicate line remover
Keep the first copy of each line and remove repeated entries.
02 / Your result
Your result will appear here.
Start with your own input or try an example.
What it does
Clean repeated entries from a pasted list while keeping its original order. Decide whether comparison should respect case, ignore surrounding whitespace, or remove blank lines. The first retained occurrence keeps its spelling and spacing so you can review a small, understandable change before copying the result.
Useful when
- Remove repeated links from a reading list without visiting them.
- Tidy repeated tags, labels, or reference numbers from an export.
- Combine overlapping plain-text lists while retaining the first occurrence.
How to use it
- Paste a list with one entry per line.
- Set case, whitespace-comparison, and blank-line options to match your data.
- Remove duplicates, review the removal counts, and copy or download the remaining list.
A few useful details
Which duplicate is kept?
The first matching line is retained in its original position. Later matching lines are removed. The tool does not sort the list or rewrite the retained text, and it reports duplicate removals separately from optional blank-line removals.
How do case and whitespace options affect matching?
By default, matching is exact and case-sensitive. Ignoring surrounding whitespace trims only the comparison key, not the output. Case-insensitive mode compares Unicode lowercase forms; it does not normalize accents or apply full language-specific case folding.
Can this recognize two different URLs for the same page?
No. It compares line text and never opens URLs or resolves redirects. Different paths, tracking parameters, or Unicode normalization forms remain different unless the selected text comparison makes them identical. Limits are 10,000 lines and 100,000 UTF-16 code units; output uses LF line endings.