Removing duplicate rows

Duplicate rows inflate counts, skew averages, and quietly distort every dashboard built on top of them — but only when they are genuine mistakes. Dotwave gives you a precise, conservative way to find and remove exact duplicates in Step 2, plus a separate tool for the near-matches that exact matching cannot catch. This article explains what Dotwave treats as a duplicate, how to check before you delete, and how to choose between the removal modes.

What counts as a duplicate

Dotwave defines a duplicate strictly: a row is a duplicate only when it is an exact whole-row match of another row — identical in every column, not just a few. If two rows agree on customer name and date but differ by a single cent in the amount, they are not duplicates; they are two distinct records. This whole-row rule is deliberately cautious. It guarantees Dotwave will never collapse two rows that differ in any meaningful field, so the duplicates it flags are always true byte-for-byte repeats and never legitimate records that merely resemble one another.

Because the match is exact, small inconsistencies defeat it. A trailing space, a different capitalization, or a slightly different spelling makes two otherwise-identical records count as distinct rows — which is exactly the situation fuzzy deduplication exists to handle.

Previewing before removing

Before you remove anything, preview the duplicates Dotwave has found. The preview lays the repeated rows out so you can see the actual records rather than trusting a count. This is the moment to confirm the duplicates are noise from a bad export and not real repeated events — a customer's second identical order, a recurring monthly charge, a genuine repeated survey response. Looking at the rows themselves is the safeguard that keeps a bulk removal from deleting data you needed.

Remove all versus review before removing

Dotwave gives you two ways to act on the duplicates it found, and the right one depends on how confident you are:

• Remove all duplicates — deletes every extra copy in one action, keeping the first occurrence of each repeated row and discarding the rest. This is the fast path for when the preview has convinced you the duplicates are all noise. It is efficient precisely because you have already checked; run it when you have no doubt.
• Review before removing — walks you through the duplicates so you can confirm the removal deliberately rather than in a single sweep. Choose this when the dataset is one where duplicates could plausibly be legitimate, or when you simply want a second look before committing. It trades a little speed for the certainty that nothing real is lost.

Both modes keep one representative of each duplicated row; the difference is only how much scrutiny you apply before the extras are dropped.

What fuzzy deduplication is and when to use each

Exact-match removal cannot see that London, london, and London all refer to the same city, because as strings they are not identical. Fuzzy deduplication closes that gap: it finds rows that are near-matches rather than exact copies — records that differ only by capitalization, stray whitespace, or minor spelling variation — and treats them as the same underlying entity. It lives in the guided cleaning dialog rather than the Step 2 table, because collapsing near-matches is a judgment call that benefits from reviewing the proposed groups.

Choose between the two by asking how the duplicates arose. If they are perfect copies from a doubled export or a repeated import, exact removal in Step 2 is the correct, safe tool. If they are the same records entered inconsistently by different people or systems, exact matching will miss them entirely and you need fuzzy deduplication. Many real cleanups use both in sequence: remove the exact duplicates first, then run fuzzy deduplication to catch the near-matches that remain.