THE LEDGERSPROUT FIELD GUIDE

How to remove duplicates from an inventory CSV without losing data

Choose inventory keys, evaluate conflicting rows and retain the right occurrence while preserving an audit copy.

Updated October 20, 2026 · LedgerSprout field guide

Back up the file and define a duplicate

An inventory CSV is often one row per SKU per location, not one row per SKU. Removing rows by SKU alone could erase an entire warehouse. Keep the original export unchanged, then write down the business rule: for example, one record for every SKU + warehouse pair. If lots or serial numbers matter, include those fields too. The deduplicator keeps one record; it does not add stock quantities or combine records.

Separate formatting differences from business differences

Two keys such as “A-10” and “ a-10 ” match only if trimming and case-insensitive matching are enabled. These options normalize the matching key, not the exported values. Internal spaces, punctuation and accents remain significant. Before enabling normalization, check whether the source system treats differently cased identifiers as distinct. A convenient cleanup rule can be wrong for a case-sensitive identifier scheme.

First and last are positions, not timestamps

If duplicate rows contain quantities 4 and 7, choosing keep first retains the first quantity and discards the other. Keep last retains the last occurrence in file order. Neither option interprets an updated_at field. If the latest record should win, first sort in a trusted spreadsheet or source system by the required date and tie-break rule, then retain the appropriate occurrence. If quantities should be summed, use an aggregation workflow instead of deduplication.

Handle blank keys before bulk removal

Several rows with blank SKU values form a duplicate group when SKU is the only selected key. This can discard unrelated records that merely share missing information. Fill missing identifiers in the source, add enough identifying columns, or separate unresolved rows for manual review. LedgerSprout explicitly treats empty key values as equal and shows the kept/removed totals. It cannot infer a missing product identity.

Verify the output with a small reconciliation

Start with a sample where you know the expected duplicate groups. Confirm which original rows survive and check the total kept plus removed equals the input record count. Review a few retained quantities and locations, not just the row count. The preview shows the first 50 retained rows; download the full output for reconciliation. Spreadsheet-safe export may prefix formula-like cells with an apostrophe, so keep the original file as your untouched audit copy. Only import the cleaned result into a live inventory system after checking that system’s required schema.