Every run looks like a new dataset
Every run looks like a fresh standalone file with no links to your existing records. Staff match rows by hand again before deciding whether to create a product or update an existing one.
Web scraping
Update scraped records only when the data has really changed
Each collection run creates another file, but staff can't safely spot changes or match them to your products. Formatting quirks and missing pages add misleading updates. We set reliable matching and comparison rules before collected values touch your records.
Discuss this problemHow We Help
We review the source IDs you are permitted to use and how they relate to the records in your own system. Titles may change, and an incomplete run can make an existing item look removed. Approved normalization (rules that clean up formatting) separates display differences from price or spec changes that matter to the business. Your publishing policy then separates verified updates from uncertain matches and missing items. Catalog owners can review important changes where appropriate. Our tests cover unchanged data, real differences and incomplete runs, so a refreshed file becomes useful evidence and not another manual matching job.
Signs This Is the Problem
Formatting creates false changes
Extra spaces and display differences show up as changes to information that has not changed. Reviewers spend time clearing updates that have no real effect on price, specs or the product link you maintain.
Missing items are treated as deletions
An item missing from a partial run gets treated as removed from the catalog. A scraping or access failure can trigger a destructive decision with no evidence that the supplier actually retired the item.
What We Investigate
Assess stable source IDs
We assess which permitted source IDs stay stable when titles or pages change. Your existing records get compared with those IDs, and ambiguous matches are set aside, not guessed from similar names.
Compare fields by what they mean
We compare fields by their actual commercial meaning. Formatting cleanup needs approved rules, so it removes harmless variation without erasing real differences in unit, quantity or spec.
Review missing items and coverage
For missing items, we examine evidence of what the run covered. An item may be absent because it was retired, access failed for a while or the run was incomplete. Those causes call for different decisions, not one automatic deletion rule.
What We Repair
Keep a maintained ID map
We maintain supported links between source and destination IDs and show you uncertain matches. Repeat collections can then update established records without treating each refreshed row as a new, unrelated product.
Apply field-by-field comparison rules
We set up approved comparisons for each field and show meaningful differences for review or publishing. Formatting-only variations follow your documented cleanup rules, while real price or spec changes keep their significance.
Separate updates from removals
Confirmed updates get handled apart from uncertain removals, under the review limits you approved. Items missing because of incomplete coverage stay pending until reliable evidence and an owner's decision support what happens next.
How We Verify the Fix
- A repeated, unchanged collection maps to existing records without creating new duplicates.
- A meaningful price or spec change appears in the reviewed change list.
- Formatting-only differences follow the agreed normalization and do not trigger false updates.
- A missing item from an incomplete run is not automatically treated as a confirmed deletion.
Questions About This Work
Is a product title a reliable ID?
Not by itself. A title can change or be shared by several products. We assess stable source references and keep uncertain matches for review. The matching rule must identify the item reliably enough to update its existing record without accidentally merging a similar product.
Should missing items be removed immediately?
Not without evidence and your business's approved policy. A failed or partial collection should not be read as the supplier retiring the item. We check coverage and flag uncertainty before letting a missing item remove or deactivate a maintained product.
Can staff approve changes before publishing?
Yes, a supported review step can let staff approve selected changes. We agree which ordinary updates are safe to apply and which price, spec or removal decisions need an owner. The review list should hold real differences, not noise from harmless formatting changes.
Tell Us What Is Happening
Tell us your collection and destination platforms. Share one item, with private details removed, that appears in two runs with an expected change. We can then inspect source identity, comparison rules and the link to your existing record.
Ready to Get Started?
Tell us about your project and we'll get back to you within 24 hours.