List ReconcileWhy identical-looking values do not match in two lists
List Reconcile ·
When identical-looking values do not match, inspect the actual characters in one example before cleaning the whole dataset. A trailing space, a different dash, or an embedded line break can be hard to see. An exact comparison treats those characters as part of the value. It also treats an article title followed by a URL as different from the title alone.
In a Reddit discussion about comparing research article lists, a user described Scopus and Medline exports with different formatting. They reported that a comparison found every entry unique despite substantial expected overlap, and that language models gave inconsistent answers. A reply suggested comparing DOI identifiers. The thread does not establish the exact cause of those mismatches. You will need to inspect a sample pair to diagnose your own files.
Reduce the problem to one expected match
Choose one value you can confidently identify in both original sources. Paste just that pair into Compare two lists, one value on each side. Start with the matching options off. If the report shows two distinct values, there is a character difference under the tool’s rules. List order is irrelevant for this membership comparison, so sorting both files will not repair that difference.
Open the source rows for each value and enable Show whitespace as JSON escapes. This makes tabs and line breaks visible as escaped characters and places quotes around strings. It can also help you notice a space before the closing quote. This view does not identify every invisible Unicode character by name; some still require closer inspection in a suitable text editor.
Separate four common causes
| A | B | Exact result | Useful check |
|---|---|---|---|
ABC-7 | abc-7 | Different | Is case meaningful? |
00123 | 00123 | Different | Is the final space accidental? |
North West | North West | Different | Count the internal spaces |
123 | 00123 | Different | Confirm the ID convention |
Ignore case makes the first pair share a comparison key. Ignore outer whitespace makes the second pair share a key. Neither option merges the third pair, because the extra space is inside the value. Neither removes leading zeros in the fourth pair. Those boundaries prevent a broad cleanup setting from silently changing every kind of identifier.
Ask whether the change is allowed for the actual data. Case may be irrelevant for a staff display name but meaningful for a technical identifier. Removing leading zeros might repair a known padded code in one workflow while merging separate IDs in another. A successful match after cleanup is evidence of the chosen rule, not proof that the records describe the same entity.
Unicode can look the same and still differ
Some accented text has more than one character representation. An accented letter can be stored as one character or as a base letter followed by a combining mark. These representations may look identical on screen. MDN’s explanation of Unicode normalization shows how normalization can bring certain equivalent representations to a common form.
List Reconcile does not apply Unicode normalization automatically. Its case option lowercases comparison text; its whitespace option trims the edges. It does not fold every accent, replace all punctuation, or map visually similar characters across alphabets. A Cyrillic letter and a Latin letter can look similar without being the same identifier character.
If Unicode normalization is part of your source’s agreed rules, apply it in a controlled preparation step and retain the originals. Document which normalization form was used. Do not replace arbitrary unfamiliar characters until a pair happens to match; that can hide a data issue or create a false match that is harder to discover later.
Check the field you are actually comparing
An exported line may contain a title, year, and URL even if it looks mostly like a title. Comparing the entire line against a title-only export will classify the values as different. For structured exports, open the complete CSV and select the title or identifier column explicitly. Review the first five records to confirm that the separator was interpreted correctly.
Quoted multiline fields need the appropriate CSV or TSV mode. In one-item-per-line mode, each physical line is an item. A title containing an embedded newline will therefore become more than one item if pasted in that mode. Use the structured file when a single record legitimately spans lines.
Also check the Header row setting. A mismatching column label at the top of the results may simply mean a header was treated as data. The different-column-order guide walks through selecting corresponding fields without rearranging the files.
Prefer an identifier when the task allows it
For research records, a shared DOI can be a better comparison key than a title that varies in capitalization, punctuation, or subtitle formatting. This only works for records with the relevant identifier present and prepared consistently. A bare identifier and the same identifier inside a URL remain different strings unless you deliberately standardize their representation.
The tool does not fetch DOI metadata, discover missing identifiers, or decide that two similar titles describe the same paper. Keep records without a usable key in a separate review queue rather than treating all blanks as one matched item. In other workflows, the equivalent choice might be an employee ID, stock code, or source-system record reference.
Rerun the small pair before the full list
Change one preparation step or matching option at a time. Once the small pair behaves as expected, add an intentionally different value as a negative example. For instance, confirm that trimming a trailing space fixes 00123 while 00124 remains separate. This guards against a cleanup rule that makes everything appear to match.
Then rerun the complete inputs using the same rules and inspect several differences. Download JSON if you need the original field strings and source references for debugging, or Excel for a spreadsheet review. Keep a note of the settings with the report. If the keys now match but their frequencies still differ, investigate duplicate counts rather than continuing to clean text that is already consistent.
The text-ID example gives you a small starting check for leading zeros and long identifiers. Use the downloadable CSVs to test the same values through your own import workflow.