Repository navigation
corrupted special characters in csv file #76
Description
Issue with improper character encoding handling during file generation or import. The system should correctly preserve all special characters (e.g., German, Scandinavian accents) using UTF‑8 or relevant encoding.
w3schools_dbo_Products.csv
Impact:
Incorrect product names displayed in downstream processes, leading to data quality issues and potential functional inconsistencies.
Expected:
All the values should retain their original special characters without corruption.
Activity
jonaraphael commentedon Mar 1, 2026
Thanks so much for opening this issue and for the clear report.
I will reproduce this first, then implement the smallest safe fix, add/update tests, and share a PR for your review.
jonaraphael commentedon Mar 1, 2026
Quick triage update: this is friendly and in scope, but fixing encoding issues safely requires the source encoding details.\n\nPlease share the original file encoding (for example UTF-8/Windows-1252/ISO-8859-1), plus exact open/save steps and whether was used. With that, I can reproduce and propose a targeted fix.
jonaraphael commentedon Mar 1, 2026
Correction to my previous comment: please confirm whether the command 'CSV: Change File Encoding' was used during your test steps.
ChandraSaiKarthik commentedon Mar 13, 2026
The file was unchanged.
Even you can find the products list in the below URL with query "select * from products;" and click "run sql" button.
https://www.w3schools.com/sql/trysql.asp?filename=trysql_select_all
jonaraphael commentedon Oct 17, 2026
You had already supplied a useful sample, and I should have followed up on it sooner. Thank you for also sharing the original source. The investigation found an encoding mismatch; #115 proposes clearer guidance for reopening it correctly. I’m sorry for the delay and grateful you took the time to help.
jonaraphael commentedon Oct 17, 2026
Thank you for the file you already provided. It contains non-UTF-8 bytes; in that sample, CP437 and CP850 produce the same accented names. The update adds clearer encoding guidance and preserves edits when a reopen is cancelled, but it does not guess or repair encodings automatically. I am keeping this open until the original fixture has been checked through the native reopen-and-save workflow. Let me know if it works for you!
Audit note: Partial: encoding diagnosis; native fixture confirmation pending. Remaining work: Verify the already supplied fixture through native Reopen with Encoding using a correct source codec, then edit/save and compare accented values and output bytes. No new sample is required to start that check.