Skip to content

uucore: treat empty locale env vars as unset (fixes #13964) - #14186

Open
MadeNavaneeth wants to merge 2 commits into
uutils:mainfrom
MadeNavaneeth:fix/ls-non-ascii-locale
Open

uucore: treat empty locale env vars as unset (fixes #13964)#14186
MadeNavaneeth wants to merge 2 commits into
uutils:mainfrom
MadeNavaneeth:fix/ls-non-ascii-locale

Conversation

@MadeNavaneeth

Copy link
Copy Markdown
Contributor

Fixes #13964

Problem

When LC_ALL is set to empty string (which POSIX says means "use the next variable in the cascade"), get_locale_from_env parsed it as the C/POSIX locale with ASCII encoding. This caused ls to escape non-ASCII filenames into octal sequences even when LC_CTYPE or LANG specified UTF-8.

Fix

Treat empty environment variables the same as unset in get_locale_from_env by filtering out empty values with .filter(|v| !v.is_empty()).

Verification

Before (with LC_ALL=, LANG=uk_UA.UTF-8, LC_CTYPE=uk_UA.UTF-8):

$ ls
'\303\266 \303\246 \303\251 latin umlauts UTF-8'
\320\272\320\270\321\200\320\270\320\273\321\226\321\207\320\275\320\260 ...

After:

$ ls
ö æ é latin umlauts UTF-8
кирилічна назва в UTF-8

This matches GNU ls behavior.

Empty LC_ALL (or LC_CTYPE, LANG) means 'use the next variable in the
cascade', per POSIX.  Previously get_locale_from_env parsed the empty
string as the C/POSIX locale with ASCII encoding, which caused ls to
escape non-ASCII filenames even when LC_CTYPE or LANG specified UTF-8.

Fixes uutils#13964
@sylvestre

Copy link
Copy Markdown
Contributor

sorry but it needs a test

@github-actions

github-actions Bot commented Aug 27, 2026

Copy link
Copy Markdown

GNU testsuite comparison:

Skip an intermittent issue tests/tail/tail-n0f (fails in this run but passes in the 'main' branch)
Skipping an intermittent issue tests/date/date-locale-hour (passes in this run but fails in the 'main' branch)
Congrats! The gnu test tests/seq/seq-epipe is no longer failing!

@codspeed-hq

codspeed-hq Bot commented Aug 27, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 3 untouched benchmarks
⏩ 409 skipped benchmarks1


Comparing MadeNavaneeth:fix/ls-non-ascii-locale (9307cb6) with main (30de660)

Open in CodSpeed

Footnotes

  1. 409 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

let locale_var = ["LC_ALL", locale_name, "LANG"]
.iter()
.find_map(|&key| std::env::var(key).ok());
.find_map(|&key| std::env::var(key).ok().filter(|v| !v.is_empty()));

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
.find_map(|&key| std::env::var(key).ok().filter(|v| !v.is_empty()));
.find_map(|&key| std::env::var(key).ok())
.filter(|v| !v.is_empty());

this is a bit clearer?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The filter needs to stay inside find_map rather than after it — moving it outside would change the semantics:

In that case, we may have incorrect behavior here:

fn is_c_locale() -> bool {
["LC_ALL", "LC_CTYPE", "LANG"]
.iter()
.find_map(|&key| std::env::var_os(key))
.filter(|v| !v.is_empty())
.is_none_or(|v| v == "C" || v == "POSIX")
}

@MadeNavaneeth

Copy link
Copy Markdown
Contributor Author

Thanks for the suggestion! The filter needs to stay inside find_map rather than after it — moving it outside would change the semantics:

// Current (correct): skips empty values, continues to next env var
.find_map(|&key| std::env::var(key).ok().filter(|v| !v.is_empty()))

// Suggested: would match `LC_ALL=""` as the first value, then drop it,
// returning None (POSIX default) instead of falling through to LANG
.find_map(|&key| std::env::var(key).ok())
.filter(|v| !v.is_empty());

The filter inside the closure ensures LC_ALL="" is treated as "not set" and the cascade continues to LC_CTYPE/LANG — which is the whole point of the fix.

Added a test (test_empty_lc_all_falls_through_to_lang) to lock in this behavior.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ls on Linux displays non-ASCII characters as escape sequences when LC_ALL is not set

3 participants