Skip to content

Lowercase non-ASCII words in words() - #95

Merged
martinsoenen merged 1 commit into
xefi:mainfrom
alies-dev:fix/multibyte-words
Oct 19, 2026
Merged

martinsoenen merged 1 commit into
xefi:mainfrom
alies-dev:fix/multibyte-words

Conversation

@alies-dev

@alies-dev alies-dev commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

words() runs the result through strtolower(), which only knows ASCII. With a Cyrillic or accented locale you get things like "Прадукцыя" or "Écrit" back with the capital still there.

This switches to mb_strtolower(), with the same mbstring check and fallback that LowercaseModifier already uses. Added a test with Cyrillic, French and German words.

Came up while writing the be_BY locale (#94).

Summary by CodeRabbit

  • Bug Fixes
    • Text word processing now removes punctuation before converting text to lowercase.
    • Uppercase non-ASCII characters are now converted correctly when the required text support is available. If it isn’t available, the system issues a warning and uses standard lowercase conversion instead.

strtolower() only touches ASCII letters, so words like "Ёлка" or "Über"
kept their capital. Use mb_strtolower() when it's available, same as
LowercaseModifier does.
@coderabbitai

coderabbitai Bot commented Oct 7, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 48369102-26ed-4ea4-83d4-dd3c3ce9c59a
📥 Commits

Reviewing files that changed from the base of the PR and between 1c78653 and f0190cc.

📒 Files selected for processing (2)
  • src/Extensions/TextExtension.php
  • tests/Unit/Extensions/TextExtensionTest.php

Included review availability: This review used your included allowance. Your plan provides up to 2 included reviews per hour; 1 remain after this review.


📝 Walkthrough

Walkthrough

TextExtension::words() removes periods and commas before lowercasing. It uses mb_strtolower() when available and otherwise warns before using strtolower(). A unit test checks lowercase results for non-ASCII words.

Changes

Word normalization

Layer / File(s) Summary
Word normalization and validation
src/Extensions/TextExtension.php, tests/Unit/Extensions/TextExtensionTest.php
words() removes periods and commas before lowercasing. It uses mb_strtolower() when available and warns before falling back to strtolower(). The test checks lowercase output for Ёлка, Écrit, and Über..

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~8 minutes

Change: Bug fix

Merge Risk: ⚪ Minimal · up to f0190

The Unicode test is compatible with the supported test setup. No issue identified here prevents merging after normal checks.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 5 functions across 2 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: lowercasing non-ASCII words in words().
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
  • Fix all pre-merge checks with AI
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@martinsoenen

Copy link
Copy Markdown
Contributor

Nice fix, thanks for the contribution !

@martinsoenen
martinsoenen merged commit 3bda135 into xefi:main Oct 19, 2026
8 checks passed
@alies-dev
alies-dev deleted the fix/multibyte-words branch October 20, 2026 00:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants