Skip to content

lexer: Update to Unicode 18 - #165289

Open
clubby789 wants to merge 1 commit into
rust-lang:mainfrom
clubby789:unicode-18
Open

lexer: Update to Unicode 18#165289
clubby789 wants to merge 1 commit into
rust-lang:mainfrom
clubby789:unicode-18

Conversation

@clubby789

@clubby789 clubby789 commented Nov 1, 2026 •
edited by rustbot
Loading

Copy link
Copy Markdown
Contributor

View all comments

Split out of #160261.
Reference update: rust-lang/reference#2399

This bumps:

  • unicode-ident: 1.0.24 -> 1.0.27
  • unicode-normalization: 0.1.25 -> 0.1.26
  • unicode-properties: 0.1.4 -> 0.1.5
  • unicode-script: 0.5.7 -> 0.5.9
  • unicode-security: 0.1.2 -> 0.1.3
  • unicode-width: 0.2.2 -> 0.2.3

Previous bump: #148321

@rustbot

rustbot commented Nov 1, 2026

Copy link
Copy Markdown
Collaborator

If the Unicode version changes are intentional,
it should also be updated in the reference at
https://github.com/rust-lang/reference/blob/HEAD/src/identifiers.md.

cc @rust-lang/lang-docs

These commits modify the Cargo.lock file. Unintentional changes to Cargo.lock can be introduced when switching branches and rebasing PRs.

If this was unintentional then you should revert the changes before this PR is merged.
Otherwise, you can ignore this comment.

@rustbot rustbot added A-CI Area: Our Github Actions CI A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue. labels Nov 1, 2026
@rustbot

rustbot commented Nov 1, 2026

Copy link
Copy Markdown
Collaborator

r? @Mark-Simulacrum

rustbot has assigned @Mark-Simulacrum.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Why was this reviewer chosen?

The reviewer was selected based on:

  • Owners of files modified in this PR: @Mark-Simulacrum

@clubby789 clubby789 changed the title Update to Unicode 18 lexer: Update to Unicode 18 Nov 1, 2026
@clubby789

clubby789 commented Nov 1, 2026

Copy link
Copy Markdown
Contributor Author

r? compiler

@rustbot rustbot assigned JohnTitor and unassigned Mark-Simulacrum Nov 1, 2026
@clubby789 clubby789 mentioned this pull request Nov 1, 2026
Open
@clubby789

clubby789 commented Nov 1, 2026

Copy link
Copy Markdown
Contributor Author

Just in case: @bors try @rust-timer queue

@rust-timer

This comment has been minimized.

Sign in to view
@rustbot rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Nov 1, 2026
rust-bors Bot pushed a commit that referenced this pull request Nov 1, 2026
@bors
Auto merge of #165289 - clubby789:unicode-18, r=<try>
c2c9c6a 
lexer: Update to Unicode 18
@rust-bors

This comment has been minimized.

Sign in to view
@clubby789 clubby789 added A-Unicode Area: Unicode T-lang Relevant to the language team and removed A-testsuite Area: The testsuite used to check the correctness of rustc T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue. A-CI Area: Our Github Actions CI labels Nov 1, 2026
@fmease fmease added I-lang-nominated Nominated for discussion during a lang team meeting. I-lang-easy-decision Issue: The decision needed by the team is conjectured to be easy; this does not imply nomination needs-fcp This change is insta-stable, or significant enough to need a team FCP to proceed. labels Nov 1, 2026
@fmease

fmease commented Nov 1, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

For T-lang:

If I may be so bold:

@rfcbot merge lang

@rust-rfcbot

rust-rfcbot commented Nov 1, 2026 •
edited by nikomatsakis
Loading

Copy link
Copy Markdown
Collaborator

@fmease has proposed to merge this. The next step is review by the rest of the tagged team members:

No concerns currently listed.

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

cc @rust-lang/lang-advisors: FCP proposed for lang, please feel free to register concerns.
See this document for info about what commands tagged team members can give me.

@rustbot rustbot added T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue. labels Nov 2, 2026
@rust-log-analyzer

This comment has been minimized.

Sign in to view
@clubby789

clubby789 commented Nov 2, 2026

Copy link
Copy Markdown
Contributor Author

The unicode-security update has introduced some new diagnostics into existing tests, which look reasonable to me

@rust-log-analyzer

This comment has been minimized.

Sign in to view
@rustbot rustbot added the T-rustdoc-frontend Relevant to the rustdoc-frontend team, which will review and decide on the web UI/UX output. label Nov 2, 2026
@rust-log-analyzer

This comment has been minimized.

Sign in to view
@rustbot

rustbot commented Nov 2, 2026

Copy link
Copy Markdown
Collaborator

clippy is developed in its own repository. If possible, consider making this change to rust-lang/rust-clippy instead.

cc @rust-lang/clippy

@rustbot rustbot added the T-clippy Relevant to the Clippy team. label Nov 2, 2026
@rust-log-analyzer

This comment has been minimized.

Sign in to view
@clubby789 clubby789 mentioned this pull request Nov 2, 2026
Merged
@clubby789

This comment was marked as outdated.

Sign in to view
ranger-ross pushed a commit to ranger-ross/cargo that referenced this pull request Nov 3, 2026
@weihanglo
Allow uncommon_codepoints in confusable test (rust-lang#17676)
2be5575 
rust-lang/rust#165289 updates the Unicode
version used by the lexer to 18, which includes a new set of characters
firing `uncommon_codepoints`. Allow this in a test to prevent failures
there.
@traviscross traviscross added the P-lang-drag-1 Lang team prioritization drag level 1. https://rust-lang.zulipchat.com/#narrow/channel/410516-t-lang label Nov 4, 2026
@traviscross

traviscross commented Nov 4, 2026

Copy link
Copy Markdown
Contributor

Thanks @clubby789, @fmease.

@rfcbot reviewed

@tmandry

tmandry commented Nov 4, 2026

Copy link
Copy Markdown
Member

@rfcbot reviewed

@rust-rfcbot rust-rfcbot added final-comment-period In the final comment period and will be merged soon unless new substantive objections are raised. and removed proposed-final-comment-period Proposed to merge/close by relevant subteam, see T-<team> label. Will enter FCP once signed off. labels Nov 4, 2026
@rust-rfcbot

rust-rfcbot commented Nov 4, 2026

Copy link
Copy Markdown
Collaborator

🔔 This is now entering its final comment period, as per the review above. 🔔

@traviscross traviscross added I-lang-radar Items that are on lang's radar and will need eventual work or consideration. and removed I-lang-nominated Nominated for discussion during a lang team meeting. I-lang-easy-decision Issue: The decision needed by the team is conjectured to be easy; this does not imply nomination P-lang-drag-1 Lang team prioritization drag level 1. https://rust-lang.zulipchat.com/#narrow/channel/410516-t-lang labels Nov 4, 2026
@rustbot

rustbot commented Nov 4, 2026

Copy link
Copy Markdown
Collaborator

This PR was rebased onto a different main commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@fmease fmease

Labels

A-CI Area: Our Github Actions CI A-testsuite Area: The testsuite used to check the correctness of rustc A-Unicode Area: Unicode disposition-merge This issue / PR is in PFCP or FCP with a disposition to merge it. final-comment-period In the final comment period and will be merged soon unless new substantive objections are raised. I-lang-radar Items that are on lang's radar and will need eventual work or consideration. needs-reference-pr This language change needs an approved Reference PR to proceed. S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) T-clippy Relevant to the Clippy team. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue. T-lang Relevant to the language team T-rustdoc-frontend Relevant to the rustdoc-frontend team, which will review and decide on the web UI/UX output.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.