Skip to content

Preserve calendar dates with UTC midnight encoding - #209

Closed
pengpengyi92 wants to merge 1 commit into
Point72:mainfrom
pengpengyi92:codex/utc-date-encoding-20261114
Closed

pengpengyi92 wants to merge 1 commit into
Point72:mainfrom
pengpengyi92:codex/utc-date-encoding-20261114

Conversation

@pengpengyi92

Copy link
Copy Markdown

Description

Encode plain Python date values at UTC midnight rather than local midnight.

_to_epoch_millis(date(...)) currently creates a naive datetime, so timestamp() uses the host timezone. On a UTC+8 Windows host, inserting date(2026, 8, 21) into a real Perspective date column stores 2026-08-20 instead. The intermediate encoding is 1787241600000; the stored date is 1787184000000, rather than the expected 1787270400000. Pre-1970 dates can also raise OSError through the local timestamp path on Windows.

Set tzinfo=UTC only in the plain-date branch. The earlier datetime branch is unchanged: aware datetimes still preserve their instant and naive datetimes retain the existing local-time semantics. No dependency, API, table-registration, server-lifecycle or version changes.

Regression tests verify scalar encoding, actual native Perspective round trips and Dashboard.apply, including a leap date and a pre-epoch date, unchanged input rows and non-temporal values. On systems with time.tzset, the date cases run under UTC, UTC+8 and UTC-5 and restore the original timezone; Windows uses its actual configured timezone. The tests do not replace datetime or Perspective with mocks.

This is separate from #208's failed-table registration fix; it changes only the date conversion line, not that PR's table-creation path.

Type of Change

  • Bug fix
  • New feature
  • Documentation update
  • Refactor / code cleanup
  • CI / build configuration
  • Other (describe below)

Validation

Base: d7a553243c2a3a654e3119140df773d1f4695791. Windows, UTC+8, Python 3.12.14, native Perspective 5.6.1, Ray 2.60.0, spaday 0.14.1.

python -m pytest raydar/tests/test_date_encoding.py raydar/tests/test_dashboard.py raydar/tests/test_ops.py raydar/tests/test_schema.py -q -o addopts= -p no:cacheprovider --import-mode=importlib --tb=short
python -m ruff check raydar
python -m ruff format --check raydar
  • Same new date tests on the detached base: 5 failed / 6 passed; candidate 11 passed.
  • Same full related selection on base: 5 failed / 50 passed; candidate 55 passed, no deselections/skips. This includes HTTP/WebSocket app and local-server stop/port-reuse tests, with localhost allowed.
  • Python lint/format pass across 20 files. Ray emits 18 existing Pydantic deprecation warnings.
  • These scopes overlap, not 66 distinct cases. The POSIX tzset matrix, full Ray actor/cluster integration, docs lint and distribution build have not been run locally; upstream CI is separate evidence.

Checklist

  • Full make lint (Python portion passed; docs lint not run)
  • Full make test (full related selection passed, not all Ray integrations)
  • Regression tests added
  • No new public operation or documented API contract
  • Changelog / version bump (not included)

AI assistance: Codex assisted with investigation, the original patch/tests and self-review. The contributor explicitly authorized DCO sign-off for this contribution after verification. No private code/data is included, and no separate human or maintainer approval is claimed.

Thanks for reviewing! Happy to adjust the date serialization approach if a different calendar-date contract is preferred.

Signed-off-by: Pengyi Peng <pengpengyi92@users.noreply.github.com>
@timkpaine

Copy link
Copy Markdown
Member

Thanks for the PR! I've moved to #210 to consolidate and trim

@timkpaine timkpaine closed this Nov 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants