Repository navigation
API: Raise when mixing SLEEF and longdouble backends - #122
Open
SwayamInSync wants to merge 22 commits into
Open
SwayamInSync wants to merge 22 commits into
SwayamInSync wants to merge 22 commits into
Conversation
…ing through double
…ly in the resolver
Member
Author
|
I believe the earlier behaviour was pretty non-deterministic and buggy, while I will be working on adding support of proper intermediate (f64) free inter-backend working (planned for future releases). We can strictly error-out the any inter-backend use for now instead of being sloppy with the implementation |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Addresses #113 for now, until #112 makes casts between backends exact.
Values of the SLEEF and longdouble backends currently move between each other in two flawed ways. Every conversion goes through
float64(#112), so it silently loses precision in both directions. Operations also disagree on the common backend (#113):np.result_typeandnp.concatenatepick SLEEF, while ufuncs pick longdouble.This PR makes the policy consistent by refusing every such operation with a
TypeErroruntil there is an exact conversion and one agreed promotion rule. Everything that does not combine the two backends is unchanged, and NumPy's own dtypes, includingnp.longdouble, work with both backends as before.Behaviour
Upstream
main(888042c) vs this branch, on x86-64 Linux:mainnp.result_type(sleef_dt, ld_dt)QuadPrecDType(backend='sleef')TypeError(sleef_arr + ld_arr).dtypeQuadPrecDType(backend='longdouble')TypeErrornp.concatenate([sleef_arr, ld_arr]).dtypeQuadPrecDType(backend='sleef')TypeError1 + 2**-112.astype(ld_dt), minus 10.0TypeError1 + 2**-60.astype(sleef_dt), minus 10.0TypeErrorld_arr[0] = QuadPrecision(1.5)0.0TypeErrornp.sqrt(sleef [4.0], out=ld_arr, casting="unsafe")0.0TypeErrorQuadPrecision(q) == qforq = 1 + 2**-112FalseTrueld_arr + 1[2.5]TypeErrorld_arr + np.float64(1)[2.5][2.5]ld_arr.astype(np.longdouble)[1.5][1.5]The error is
TypeError: Cannot mix QuadPrecision backends 'sleef' and 'longdouble'. Where NumPy checks the casting rule before calling our resolver (gufuncs,astype(..., casting="same_kind")), it is NumPy's ownCannot cast ...TypeErrorinstead.Backwards-incompatible changes
out=arrays have different backends,astype,np.copyto, slice and item assignment,np.result_type,np.promote_types,np.concatenate, andQuadPrecision(value, backend=...)given aQuadPrecisionof the other backend now raise.np.arrayof mixedQuadPrecisionscalars gives an object array.intandfloatuse the default SLEEF descriptor, so they raise with longdouble arrays too. This includes NumPy functions that pass Python scalars internally:np.ones,np.full,np.ones_like,mean,var,std,np.median,np.isclose,np.allcloseandnp.round. The workarounds are NumPy scalars or arrays, e.g.ld / np.float64(2),np.full(3, np.longdouble(1), dtype=ld.dtype)ornp.ones(3).astype(ld.dtype). SLEEF arrays are unaffected.Bug fixes
QuadPrecisioninto an array of the other backend, or casting an object array of them, stored the bytes under the wrong backend (0.0instead of1.5above).out=array of the other backend andcasting="unsafe"wrote values in the wrong format.QuadPrecision(q)rounded a same-backendqthroughfloat64; it now copies exactly.Implementation
quad_to_quad_resolve_descriptorsis the single point that refuses a cross-backend cast. The cast loop's cross-backend branch andquad_to_quad_same_value_checkare kept as they are, so BUG: inter-backend casts lose precision by converting through double #112 only has to replace thefloat64conversions and allow the cast in the resolver.common_instance, the binary, comparison and unary resolvers,setitemand the scalar paths raise through shared helpers indtype.h(quad_set_backend_mismatch_error,quad_loop_backend,quad_out_loop_descr).float64-based same_value tests for cross-backend casts are kept as strict xfails taggedgh-112. Once casts between backends are allowed, they XPASS and fail CI until the markers are removed.Tests and docs
tests/test_backend_mixing.pycovers every path above, the literal limitation, the workarounds, and that NumPy dtypes are unaffected. The new tests fail onmain.docs/api/core.md, and the changes above in the README changelog under "Unreleased".CI
big_endian.ymlskips pandas 3.1.0 on s390x. Its sdist does not build there (pandas-dev/pandas#71548, fix planned for 3.1.1), and s390x has no pandas wheels.pandas!=3.1.0keepstest_pandas_strreprunning on 3.0.6 and moves to 3.1.1 automatically.AI Disclosure
Used claude code as assistant for testing and writing PR/docs