Tracking issue for RFC 2043: Add align_offset intrinsic (formerly: and [T]::align_to function) #44488
Description
This is a tracking issue for the RFC "Add align_offset intrinsic and [T]::align_to function " (rust-lang/rfcs#2043). align_to is stable, so this tracks just align_offset now.
Steps:
- Implement the RFC (align_offset: Add align_offset intrinsic #43903; align_to: Implement [T]::align_to #50319)
- Adjust documentation (see instructions on forge)
- Stabilization PR (see instructions on forge):
align_to: stabilize slice_align_to #53754
Unresolved questions:
- "produce a lint in case
sizeof<T>() % sizeof<U>() != 0and in case the expansion is not part of a monomorphisation, since in that casealign_tois statically known to never be effective
Activity
Rollup merge of rust-lang#44537 - oli-obk:memchr, r=alexcrichton
Rollup merge of rust-lang#44537 - oli-obk:memchr, r=alexcrichton
Rollup merge of rust-lang#44537 - oli-obk:memchr, r=alexcrichton
Rollup merge of rust-lang#46812 - kennytm:fix-align-offset-sign, r=pe…
SimonSapin commentedon Mar 17, 2018
@oli-obk Is it useful to stabilize
align_offsetbeforealign_tois implemented?oli-obk commentedon Mar 17, 2018
Generally yes, because that's the key feature. Align_to is just convenience
SimonSapin commentedon Mar 17, 2018
Alright, then. Let’s stabilize
align_offsetand leave this issue open for trackingalign_to.@rfcbot fcp merge
rfcbot commentedon Mar 17, 2018
Team member @SimonSapin has proposed to merge this. The next step is review by the rest of the tagged team members:
Concerns:
motivation-and-ergonomicsresolved by Tracking issue for RFC 2043: Addalign_offsetintrinsic (formerly: and[T]::align_tofunction) #44488 (comment)Once a majority of reviewers approve (and none object), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!
See this document for info about what commands tagged team members can give me.
alexcrichton commentedon Mar 19, 2018
@rfcbot concern motivation-and-ergonomics
The original motivation for these functions (unless it's changed, which it may have!) I believe was for miri and const evaluating various functions. Is that still the main motivation for these functions? If so do we expect that more things to be available on stable once these are stabilized?
I'm a little unclear about how and when such an intrinsic would be used so I'm mostly looking to straighten up my own thinking! In particular the clause "If it is not possible to align the pointer, the implementation returns usize::max_value()." I'm having trouble wrapping my head around in how it'd be expected to get used.
Is the long term plan to make this a
const fnbut otherwise disallow operations like*mut T as usizein aconst fn?In terms of ergonomics I'd also naively expect a signature like:
but would that create the same problems it's trying to avoid?
41 remaining items
jethrogb commentedon Oct 11, 2019
What is the point of this function? I'm dealing with hardware where if the pointer is not aligned, the instruction will fault. So I need to implement this function myself because this function is not guaranteed to be useful?
oli-obk commentedon Oct 11, 2019
The point of this function that if it returns an offset that is out of bounds of your allocation, you should run your backup logic that does not depend on alignment. Imagine having a
*mut [u8; 2]that you try to align to be usable as*mut u16, if you get an offset of1(because the original pointer was not aligned), you can't use the resulting pointer. In this case using your operation would be UB because you'd be working outside of the allocation.Platforms like
mirireturningusize::max_value()is essentially just a way of signaling that you need to run the backup logic that you have to have anyway in case the alignment moves beyond your bufferjethrogb commentedon Oct 11, 2019
What backup logic? As stated, in my use case it is not possible to use an unaligned value.
oli-obk commentedon Oct 11, 2019
In that case, what happens if you get an aligned value but it's outside of your allocation? You can't make a memory location aligned, you can only offset a pointer until it is aligned. But offsetting a pointer may go beyond the allocation your pointer points to
EDIT: maybe you need to make this decision at a higher level? e.g. by using the corresponding slice method (https://doc.rust-lang.org/std/primitive.slice.html#method.align_to) ?
RalfJung commentedon Oct 11, 2019
This function is for cases where e.g. you have a
&mut [u8]that could be large, and you want to get the "aligned center" of that slice as&mut [u64]for more efficient processing. Maybe your case is different, in which case maybe this function is not for you.But as @oli-obk said, you need backup logic anyway because the pointer you are getting might not be sufficiently aligned, and the buffer might not be large enough to find something aligned "in the middle". What do you want to do in that case? I think we need a bit more context here.
jethrogb commentedon Oct 11, 2019
No this is pretty much exactly the case for me. But I know that my input buffer is guaranteed to be large enough to hold the "aligned center".
RalfJung commentedon Oct 11, 2019
Well, that is a subtly different use-case -- not what this function was designed for, and also one that Miri currently does not support. (There are trade-offs here: supporting that means being less good at detecting misaligned memory accesses.)
In particular, what do you do with the unaligned prefix and suffix? Usually they also need processing, and that's why you need fall-back code.
You could always use
align_toand then assert that the center part is non-empty. Or you just do the usual "round ptr up to next multiple of alignment"; you don't actually need all the complexity inalign_to(which is mostly caused by returning a result measured in units ofsize_of::<T>()).jethrogb commentedon Oct 11, 2019
I'm talking about
ptr::align_offset, which, in my mind, is exactly supposed to do the rounding up. The documentation I quoted earlier is from that function.I'm using
mmapto try to allocate naturally-aligned memory by asking for twice the size I need, computing the right range, and thenmunmapping the rest.RalfJung commentedon Oct 11, 2019
Yeah that's not at all the use-case for which this was designed. If you want to implement an aligned allocator based on an unaligned one, this function currently won't help you. libstd in fact also contains such code because Windows.
Oh but it is doing way more than that. Quoting from the docs:
That's what makes this function complicated. It actually solves an integer modulo congruence.
jethrogb commentedon Oct 11, 2019
@RalfJung
Tisu8for me.It seems weird to me that
ptr::align_offsetshould not be used to align pointer values except in some special cases. It's not clear at all from the documentation that you shouldn't use this function (except for the warning that the function might not give a useful output).RalfJung commentedon Oct 11, 2019
I guessed as much. So? All I said is, the function is more general than your use-case, making it way more complicated than "rounding up", and its reason to exist is not what you seem to think it is (to remove some simple rounding-up code from allocators -- an extremely rare thing to write).
You are looking at a very complicated tool with subtle trade-offs, wanting to use it in its simplest possible form (ignoring 99% of its functionality) and then complain that it doesn't exactly do what you want it to do. Well, I am afraid the tradeoffs that informed its design are different than yours. Probably this is not the tool you are looking for.
Assuming allocation is expensive, it also shouldn't be too much of a problem to call
align_offsetand thenassertthat your result is< align.You (like, the general public) should use it. You (@jethrogb) are just misinterpreting what it should be used for.
RalfJung commentedon Oct 11, 2019
In fact the docs say quite clearly:
So, if that's not a constraint you can live with, that's okay -- just don't use this function.
jethrogb commentedon Oct 11, 2019
Except that won't work, because:
So when I do the assertion (which I'd be fine with), the implementation is free to unconditionally insert an assertion failure. Unless you think that that's not the case, in which case this is a documentation issue.
I definitely can't live with this, because now I still need to implement a "round up" function in case this function failed. Then I might as well just not use this function.
I don't think pointer alignment manipulation is at all rare when dealing with low-level code, something Rust is supposed to be good at. Definitely not limited to allocators. I will not be the first person to come across this function when looking at the
ptrdocs thinking it would be useful.RalfJung commentedon Oct 11, 2019
Correct.
The current situation is that the libstd/rustc implementation will always succeed to align if
size_of::<T>() == 1. However, the Miri implementation will not. The doc has to cover all implementations, but you could decide to rely on a specific one.So far, allocators are the only use-case I am aware of that would need proper alignment for correctness, not just performance. Most of the time you are fed in data you have to work with; allocation is the only time when you are asked to produce memory with a certain alignment. But I am mostly unfamiliar with embedded programming and many other ways of using Rust, so if anyone reading along also has a use-case that is affected by this, please chime in. :)
For your use-case, a "please round up this pointer/integer to alignment X" function seems to be much more useful though than something that just computes the offset?