[WIP] Forbid object lifetime changing pointer casts #136776
Conversation
rustbot
added
S-waiting-on-review
|
@bors try |
…, r=<try> [WIP] Forbid object lifetime changing pointer casts Fixes rust-lang#136702 r? `@ghost`
|
☀️ Try build successful - checks-actions |
|
@craterbot check |
|
👌 Experiment ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more |
craterbot
added
S-waiting-on-crater
|
🚧 Experiment ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more |
|
🎉 Experiment
|
craterbot
added
S-waiting-on-review
|
Most of these are on github; in terms of crates.io regressions all we have is:
Overall, 142 regressions are caused by EDIT: Ah, there's also |
traviscross
added
the
I-lang-nominated
|
We discussed this in the lang triage call today. We wanted to think more about it, so we're leaving it nominated to discuss again. |
|
@BoxyUwU Do you think it would be possible to implement this as an FCW? We talked about this in lang triage today and would prefer to start with that if we can. If it's not feasible, a hard error can also work (I would say though that we should upstream PRs to any crates we break). Another small thing I noticed is that the error message links to the Nomicon section on variance, but it would be ideal to link to a tracking issue or something describing this issue in particular. |
|
To add on to what tmandry, said, in our discussions we did feel that the approach taken in this PR is generally the right way forward, and we're happy to see this progress so as to help clear the way for cc @rust-lang/lang |
|
@tmandry I do expect it to be possible to FCW this. We can likely do something hacky around to fully emulate the fix (but as a lint), but if that doesn't work out all the regression we found were relatively "simple" cases that can probably be taken advantage of (if need be) to lint a subset of the actual cases we'd break with this PR edit: see compiler-errors' comment, I'm not so convinced this will be possible to FCW anymore and will likely investigate improving the diagnostics here. I've already filed PRs to the affected crates to migrate them over to a transmute to avoid the breakage if this lands |
|
I was thinking earlier that it may be possible to implement a lint to detect, but it seems to me that MIR borrowck is not equipped to implement such a lint. Specifically, it seems near impossible to answer whether a region outlives constraint (like, To fix this would require some significant engineering effort to refactor how NLL processes its region graph to make it easier to clone and reprocess with new constraints. |
…uto_to_object-hard-error, r=oli-obk Make `ptr_cast_add_auto_to_object` lint into hard error In Rust 1.81, we added a FCW lint (including linting in dependencies) against pointer casts that add an auto trait to dyn bounds. This was part of work making casts of pointers involving trait objects stricter, and was part of the work needed to restabilize trait upcasting. We considered just making this a hard error, but opted against it at that time due to breakage found by crater. This breakage was mostly due to the `anymap` crate which has been a persistent problem for us. It's now a year later, and the fact that this is not yet a hard error is giving us pause about stabilizing arbitrary self types and `derive(CoercePointee)`. So let's see about making a hard error of this. r? ghost cc `@adetaylor` `@Darksonn` `@BoxyUwU` `@RalfJung` `@compiler-errors` `@oli-obk` `@WaffleLapkin` Related: - rust-lang#135881 - rust-lang#136702 - rust-lang#136776 Tracking: - rust-lang#127323 - rust-lang#44874 - rust-lang#123430
…uto_to_object-hard-error, r=oli-obk Make `ptr_cast_add_auto_to_object` lint into hard error In Rust 1.81, we added a FCW lint (including linting in dependencies) against pointer casts that add an auto trait to dyn bounds. This was part of work making casts of pointers involving trait objects stricter, and was part of the work needed to restabilize trait upcasting. We considered just making this a hard error, but opted against it at that time due to breakage found by crater. This breakage was mostly due to the `anymap` crate which has been a persistent problem for us. It's now a year later, and the fact that this is not yet a hard error is giving us pause about stabilizing arbitrary self types and `derive(CoercePointee)`. So let's see about making a hard error of this. r? ghost cc ``@adetaylor`` ``@Darksonn`` ``@BoxyUwU`` ``@RalfJung`` ``@compiler-errors`` ``@oli-obk`` ``@WaffleLapkin`` Related: - rust-lang#135881 - rust-lang#136702 - rust-lang#136776 Tracking: - rust-lang#127323 - rust-lang#44874 - rust-lang#123430
…uto_to_object-hard-error, r=oli-obk Make `ptr_cast_add_auto_to_object` lint into hard error In Rust 1.81, we added a FCW lint (including linting in dependencies) against pointer casts that add an auto trait to dyn bounds. This was part of work making casts of pointers involving trait objects stricter, and was part of the work needed to restabilize trait upcasting. We considered just making this a hard error, but opted against it at that time due to breakage found by crater. This breakage was mostly due to the `anymap` crate which has been a persistent problem for us. It's now a year later, and the fact that this is not yet a hard error is giving us pause about stabilizing arbitrary self types and `derive(CoercePointee)`. So let's see about making a hard error of this. r? ghost cc ```@adetaylor``` ```@Darksonn``` ```@BoxyUwU``` ```@RalfJung``` ```@compiler-errors``` ```@oli-obk``` ```@WaffleLapkin``` Related: - rust-lang#135881 - rust-lang#136702 - rust-lang#136776 Tracking: - rust-lang#127323 - rust-lang#44874 - rust-lang#123430
Rollup merge of rust-lang#136764 - traviscross:TC/make-ptr_cast_add_auto_to_object-hard-error, r=oli-obk Make `ptr_cast_add_auto_to_object` lint into hard error In Rust 1.81, we added a FCW lint (including linting in dependencies) against pointer casts that add an auto trait to dyn bounds. This was part of work making casts of pointers involving trait objects stricter, and was part of the work needed to restabilize trait upcasting. We considered just making this a hard error, but opted against it at that time due to breakage found by crater. This breakage was mostly due to the `anymap` crate which has been a persistent problem for us. It's now a year later, and the fact that this is not yet a hard error is giving us pause about stabilizing arbitrary self types and `derive(CoercePointee)`. So let's see about making a hard error of this. r? ghost cc ```@adetaylor``` ```@Darksonn``` ```@BoxyUwU``` ```@RalfJung``` ```@compiler-errors``` ```@oli-obk``` ```@WaffleLapkin``` Related: - rust-lang#135881 - rust-lang#136702 - rust-lang#136776 Tracking: - rust-lang#127323 - rust-lang#44874 - rust-lang#123430
…ject-hard-error, r=oli-obk Make `ptr_cast_add_auto_to_object` lint into hard error In Rust 1.81, we added a FCW lint (including linting in dependencies) against pointer casts that add an auto trait to dyn bounds. This was part of work making casts of pointers involving trait objects stricter, and was part of the work needed to restabilize trait upcasting. We considered just making this a hard error, but opted against it at that time due to breakage found by crater. This breakage was mostly due to the `anymap` crate which has been a persistent problem for us. It's now a year later, and the fact that this is not yet a hard error is giving us pause about stabilizing arbitrary self types and `derive(CoercePointee)`. So let's see about making a hard error of this. r? ghost cc ```@adetaylor``` ```@Darksonn``` ```@BoxyUwU``` ```@RalfJung``` ```@compiler-errors``` ```@oli-obk``` ```@WaffleLapkin``` Related: - rust-lang/rust#135881 - rust-lang/rust#136702 - rust-lang/rust#136776 Tracking: - rust-lang/rust#127323 - rust-lang/rust#44874 - rust-lang/rust#123430
…ject-hard-error, r=oli-obk Make `ptr_cast_add_auto_to_object` lint into hard error In Rust 1.81, we added a FCW lint (including linting in dependencies) against pointer casts that add an auto trait to dyn bounds. This was part of work making casts of pointers involving trait objects stricter, and was part of the work needed to restabilize trait upcasting. We considered just making this a hard error, but opted against it at that time due to breakage found by crater. This breakage was mostly due to the `anymap` crate which has been a persistent problem for us. It's now a year later, and the fact that this is not yet a hard error is giving us pause about stabilizing arbitrary self types and `derive(CoercePointee)`. So let's see about making a hard error of this. r? ghost cc ```@adetaylor``` ```@Darksonn``` ```@BoxyUwU``` ```@RalfJung``` ```@compiler-errors``` ```@oli-obk``` ```@WaffleLapkin``` Related: - rust-lang/rust#135881 - rust-lang/rust#136702 - rust-lang/rust#136776 Tracking: - rust-lang/rust#127323 - rust-lang/rust#44874 - rust-lang/rust#123430
…ject-hard-error, r=oli-obk Make `ptr_cast_add_auto_to_object` lint into hard error In Rust 1.81, we added a FCW lint (including linting in dependencies) against pointer casts that add an auto trait to dyn bounds. This was part of work making casts of pointers involving trait objects stricter, and was part of the work needed to restabilize trait upcasting. We considered just making this a hard error, but opted against it at that time due to breakage found by crater. This breakage was mostly due to the `anymap` crate which has been a persistent problem for us. It's now a year later, and the fact that this is not yet a hard error is giving us pause about stabilizing arbitrary self types and `derive(CoercePointee)`. So let's see about making a hard error of this. r? ghost cc ```@adetaylor``` ```@Darksonn``` ```@BoxyUwU``` ```@RalfJung``` ```@compiler-errors``` ```@oli-obk``` ```@WaffleLapkin``` Related: - rust-lang/rust#135881 - rust-lang/rust#136702 - rust-lang/rust#136776 Tracking: - rust-lang/rust#127323 - rust-lang/rust#44874 - rust-lang/rust#123430
BoxyUwU
added
the
I-lang-easy-decision
|
Hi @rust-lang/lang I'm "re nominating" this (even though it was already nominated lol). I'm not asking for a decision as to whether the breakage here is acceptable/okay, instead I'd like to see if lang is fine delegating the decision here to the types team or if y'all would like to be kept in the loop here and be included on an FCP here (which I think would be perfectly fine :-)). From my POV lang already FCP'd the decision to require VTables on raw pointers to be valid so this is effectively just enforcing that as we should have been and as such could be seen as "just" a small tweak to the implementation of the type system which historically the types team has had autonomy over. |
|
@rustbot label -I-lang-easy-decision We've been discussing this nomination near the start of every meeting to track status and see what our options might be due to what this is blocking. Niko has been checking in with people every week. What Niko had discussed later yesterday in the RfL call on this is that the best option might be to go to a hard error but with targeted diagnostics that would detect this and issue a suggested code change. My own estimation is that this going to remain a lang + types matter, given our interest in this and given that there's likely to be non-trivial breakage here, and that we're likely to have lang views on what represents an acceptable user experience for this migration. |
rustbot
removed
the
I-lang-easy-decision
|
@rustbot labels -I-lang-nominated We discussed this in our meeting today. Meeting consensus is that given that warning is not feasible we are in favor of going forward with this change with the proviso that we will have an error message with actionable instructions and open PRs against known regressions. Side note, informal design axioms for breaking changes...
|
rustbot
removed
the
I-lang-nominated
|
PRs against affected crates have been opened and can be seen here:
There were three regressions I've not filed PRs against:
It feels a bit awkward to bring up after having filed these PRs but regardless it seems like due diligence to ask anyway; is it worth considering an alternative fix to this problem with arbitrary self types? A couple options: Allow lifetime casts in unsafe code onlySomeone asked on one of the PRs whether it would be reasonable to allow this code to continue to work when the code is placed in an unsafe block. This would mean that behaviour of This feels somewhat dubious to me as it is not super clear that a safety invariant is being introduced when Regardless- it would solve the soundness bug and minimize the breakage to some extent. Looking at the regressions this would only avoid breaking a few of the affected crates, but this does include the We could potentially only do this as a migration strategy by breaking this even in unsafe contexts across an edition where it's more "morally correct" to make a breaking change. (This would be my preference if we do this as having this as intentional behaviour would likely be quite bad for teachability of unsafe, see followup comments) @RalfJung I imagine you would probably have opinions about muddling the waters around what unsafe code does in this way (?) Require construction of smart pointers that implement
|
BoxyUwU
added
the
I-lang-nominated
Purely conceptually, it seems fine to me to say that some For this concrete question that would mean we have to allow such invalid-lifetime dyn trait values to exist temporarily (i.e., they satisfy the language invariant). Is that where we stand today, i.e.,, Miri would accept the |
This is quite dubious. We're retroactively attaching more safety obligations to an existing operation, and then if there's UB somewhere we tell you its your fault since you wrote Does |
|
Is it accurate to say that the UB being "added" here is not detectable within e.g. Miri, because we lack the information about the lifetimes present to enforce that you didn't mess this up? It seems unfortunate if that's true, because I could easily see there being code out there that didn't satisfy this safety obligation but is already using transmute for other reasons. It's pretty common I think to see casts to I don't see a clear alternative to this -- I think we are sort of stuck given past decisions -- but I hadn't seen that question brought up so wanted to raise it here. Or maybe I've misunderstood, and we're actually not adding UB from violating this condition -- merely working to prevent it, and only if you happen to explicitly do something "wrong" does your code actually break. (Essentially saying that you shouldn't leak such a value to safe code, but there's no UB from just having it). |
| = help: consider adding the following bound: `'a: 'b` | ||
| = note: requirement occurs because of a mutable pointer to `dyn Trait<'_>` | ||
| = note: mutable pointers are invariant over their type parameter | ||
| = help: see <https://doc.rust-lang.org/nomicon/subtyping.html> for more information about variance |
There was a problem hiding this comment.
Is it possible for us to note something in these errors pointing at some docs for why this is a bad idea? I could easily see someone just changing this to a transmute without realizing this is an intentional limitation of as casts.
I guess this falls under "dyn Trait metadata is invalid if it is not a pointer to a vtable for Trait that matches the actual dynamic trait the pointer or reference points to" in some sense (from https://doc.rust-lang.org/nightly/nomicon/what-unsafe-does.html) but maybe that should be clarified to say that it's not just "trait" but rather "trait and lifetime bounds on it" (or explicitly note this is a safety, not validity, invariant)...
There was a problem hiding this comment.
Yeah I'm gonna try improve the diagnostics here before this can land as it's not a very helpful error message to get both from the POV of someone whose code just broke, or from the POV of someone who just tried to ptr cast the lifetimes in new code
With regard to Ralf's ping, I think that this would be a teaching disaster and a regretful wart on the language unless this is just to provide a smoother deprecation period. |
My understanding is that there's no immediate UB when doing the wrong-lifetime cast, but there can be UB further down the road since now we can make virtual function calls we shouldn't have been able to make. So, the cast breaks a library/safety invariant, but not a language/validity invariant. Miri can only check language invariants. |
On stable I don't think you can actually do anything "wrong" with these pointer casts as the only way to dispatch through the vtable requires going through unsafe code, either via reborrowing to get a reference instead of a pointer, or by going through an unsafe If you (incorrectly) used unsafe to do those operations that situation is somewhat analogous to the case with arbitrary self types where no language level UB has been reached but it's possible to perform a vtable call without where clauses being satisfied. I'm not sure if you can really escalate this "without where clauses being satisified" into language level UB. Even with arbitrary self types, it's a raw pointer so it's not safe to simply dereference and get a value out of it that is incorrectly believed to live for longer than it actually should. If there's a safety invariant somewhere that the pointer is valid for reads of the pointee type then it's also not problematic as that rules out having these kinds of pointers passed in. This is the same kind of logic as to why these casts don't cause any problems for std's smart pointers as they all require unsafe to construct from a raw pointer. So both on stable and with arbitrary_self_types I don't believe Miri can/should detect anything here and I'm also not confident you could actually escalate this into language level UB detectable by Miri. We can't not generate the methods with unsatisfiable where clauses for the vtable as they're only unsatisfiable due to lifetime bounds which we don't really have the ability to reason about in this way. Regardless its certainly wrong for the type system to allow this in safe code...
This is roughly my opinion too 👍 I would be quite concerned about the teachability of this and would only want to go ahead with this if the intention was to follow up with making it hard error in future editions. |
I am that person that suggested this, and after mulling it over a bit more, I also think it would be a mistake and the transmute makes a lot more sense here (besides the teachability of it which would also be increadibly complicated to understand). In my case, I need to go through a Now, I do have a question about what will be allowed and what will be forbidden. The test cases in this PR so far only mention
Supposedly though, the problem and change also touches |
| && let ty::Dynamic(dst_tty, ..) = dst_tail.kind() | ||
| if let ty::Dynamic(src_tty, src_region_bound, ..) = src_tail.kind() | ||
| && let ty::Dynamic(dst_tty, dst_region_bound, ..) = | ||
| dst_tail.kind() | ||
| && src_tty.principal().is_some() | ||
| && dst_tty.principal().is_some() |
There was a problem hiding this comment.
Is the "when there aren't principal traits" still true? It seems like I could get into trouble with a fn (self: dyn Send + 'static), right?
|
Thanks! Mostly trying to make sure we're actually fully closing the intended soundness hole... IIUC, it sounds like when we say:
I think we're referring to the decision in #101336... which defined the safety invariant as:
That is actually somewhat ill-specified. In order for us to avoid bugs, I think it would help to make that a bit more specific. I believe we are saying is that the safety invariant for the vtable is:
I think (2) only has a gap with respect to the principal lifetime (i.e., what this PR is closing) because we'd previously (in #120248) closed the other gaps by saying:
That PR notes that we still allow two cases:
I think that implies that we should go edit the reference (after this PR merges) to update the (Also left one inline comment on a possibly missing case in the PR?) |
Fixes #136702
r? @ghost