Support default-extras config in the project interface.

[blueraft]

Summary

Depends on #12964. Closes #8607. Closes #13174.

Test Plan

cargo test.

Note: The draft PEP 771 uses the term default-optional-dependency-keys, but I've gone with default-extras here, since something like --no-default-optional-dependency-keys felt a bit too cumbersome to type. Let me know if we'd prefer sticking with the PEP’s naming instead.

[Gankra]

Apologies for the delay, reviewing now!

[Gankra]

I need to go back and read the PEP for what they do and don't specify but I wanna call out this fact right away: the --only semantics you've implemented here are in fact substantially different from the ones for dependency-groups.

To be clear, I think you're probably right to do so, given what I know of extras, but I want to make sure we're going eyes wide open into that fact.

With --only-group it also disables the actual package and its normal dependencies. This makes perfect sense for dependency-groups, because they're often "some devtools or whatever" and so "install the devtools but not the actual project" is indeed something you might want.

But "install only the extras for a package, and not the package" would generally be a nonsense thing to ask for, right? So we indeed don't want that?

If so then ExtrasSpecificationInner::prod should be removed, as the distinction it's intended to track is meaningless, and it's essentially unused in your implementation. The one place it is used is in ExtrasSpecificationInner::is_empty where I don't think it's really necessary (inclined to say "incorrect" but not 100% confident of that). By contrast the equivalent DependencyGroup API is used in several places, where it makes us throw out a ton of packages.

At that point it's also a question of whether --only-extra foo is carrying its weight, as it would just be sugar for --no-default-extras --extra foo.

[Gankra]

The impl otherwise looks reasonable (more detailed review of the main refactor in the other PR).

[Gankra]

rad that you added all these tests, it would be nice to include tests for the other commands you added support to as well.

[blueraft]

yes I'll add them!

[blueraft]

should I add them for the default groups for those commands too? They weren't there for the other commands last time I checked

[Gankra]

I wouldn't complain if you did but no biggie if you don't.

[blueraft]

But "install only the extras for a package, and not the package" would generally be a nonsense thing to ask for, right? So we indeed don't want that?

Yes I think so.

At that point it's also a question of whether --only-extra foo is carrying its weight, as it would just be sugar for --no-default-extras --extra foo.

Good point! I think we could skip adding a --only-extra argument here.

That said, the draft PEP 771 proposes the pip interface so that specifying an extra would implicitly disable default extras. If the PEP gets accepted, we will have to make some changes. 😅

[Gankra]

Ok reading the PEP more closely, notable details:

If extras are explicitly given in a dependency specification, the default extras are ignored. Otherwise, the default extras are installed.

If the same package is specified multiple times in an installation command or dependency tree, the default extras must be installed if any of the instances of the package are specified without extras.

Note that package[] would continue to be equivalent to package and would not be provided as a way to install without default extras (see the Rejected Ideas section for the rationale).

We also note that some tools such as pip currently ignore unrecognized extras, and emit a warning to the user to indicate that the extra has not been recognized, e.g:

$ pip install package[non-existent-extra]
WARNING: package 3.0.0 does not provide the extra 'non-existent-extra'

For tools that behave like this (rather than raising an error), if an extra is recognized as invalid in a dependency specification, it should be ignored and treated as if the user has not passed an explicit extra. If none of the provided extras are valid, default extras should be installed.

In some cases, package maintainers may want to facilitate installing packages without any default extras. In this case, as will be shown in more detail in Examples, the best approach is to define an extra which could be called e.g. minimal or nodefault (the naming would be up to the package maintainer) which would be an empty set of dependencies. If this extra is specified, no default extras will be included, so that e.g. package[minimal] would include only required dependencies and no extras. Note that this requires no additional specification and is a natural consequence of the rule described in Overriding default extras.

The most significant backward-compatibility aspect is related to assumptions packaging tools make about extras – specifically, this PEP changes the assumption that extras are no longer exclusively additive in terms of adding dependencies to the dependency tree, and specifying some extras can result in fewer dependencies being installed (they go on to detail a case where pip freeze simplifies package[minimal] to package, which then results in defaults still applying when you pip install -r requirements.txt).

[charliermarsh]

But "install only the extras for a package, and not the package" would generally be a nonsense thing to ask for, right? So we indeed don't want that?

Just confirming that, yes, extras are purely additive whereas groups are designed to be sensical "without the package". So only installing a given extra (but not the base package) is probably not something we should support.

[Gankra]

The pep also links a draft pip implementation pypa/pip@main...wheelnext:pip:pep_771

[Gankra]

It's interesting because a lot of the PEP's semantics are for when talking about the package as a dependency, but the scope of this PR, by nature of it being a uv extension, means it has no(?) application to the package as a dependency.

So it's really more like dependency-groups where it's purely a "developer of this project" experience improvement. Very interesting, I'm not sure if the CLI flags want to reproduce the behaviour of the PEP.

Although really all the difference is that --extra foo becomes --only-extra foo?

[blueraft]

So it's really more like dependency-groups where it's purely a "developer of this project" experience improvement. Very interesting, I'm not sure if the CLI flags want to reproduce the behaviour of the PEP.

Yes, I think since many of these changes involve package metadata, the resolver will need to handle them. But if the PEP is accepted, I expect it will also influence local development workflows. For example, if a package defines default-optional-dependency-keys, we’d likely want to install those by default. However, would we want to maintain two separate fields for it then?

Although really all the difference is that --extra foo becomes --only-extra foo?

As far as I can tell, yes that seems to be the key difference for us. We could choose to handle this differently in the project interface though.

[Gankra]

However, would we want to maintain two separate fields for it then?

We would do the same thing we did with our old non-standard support for dependency-groups, where we take both sets of fields and merge them intelligently, with the understanding that everyone will want to move to the standard system when it's available (the maintenance burden is minimal).

[Gankra]

Flagging "do we want --extra foo to have the semantics of --only-extra foo" as a design blocker on landing this.

Context: the (draft) PEP for default-extras specifies that mypackage installs the default extras, while mypackage[foo] actually disables the defaults and installs only foo (the user can do something like mypackage[recommended, foo] to recover the defaults, or the developer of the package can make foo inherit recommended if they don't like this behaviour).

If you want uv sync --extra foo to be "as if" you installed mypackage[foo] it would follow that --extra foo is in fact --only-extra foo.