Add --use-editable-components-only flag #28140

smackesey · 2025-02-28T20:29:46Z

Summary & Motivation

Adds a --use-editable-components-only flag to complement --use-editable-dagster.

--use-editable-components-only is an option to commands that scaffold projects dg init and dg project scaffold. It will bring in dagster_components as an editable install but not dagster and other libs. This simulates the environment into which dg/components is released, so long as it is on a separate release schedule from Dagster core.
Update all tests that were using --use-editable-dagster to instead use --use-editable-components-only.
Refactor these options into a shared group.

How I Tested These Changes

New unit tests.

smackesey · 2025-02-28T20:30:03Z

[components] Reenable dg dev deployment test on windows #27815
Add --use-editable-components-only flag #28140 👈 (View in Graphite)
[components] Fix flaky module -> path resolution #28127
master

This stack of pull requests is managed by Graphite. Learn more about stacking.

graphite-app · 2025-02-28T22:27:36Z

python_modules/libraries/dagster-dg/dagster_dg/cli/shared_options.py

+            ["--disable-cache"],
+            is_flag=True,
+            default=DgCliConfig.disable_cache,
+            help="Disable the cache..",


The help text has a double period (..) which appears to be a typo. Should be Disable the cache.

Spotted by Graphite Reviewer

Is this helpful? React 👍 or 👎 to let us know.

graphite-app · 2025-02-28T22:28:22Z

python_modules/libraries/dagster-dg/dagster_dg/cli/shared_options.py

+            ["--disable-cache"],
+            is_flag=True,
+            default=DgCliConfig.disable_cache,
+            help="Disable the cache..",


The help text has a double period (..) which appears to be a typo. Should be "Disable the cache."

Spotted by Graphite Reviewer

Is this helpful? React 👍 or 👎 to let us know.

schrockn · 2025-03-01T01:02:42Z

python_modules/libraries/dagster-dg/dagster_dg/scaffold.py

+# the dependencies themselves differently depending on whether we are using editable dagster or
+# not. This is because `tool.uv.sources` only seems to apply to direct dependencies of the package,
+# so any 2+-order Dagster dependency of our package needs to be listed as a direct dependency in the
+# editable case.


Link to the relevant github issue in the uv repo in the comment

astral-sh/uv#9446 because i had it handy

schrockn

Makes sense. I'd like @gibsondan to take a pass at this before commit as he should be aware of all these venv shenanigans

gibsondan

ok so it seems like the underlying issue here is that dagster-components is not pinned to dagster like all our other libraries. Is the plan to eventually change that once components dev is more stable?

gibsondan · 2025-03-02T16:13:27Z

python_modules/libraries/dagster-dg/dagster_dg/cli/shared_options.py

+            ),
+        ),
+        click.Option(
+            ["--use-editable-components-only"],


I'm finding the name here pretty confusing in a way that --use-editable-dagster was not, because it makes me think that this is referring to the components themselves, rather than dagster-components the python package.

Maybe this is not the end of the world for a temporary hidden flag that is really only for development, but just flagging. you could consider going even more verbose and make this is like --use-editable-components-package-only

yeah I agree it's not clear. Will update to --use-editable-components-package-only.

gibsondan · 2025-03-02T16:35:34Z

python_modules/libraries/dagster-dg/dagster_dg/scaffold.py

+# the dependencies themselves differently depending on whether we are using editable dagster or
+# not. This is because `tool.uv.sources` only seems to apply to direct dependencies of the package,
+# so any 2+-order Dagster dependency of our package needs to be listed as a direct dependency in the
+# editable case.


astral-sh/uv#9446 because i had it handy

smackesey · 2025-03-02T16:39:50Z

Yes IIRC the plan is to actually just merge the entire dagster-components package into dagster.

smackesey mentioned this pull request Feb 28, 2025

[components] Fix flaky module -> path resolution #28127

Merged

smackesey force-pushed the sean/components/use-editable-components-only branch 2 times, most recently from aa7232d to 827338c Compare February 28, 2025 22:11

smackesey force-pushed the sean/components/reenable-validate-workspace-test branch from 30ea1c2 to 7c2a88f Compare February 28, 2025 22:11

smackesey mentioned this pull request Feb 28, 2025

[components] Reenable dg dev deployment test on windows #27815

Open

smackesey force-pushed the sean/components/use-editable-components-only branch from 827338c to 8c39154 Compare February 28, 2025 22:26

smackesey force-pushed the sean/components/reenable-validate-workspace-test branch from 7c2a88f to 9b2617a Compare February 28, 2025 22:26

smackesey marked this pull request as ready for review February 28, 2025 22:26

graphite-app bot reviewed Feb 28, 2025

View reviewed changes

smackesey changed the base branch from sean/components/reenable-validate-workspace-test to graphite-base/28140 February 28, 2025 22:31

smackesey force-pushed the sean/components/use-editable-components-only branch from 8c39154 to 071ea34 Compare February 28, 2025 22:31

smackesey force-pushed the graphite-base/28140 branch from 9b2617a to 5451c5c Compare February 28, 2025 22:31

smackesey changed the base branch from graphite-base/28140 to sean/components/reenable-validate-workspace-test February 28, 2025 22:31

smackesey force-pushed the sean/components/reenable-validate-workspace-test branch from 5451c5c to 6207228 Compare February 28, 2025 22:48

smackesey force-pushed the sean/components/use-editable-components-only branch 2 times, most recently from c157fef to 2a66b9c Compare February 28, 2025 23:00

smackesey requested review from benpankow and schrockn February 28, 2025 23:14

schrockn requested a review from gibsondan March 1, 2025 01:00

schrockn reviewed Mar 1, 2025

View reviewed changes

smackesey changed the base branch from sean/components/reenable-validate-workspace-test to graphite-base/28140 March 1, 2025 02:33

smackesey force-pushed the graphite-base/28140 branch from 6207228 to b60e419 Compare March 1, 2025 02:34

smackesey force-pushed the sean/components/use-editable-components-only branch from 2a66b9c to f676ec0 Compare March 1, 2025 02:34

smackesey changed the base branch from graphite-base/28140 to master March 1, 2025 02:34

gibsondan approved these changes Mar 2, 2025

View reviewed changes

smackesey force-pushed the sean/components/use-editable-components-only branch from f676ec0 to fa4fe9a Compare March 2, 2025 17:46

Add --use-editable-components-only flag

10ea579

smackesey force-pushed the sean/components/use-editable-components-only branch from fa4fe9a to 10ea579 Compare March 2, 2025 17:54

smackesey merged commit ff57fe9 into master Mar 2, 2025
6 checks passed

smackesey deleted the sean/components/use-editable-components-only branch March 2, 2025 18:09

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add --use-editable-components-only flag #28140

Add --use-editable-components-only flag #28140

smackesey commented Feb 28, 2025 •

edited

Loading

smackesey commented Feb 28, 2025 •

edited

Loading

graphite-app bot Feb 28, 2025

graphite-app bot Feb 28, 2025

schrockn Mar 1, 2025

gibsondan Mar 2, 2025

schrockn left a comment

gibsondan left a comment

gibsondan Mar 2, 2025

smackesey Mar 2, 2025

gibsondan Mar 2, 2025

smackesey commented Mar 2, 2025

Add --use-editable-components-only flag #28140

Add --use-editable-components-only flag #28140

Conversation

smackesey commented Feb 28, 2025 • edited Loading

Summary & Motivation

How I Tested These Changes

smackesey commented Feb 28, 2025 • edited Loading

graphite-app bot Feb 28, 2025

Choose a reason for hiding this comment

graphite-app bot Feb 28, 2025

Choose a reason for hiding this comment

schrockn Mar 1, 2025

Choose a reason for hiding this comment

gibsondan Mar 2, 2025

Choose a reason for hiding this comment

schrockn left a comment

Choose a reason for hiding this comment

gibsondan left a comment

Choose a reason for hiding this comment

gibsondan Mar 2, 2025

Choose a reason for hiding this comment

smackesey Mar 2, 2025

Choose a reason for hiding this comment

gibsondan Mar 2, 2025

Choose a reason for hiding this comment

smackesey commented Mar 2, 2025

smackesey commented Feb 28, 2025 •

edited

Loading

smackesey commented Feb 28, 2025 •

edited

Loading