Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build graph from expressive operators #31

Open
big-o opened this issue Nov 3, 2023 · 0 comments
Open

Build graph from expressive operators #31

big-o opened this issue Nov 3, 2023 · 0 comments

Comments

@big-o
Copy link
Collaborator

big-o commented Nov 3, 2023

Rather than a factory method, allow users to construct a graph by applying operators to estimators, for example:

dag1 = (
    NamedStep(est1, "step1")
    | NamedStep(est2, "step2")
    | (
        NamedStep(est3, "step3")
        & NamedStep(est4, "step4")
    )
    | NamedStep(est5, "step5")
)

This would create a linear pipeline from est1 -> est5, but with the second step feeding both steps 3 and 4, and step 5 receiving a concatenation of the two outputs from steps 3 and 4.

Complex dag construction can then be broken down into multiple statements too:

dag2 = (
    dag1.get_step("step2")
    | NamedStep(est6, "step6")
    | NamedStep(est7, "step7")
)

...would effectively create a new dag that is the original one with an extra branch added.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant