`GraphStore` definition + `Data` and `HeteroData` integration #4816

mananshah99 · 2022-06-17T19:00:11Z

This PR defines a very lightweight MaterializedGraph that supports basic put and get operations of edge indices. It additionally lets Data and HeteroData inherit from and implement this abstraction.

…into feature_store_pt1

codecov · 2022-06-17T19:11:27Z

Codecov Report

Merging #4816 (f8b8d9e) into master (4b30b6d) will decrease coverage by 1.82%.
The diff coverage is 94.54%.

@@            Coverage Diff             @@
##           master    #4816      +/-   ##
==========================================
- Coverage   84.53%   82.70%   -1.83%     
==========================================
  Files         325      326       +1     
  Lines       17430    17518      +88     
==========================================
- Hits        14735    14489     -246     
- Misses       2695     3029     +334

Impacted Files	Coverage Δ
torch_geometric/data/feature_store.py	`88.80% <90.00%> (ø)`
torch_geometric/data/data.py	`91.03% <93.02%> (+0.15%)`	⬆️
torch_geometric/data/graph_store.py	`95.00% <95.00%> (ø)`
torch_geometric/data/hetero_data.py	`93.84% <100.00%> (+0.20%)`	⬆️
torch_geometric/typing.py	`100.00% <100.00%> (ø)`
torch_geometric/nn/models/dimenet_utils.py	`0.00% <0.00%> (-75.52%)`	⬇️
torch_geometric/nn/models/dimenet.py	`14.51% <0.00%> (-53.00%)`	⬇️
torch_geometric/nn/conv/utils/typing.py	`81.25% <0.00%> (-17.50%)`	⬇️
torch_geometric/nn/inits.py	`67.85% <0.00%> (-7.15%)`	⬇️
torch_geometric/nn/resolver.py	`90.00% <0.00%> (-6.67%)`	⬇️
... and 7 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4b30b6d...f8b8d9e. Read the comment docs.

torch_geometric/data/hetero_data.py

torch_geometric/data/materialized_graph.py

Padarn · 2022-06-18T04:25:31Z

torch_geometric/data/materialized_graph.py

+                "An edge layout is required to store an edge index, but one "
+                "was not provided.")
+
+        # NOTE implementations should take care to ensure that `SparseTensor`


What does that mean in this case?

I've updated this to work with Tuple[Tensor, Tensor] directly; lmkwyt!

torch_geometric/data/materialized_graph.py

test/data/test_data.py

rusty1s · 2022-06-19T16:25:15Z

torch_geometric/data/materialized_graph.py

+
+
+@dataclass
+class EdgeAttr(CastMixin):


This term is now a bit overloaded.

In what manner? I think it aligns well with TensorAttr in FeatureStore (they are both attributes of objects we are trying to add to the respective stores).

We usually refer to data.edge_attr for edge attributes/features.

Ah, right. I think it's probably okay to leave this overloaded term for now (mostly since I can't think of a better one), and if you have any ideas we can do a quick refactor.

torch_geometric/data/materialized_graph.py

for more information, see https://pre-commit.ci

…ch_geometric into feature_store_pt2

for more information, see https://pre-commit.ci

…ch_geometric into feature_store_pt2

for more information, see https://pre-commit.ci

…ch_geometric into feature_store_pt2

mananshah99 · 2022-06-21T01:15:51Z

@rusty1s @Padarn @wsad1: thank you for the feedback! This PR has been updated accordingly. A few notable changes:

MaterializedGraph is now named GraphStore
EdgeAttr has been cleaned up, and overridden in Data (just like TensorAttr)
We now support 'coo', 'csr', and 'csc' as arguments (instead of the full Enum definition)
GraphStore now directly operates on EdgeTensorType = Tuple[Tensor, Tensor], as suggested by @rusty1s. This alleviates issues with integrating SparseTensor and cleans up the implementation.
- Additionally, Data and HeteroData's implementations of GraphStore abstract methods are unified and quite simple. For put, based on the passed EdgeLayout, they construct a corresponding Tensor or SparseTensor and store that object internally. For get, based on the passed EdgeLayout, they construct a Tuple[Tensor, Tensor] and return this back to the caller.

Let me know what you think.

rusty1s

I am super impressed. This is clean.

torch_geometric/data/graph_store.py

rusty1s · 2022-06-21T13:36:37Z

torch_geometric/data/materialized_graph.py

+
+
+@dataclass
+class EdgeAttr(CastMixin):


We usually refer to data.edge_attr for edge attributes/features.

torch_geometric/data/graph_store.py

torch_geometric/data/data.py

torch_geometric/data/graph_store.py

torch_geometric/data/data.py

Padarn · 2022-07-03T08:01:08Z

Sorry I was really busy and didn't read your updates earlier @mananshah99 - looks awesome and thanks for your responses.

mananshah99 added 12 commits June 15, 2022 00:18

init

ff7066a

update

1688803

Merge branch 'master' of https://github.com/pyg-team/pytorch_geometric …

60181f2

…into feature_store_pt1

fix, todo: failing tests

59198b6

fix to_heterodata

82a10e0

CHANGELOG

996591f

fix

a2a099f

update

cdd1a72

fix

2991fbc

init

4038892

init 2

cc0b39c

lint

3918748

mananshah99 added feature 0 - Priority P0 labels Jun 17, 2022

mananshah99 self-assigned this Jun 17, 2022

mananshah99 mentioned this pull request Jun 17, 2022

[Roadmap] Remote Backend Support and Integration 🚀 #4806

Open

14 tasks

CHANGELOG

5824d90

mananshah99 added the data label Jun 17, 2022

fix

396c72d

mananshah99 marked this pull request as ready for review June 17, 2022 19:08

mananshah99 requested a review from rusty1s June 17, 2022 19:08

mananshah99 added 2 commits June 17, 2022 23:30

update materialized_graph

a46f60f

lint

403f03f

Padarn reviewed Jun 18, 2022

View reviewed changes

torch_geometric/data/hetero_data.py Outdated Show resolved Hide resolved

Padarn reviewed Jun 18, 2022

View reviewed changes

torch_geometric/data/materialized_graph.py Outdated Show resolved Hide resolved

Padarn reviewed Jun 18, 2022

View reviewed changes

torch_geometric/data/materialized_graph.py Outdated Show resolved Hide resolved

Padarn reviewed Jun 18, 2022

View reviewed changes

torch_geometric/data/materialized_graph.py Outdated Show resolved Hide resolved

Padarn reviewed Jun 18, 2022

View reviewed changes

torch_geometric/data/materialized_graph.py Outdated Show resolved Hide resolved

Padarn reviewed Jun 18, 2022

View reviewed changes

torch_geometric/data/materialized_graph.py Outdated Show resolved Hide resolved

wsad1 reviewed Jun 18, 2022

View reviewed changes

torch_geometric/data/materialized_graph.py Outdated Show resolved Hide resolved

rusty1s reviewed Jun 19, 2022

View reviewed changes

Base automatically changed from feature_store_pt1 to master June 20, 2022 17:20

mananshah99 and others added 13 commits June 20, 2022 17:33

merge

4750cb6

[pre-commit.ci] auto fixes from pre-commit.com hooks

aa46523

for more information, see https://pre-commit.ci

isort

a3f083e

Merge branch 'feature_store_pt2' of https://github.com/pyg-team/pytor…

04ee394

…ch_geometric into feature_store_pt2

MaterializedGraph -> GraphStore

385d8a8

[pre-commit.ci] auto fixes from pre-commit.com hooks

5e6badb

for more information, see https://pre-commit.ci

update

aa94d8c

Merge branch 'feature_store_pt2' of https://github.com/pyg-team/pytor…

d1e6ee0

…ch_geometric into feature_store_pt2

update

3cf6d95

Union[Tensor, Tensor]:

d857d5c

[pre-commit.ci] auto fixes from pre-commit.com hooks

b49728e

for more information, see https://pre-commit.ci

updates

653dfd8

Merge branch 'feature_store_pt2' of https://github.com/pyg-team/pytor…

92f82cb

…ch_geometric into feature_store_pt2

mananshah99 changed the title ~~MaterializedGraph definition + Data and HeteroData integration~~ GraphStore definition + Data and HeteroData integration Jun 21, 2022

adj, adj_t respect is_sorted

5c1d38e

rusty1s approved these changes Jun 21, 2022

View reviewed changes

rusty1s reviewed Jun 21, 2022

View reviewed changes

torch_geometric/data/graph_store.py Show resolved Hide resolved

rusty1s reviewed Jun 21, 2022

View reviewed changes

torch_geometric/data/data.py Show resolved Hide resolved

mananshah99 added 3 commits June 21, 2022 20:33

comments

acf6c4a

view

471dc5a

update

f8b8d9e

mananshah99 merged commit b274fbd into master Jun 21, 2022

mananshah99 deleted the feature_store_pt2 branch June 21, 2022 23:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`GraphStore` definition + `Data` and `HeteroData` integration #4816

`GraphStore` definition + `Data` and `HeteroData` integration #4816

mananshah99 commented Jun 17, 2022

codecov bot commented Jun 17, 2022 •

edited

Loading

Padarn Jun 18, 2022

mananshah99 Jun 21, 2022

rusty1s Jun 19, 2022

mananshah99 Jun 20, 2022

rusty1s Jun 21, 2022

mananshah99 Jun 21, 2022

mananshah99 commented Jun 21, 2022

rusty1s left a comment

rusty1s Jun 21, 2022

Padarn commented Jul 3, 2022



		@dataclass
		class EdgeAttr(CastMixin):



		@dataclass
		class EdgeAttr(CastMixin):

GraphStore definition + Data and HeteroData integration #4816

GraphStore definition + Data and HeteroData integration #4816

Conversation

mananshah99 commented Jun 17, 2022

codecov bot commented Jun 17, 2022 • edited Loading

Codecov Report

Padarn Jun 18, 2022

Choose a reason for hiding this comment

mananshah99 Jun 21, 2022

Choose a reason for hiding this comment

rusty1s Jun 19, 2022

Choose a reason for hiding this comment

mananshah99 Jun 20, 2022

Choose a reason for hiding this comment

rusty1s Jun 21, 2022

Choose a reason for hiding this comment

mananshah99 Jun 21, 2022

Choose a reason for hiding this comment

mananshah99 commented Jun 21, 2022

rusty1s left a comment

Choose a reason for hiding this comment

rusty1s Jun 21, 2022

Choose a reason for hiding this comment

Padarn commented Jul 3, 2022

`GraphStore` definition + `Data` and `HeteroData` integration #4816

`GraphStore` definition + `Data` and `HeteroData` integration #4816

codecov bot commented Jun 17, 2022 •

edited

Loading