You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Since Atomic Data uses a lot of data fetching, some calls might take a while. Sometimes, multiple resources will need to be fetched at once. Instead of executing each request serially, having an parallelized async process for this would minimize execution time. Currently, every call is blocking in atomic_lib, and the HTTP library ureq is sync, too. Perhaps this will cause issues, but we can change ureq for some other HTTP library.
I'm kind of new to rust, and even more so to async programming, so I need to write down my thoughts on this. First, let's identify which processes might have the highest 'blocking' time. E.g. serializing internal AD3 to JSON. This requires all properties to be fetched, and these might not have been loaded yet.
Now, if we'd start implementing async, a lot needs to happen:
ureq has to be replaced by some async HTTP(S) library, preferably with rustls support. Maybe reqwest or surf?
a lot of the current functions (serializers that depend on store, get_resource) will have to become async, which will also 'infect' all functions that might use these.
the Storelike trait (and perhaps more) need to use async_trait
The text was updated successfully, but these errors were encountered:
I'm not entirely sure whether async is the way to go for this library. Sure, some things could made faster using async (fetching multiple resources, for example), but it comes at a price: runtime overhead.
Since Atomic Data uses a lot of data fetching, some calls might take a while. Sometimes, multiple resources will need to be fetched at once. Instead of executing each request serially, having an parallelized async process for this would minimize execution time. Currently, every call is blocking in
atomic_lib
, and the HTTP libraryureq
is sync, too. Perhaps this will cause issues, but we can changeureq
for some other HTTP library.I'm kind of new to rust, and even more so to async programming, so I need to write down my thoughts on this. First, let's identify which processes might have the highest 'blocking' time. E.g. serializing internal AD3 to JSON. This requires all properties to be fetched, and these might not have been loaded yet.
Now, if we'd start implementing async, a lot needs to happen:
The text was updated successfully, but these errors were encountered: