You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
atomic-server does a lot of async stuff, as an Actix project.
However, atomic-lib does not. @AlexMikhalev recently added paralellization to Db::all_resources, which helped speed it up. There are probably other places, too, where it might help to spin up some threads.
I'm not sure where we can expect to gain a lot by paralellization, but I expect that Db::query and Commit::apply_opts as costly functions that do a lot of loopy stuff could speed up quite a bit.
The text was updated successfully, but these errors were encountered:
We have to rethink async and parallelisation in connection with the discussion #433 - Sled:IVec doesn't have Sync implemented so can't scale reads via Arc, we can use https://crates.io/crates/dashmap as cache, serialize it to disk using serde and then at some point migrate from sled if or when we need to. I also saw the news that sled is sunsetting, but I haven't seen confirmation.
For parallel we need to make a choice: are we going to add tokio as a dependency for atomic-data-rust or we need to make calls to parallelized functions - like all_resources sync from atomic-data-server. You can't mix both - tokio async and any other threadpool executors.
It's possible to use tokio for CPU-bound tasks, see here and here but it requires dedicated effort.
atomic-server
does a lot of async stuff, as an Actix project.However,
atomic-lib
does not. @AlexMikhalev recently added paralellization toDb::all_resources
, which helped speed it up. There are probably other places, too, where it might help to spin up some threads.I'm not sure where we can expect to gain a lot by paralellization, but I expect that
Db::query
andCommit::apply_opts
as costly functions that do a lot of loopy stuff could speed up quite a bit.The text was updated successfully, but these errors were encountered: