This issue is stale because it has been open 120 days with no activity. Remove stale label or comment or this will be closed in 10 days.

This issue is stale because it has been open 120 days with no activity. Remove stale label or comment or this will be closed in 10 days.

This issue is stale because it has been open 120 days with no activity. Remove stale label or comment or this will be closed in 10 days.

@cblmemo just wondering if this is still relevant

@cblmemo just wondering if this is still relevant

Yes

@cblmemo just wondering if this is still relevant

Yes

Could I try to work on this?

@cblmemo just wondering if this is still relevant

Yes

Could I try to work on this?

Sounds great! cc @Michaelvll for a look here

Perfect. @cblmemo if you could assign this issue to me, I'll get started starting this weekend

Perfect. @cblmemo if you could assign this issue to me, I'll get started starting this weekend

Done

@cblmemo
ok so I had the opportunity to look through the SkyServe code over the weekend. This is my general plan on how to implement this issue. If this general idea looks good, then I'll go ahead with the implementation and submit a draft PR with more details on the specs:

• The caching will be done by adding a new column to the serve_state sqlite3 database (under ~/.sky/serve/service.db). This will be named "endpoint", default to None, and will be populated by the "initial" query for status. Though, considering that the sky api server is moving to a client server architecture, api server dbs may not be local anymore, so it might be a better idea to actually create a new, local database (maybe under ~/.sky/serve/cache?) to just cache the state instead of querying the api server. I would appreciate feedback on this part as my understanding is that the local sky api server is the one currently storing state data under the ~/.sky folder
• I am assuming that we want caching only for the --endpoint flag (correct me if I am wrong), so when we receive such a request then we will check the cache for the endpoint, and if it exists, return, else, query and store.
• I am also thinking of adding a --force-refresh (can be renamed) flag so that users have the choice to force a re-query to the ground truth skyserve controller.

@cblmemo ok so I had the opportunity to look through the SkyServe code over the weekend. This is my general plan on how to implement this issue. If this general idea looks good, then I'll go ahead with the implementation and submit a draft PR with more details on the specs:

• The caching will be done by adding a new column to the serve_state sqlite3 database (under ~/.sky/serve/service.db). This will be named "endpoint", default to None, and will be populated by the "initial" query for status. Though, considering that the sky api server is moving to a client server architecture, api server dbs may not be local anymore, so it might be a better idea to actually create a new, local database (maybe under ~/.sky/serve/cache?) to just cache the state instead of querying the api server. I would appreciate feedback on this part as my understanding is that the local sky api server is the one currently storing state data under the ~/.sky folder
• I am assuming that we want caching only for the --endpoint flag (correct me if I am wrong), so when we receive such a request then we will check the cache for the endpoint, and if it exists, return, else, query and store.
• I am also thinking of adding a --force-refresh (can be renamed) flag so that users have the choice to force a re-query to the ground truth skyserve controller.

• I think putting that into the service db on the api server makes sense since this cache should be shared by all users for this api server.
• --force-refresh looks good to me.

cc @Michaelvll for a look here