LiteFS Cloud is being sunset next month

[MoSattler]

Fly io is sunsetting LiteFS Cloud:

We’ve decided to wind it down. LiteFS Cloud will continue to be available until October 15, 2024. - Fly.io blog

This might require some updates in the readme, like here:

LiteFS Cloud is a service offered by Fly.io for managing backup and restore functionality. - Epic-Stack Database Docs

[kentcdodds]

Thanks for opening this. I need to evaluate the options and decide which to go with. Got tied up with the Epic React v2 launch. Hopefully I'll have an answer (and maybe migration guide) for what's next before the sunset date.

I'm definitely open to suggestions, especially from folks who have tried alternatives and liked them.

[MoSattler]

@benbjohnson made some recommendations:

Disaster Recovery Without LiteFS Cloud

There are two good options.

LiteFS + Litestream

You can use Litestream to maintain your own backups to an S3-compatible object storage such as Tigris. That’s what we do internally. If you’re running a static primary, you can point Litestream at your primary node in your cluster.

Turso

If you’re interested in running a fully-managed SQLite service, we like Turso. You can find more information on their Quick Start page.

[buildtheui]

Hi everyone, I wanted to share my experience migrating to static leasing with Litestream and Tigris. Although I'm not using the Epic Stack, I was heavily inspired by it to build my backend in Golang using LiteFS and LiteFS Cloud. Since the concepts are similar, I thought it would be helpful to share my progress.

I followed the short migration guide from Fly.io and encountered a few challenges along the way. I’ve documented the issues here. However, I’ve hit a roadblock: I can't figure out how to set the lease.hostname to the primary machine ID for all nodes (both primary and replicas).

I’m not a Fly.io expert, so if anyone has a better approach or suggestions, this would help everyone.

[buildtheui]

As another option, @stephen built a drop-in solution for Litefs cloud, more details here

[stephen]

👋 litefs-backup is a stripped down fork of the original LiteFS Cloud implementation, so you're self-hosting your backup solution.

My primary motivation was to continue to have support for LiteFS' dynamic/consul leasing.

[MoSattler]

Looks like so far LiteFS Cloud is still available 🤔

[MrSucik]

some functionalities are shut down already

[MoSattler]

Ah really? which ones?

[timothymalcham]

Just checking to see if anyone has used litefs-backup or another solution successfully with the Epic Stack? Did it work as a drop-in replacement, any issues? I've got a couple client projects built with the Epic Stack and would like to implement something automated for backups.

[MoSattler]

I have bitten the bullet and migrated to Postgres :(

[davidesigner]

Hi everyone!

Have any of you implemented Turso in the Epic Stack?

I found this example: https://github.com/tursodatabase/epic-stack made by @giovannibenussi, but I'm not sure I understand the migration process:

• In the local environment: I see: cat prisma/migrations/20230914194400_init/migration.sql | sqlite3 ./data.db, so if I understand, the migration applied by Prisma when we do prisma migrate reset is not applied to the local database?
• In production environment: I see turso db shell epic-stack-example < prisma/migrations/20230914194400_init/migration.sql, so we need to manually apply migrations to the main Turso database in production? Maybe we should create a service that automatically applies migrations during the deployment step in Github Actions?

I have other questions too, is LiftFS still recommended when using Turso?

Any help would be appreciated, thanks 🙏

[giovannibenussi]

Hey, Giovanni from Turso here 🙋

• You're right, prisma migrate reset is not applied to the local database so you'd need to do this manually.
• Same, you need to do this manually in production.

We need to coordinate with the Prisma team to improve this whole process.

Regarding LiteFS, you don't need it when using Turso. You can create a Turso account and don't worry about servers anymore.
Also, if you're on an environment with access to the file-system (like Fly), you can use Embedded Replicas to replicate your database in your server to enable microsecond-level read operations.

Let me know if you need help with anything else or if there's anything that we can improve 😄

[davidesigner]

Hi Giovanni, thanks for your reply.

Are you planning to improve the sync with Prisma in the near future? If so, when?

I certainly won't be using Turso until it's implemented, because I couldn't just have a smooth deployment with no downtime.

Without it, I'd have to develop a migration management overlay...

I think Turso is definitely a good alternative if it can be easily implemented in the Epic stack.

Have a nice day,
David

[giovannibenussi]

Hey, I no longer work at Turso so I'm not sure about timelines. I'd ask in their Discord to get a better answer!

[kentcdodds]

I still need to add a built-in solution for this, but now that we have Tigris support built-into the Epic Stack, we could probably use litestream to handle auto-backups. I think @devneill has done something similar to s3.

[devneill]

I'll take a look at this early next week and see if I can help 🧐

[devneill]

Hopefully this can help.

I managed to add backups via Github actions and Tigris.

Github action to trigger the backup daily:
https://github.com/devneill/getfranki-08-24/blob/main/.github/workflows/daily-backup.yml

Resource endpoint in my app to handle the request:
https://github.com/devneill/getfranki-08-24/blob/main/app/routes/api%2B/backup.ts

The endpoints calls the backDB() function in here:
https://github.com/devneill/getfranki-08-24/blob/main/app/utils/db.server.ts

Which calls this script:
https://github.com/devneill/getfranki-08-24/blob/main/other/backup_db.sh

That script makes a copy of the db and sends it to an s3 bucket, which is in Tigris.

---

I will open a PR to add a better version of this to the Epic Stack once I have more time (after the Epic Web Conference.)

[kentcdodds]

This is awesome. Thank you for sharing @devneill!

[taylrd]

Thank you for sharing @devneill :)