Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add support for custom template databases #194

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

munjalpatel
Copy link

Description

This PR introduces a new flag, --template-db, allowing users to specify the template database used when creating a temporary database. Previously, template0 was hard-coded, limiting the ability to validate migrations in scenarios requiring a different template DB.

Motivation

We frequently use various Postgres extensions that rely on different template databases for initialization, and we want to leverage migration validation against a temporary database. The prior hard-coded template0 prevented using migration validation with those extensions. By making the template configurable, we can accommodate more specialized Postgres environments and maintain robust validation flows.

Testing

  • Added unit tests to validate the new --template-db flag.
  • Verified downstream acceptance tests pass without regressions.
  • Confirmed correct inheritance of objects from the user-specified template DB in the newly created temporary database.

Copy link

cla-assistant bot commented Mar 2, 2025

CLA assistant check
All committers have signed the CLA.

Copy link
Collaborator

@bplunkett-stripe bplunkett-stripe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks really solid to me! Some small suggestions. Thanks for making these changes!

@@ -32,11 +33,11 @@ func TestParseTimeoutModifierStr(t *testing.T) {
},
{
opt: "timeout=15m",
expectedErrContains: "could not find key",
expectedErrContains: "could not find key", // "pattern" missing
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: can we remove these. I understand why they were included, but I prefer not to have these comments, since they might quickly become outdated

@@ -118,3 +119,34 @@ func TestParseInsertStatementStr(t *testing.T) {
})
}
}

func TestPlanFlagsTemplateDBDefault(t *testing.T) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's probably okay to not include these, since we don't have analog tests. The functionality it is testing is pretty straightforward too, i.e., flag parsing

metadataTable: DefaultOnInstanceMetadataTable,
rootDatabase: "postgres",
logger: log.SimpleLogger(),
templateDatabase: DefaultTemplateDatabase,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of making this a parameter of the factory, let's make this a parameter of the Create function, since one factory doesn't need to be restricted to one type of template

&flags.templateDb,
"template-db",
"template0",
"Template database to use when creating temporary databases",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe add to this doc string: "This is used to derive implicit schema objects for the target schema"

@@ -173,6 +177,13 @@ func createPlanFlags(cmd *cobra.Command) *planFlags {

cmd.Flags().Var(&flags.outputFormat, "output-format", "Change the output format for what is printed. Defaults to pretty-printed human-readable output. (options: pretty, json)")

cmd.Flags().StringVar(
&flags.templateDb,
"template-db",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: name schema-template-db

@munjalpatel
Copy link
Author

@bplunkett-stripe Thanks for the suggestions on this PR! I’m happy to incorporate improvements.

However, in my testing, I noticed an issue: when we create a new database from a template that already has extensions preloaded, and our SQL migrations don’t include CREATE EXTENSION … statements, pg-schema-diff tries to remove those extensions.

Because of that, I’m not sure how much value this PR provides unless there’s a more straightforward way to ignore everything inherited from the template database (tables, functions, extensions, etc.). Do you have any thoughts on how we might address this?

@bplunkett-stripe
Copy link
Collaborator

bplunkett-stripe commented Mar 5, 2025

@bplunkett-stripe Thanks for the suggestions on this PR! I’m happy to incorporate improvements.

However, in my testing, I noticed an issue: when we create a new database from a template that already has extensions preloaded, and our SQL migrations don’t include CREATE EXTENSION … statements, pg-schema-diff tries to remove those extensions.

Because of that, I’m not sure how much value this PR provides unless there’s a more straightforward way to ignore everything inherited from the template database (tables, functions, extensions, etc.). Do you have any thoughts on how we might address this?

Hmmm I'm not sure why that would be occurring. @munjalpatel, could you:

  • pg_dump your template database (schema only)
  • pg_dump your current database created from the template (schema only)
  • Send your target ddl
  • Send the command you're running and it's output

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants