outstanding issues with dynamic refactor

[kernel-io]

As discussed on Discord here are the outstanding issues I am encountering in my application while using the dynamic refactor of ash postgres.

1. Inability to use DB enums as Ash explicitly casts the bindings to a varchar instead of just leaving it uncast - i.e: SELECT * FROM burgers WHERE status = $1::varchar (instead of just = $1) - also the storage_type override on Ash.Type has no effect on this.

2. Ash gets confused while trying to load an aggregate that has a filter on it
   count :total_leased_count, :assets, filter: expr(is_nil(active_order_service.id) == false)

In my case I have AssetType which has many Assets, which in turn has one active OrderService (has_one :active_order_service, OrderService, filter: expr(fragment("? @> now()", period))), I have a count aggregate defined on AssetType which counts the number of Assets with an active OrderService, OrderService has a range type column called period.

error is unknown error: {%Ecto.SubQueryError{exception: %Ecto.QueryError{message: \"could not find named binding 'as(nil)') while trying to load the AssetType with the count

3. I have previously needed to define the filter on the aggregate and relationship for it to work correctly, I can't tell if this has been fixed as point 2 is blowing up first.

4. There was an issue with GROUP BY on aggregates, which I think has been resolved, but I can't fully test because of 2.

[kernel-io]

update: Enums are still not fixed, however the query builder seems to be respecting the storage_type callback now at least. Postgres still complaining though, so maybe the fix is to not try cast it to anything and just let it be a string literal.

I'm also having issues with trying to load the aggregate through a relationship
"unknown error: {%RuntimeError{message: \"Error while building reference: active_order_service.id\"}, [{AshPostgres.Expr, :do_dynamic_expr, 5, [file: 'lib/expr.ex', line: 562, error_info: %{module: Exception}]}, {AshPostgres.Expr, :do_dynamic_expr, 5, [file: 'lib/expr.ex', line: 75]}, {AshPostgres.Expr, :do_dynamic_expr, 5, [file: 'lib/expr.ex', line: 325]}, {AshPostgres.Aggregate, :add_subquery_aggregate_select, 3, [file: 'lib/aggregate.ex', line: 448]}, {AshPostgres.Aggregate, :\"-add_aggregate_to_subquery/4-fun-4-\", 3, [file: 'lib/aggregate.ex', line: 289]}, {List, :do_update_at, 3, [file: 'lib/list.ex', line: 1245]}, {AshPostgres.Aggregate, :add_aggregate_to_subquery, 4, [file: 'lib/aggregate.ex', line: 266]}, {AshPostgres.Aggregate, :\"-add_aggregates/4-fun-4-\", 4, [file: 'lib/aggregate.ex', line: 110]}, {Enumerable.List, :reduce, 3, [file: 'lib/enum.ex', line: 4474]}, {Enum, :reduce_while, 3, [file: 'lib/enum.ex', line: 2442]}, {AshPostgres.Aggregate, :add_aggregates, 4, [file: 'lib/aggregate.ex', line: 15]}, {Ash.Query.Aggregate, :\"-value_request/7-fun-6-\", 5, [file: 'lib/ash/query/aggregate.ex', line: 338]}, {Ash.Engine.Request, :do_try_resolve_local, 4, [file: 'lib/ash/engine/request.ex', line: 962]}, {Ash.Engine.Request, :do_next, 1, [file: 'lib/ash/engine/request.ex', line: 288]}, {Ash.Engine.Request, :next, 1, [file: 'lib/ash/engine/request.ex', line: 206]}, {Ash.Engine.RequestHandler, :handle_continue, 2, [file: 'lib/ash/engine/request_handler.ex', line: 56]}, {:gen_server, :try_dispatch, 4, [file: 'gen_server.erl', line: 695]}, {:gen_server, :loop, 7, [file: 'gen_server.erl', line: 437]}, {:proc_lib, :init_p_do_apply, 3, [file: 'proc_lib.erl', line: 226]}]}"

this is when I try to get the count of leased assets from asset type.

i.e: On AssetType
count :total_leased_count, :assets, filter: expr(is_nil(active_order_service.id) == false)

Asset
has_one :active_order_service, OrderService, filter: expr(fragment("? @> now()", period))

[zachdaniel]

@kernel-io is there a new error message for the storage_type issue?

[kernel-io]

postgres saying the type doesn't exist (when it does)

[kernel-io]

okay, so I have my custom types defined in each tenant's schema, the issue is that we aren't qualifying which schema the custom type is defined in, so postgres is looking for it in the db roles search_path, which doesn't contain that custom type. would it be easy to specify the schema the type is in? (i.e: prefix().my_status_type) or something.

[zachdaniel]

I think the primary thing we could do here is to have a callback like def cast_in_query?(), do: false on the type. Alternatively, we could have ash_postgres set the search path before running a query to include the tenant, but I'm not as big of a fan of that option. We could potentially tell ash to scope the type to a given prefix, but that also comes with issues, because its not necessarily given that a query is entirely in a single tenant's schema. It may also be partially using things from the main schema, and potentially from other tenants (in the future, not right now).

[zachdaniel]

Okay, the latest master has that fix, so you can add def cast_in_query?, do: false to your enum types.

[kernel-io]

all sorted :)