JSON: InvalidOperationException by deleting entity with owned entities

[goodstas]

File a bug

I have Person entity which contains some complex property Details which contains list of complex property SubDetail .
I want to store property Details into the jsonb type column of table Person.

public class Person : ISoftDeleteEntity
 {
     public Guid Id { get; set; }

     public string Name { get; set; }

     public Details PersonalDetails { get; set; }

     public bool? IsDeleted { get; set; }

     public DateTime? DeletionTime { get; set; }
 }

public class Details
{
    public bool BoolDetail { get; set; }

    public string StrDetail { get; set; }

    public List<SubDetail> SubDetails { get; set; }
}

public class SubDetail
{
    public int IntSubDetail { get; set; }
    public string StrSubDetail { get; set; }

    public double DoubleSubDetail { get; set; }
}

I read that the it's still NOT possible to save complex property to Json column so i used OwnsOne and OwnsMany methods to map everything in my db context.
In addition i want to implement Soft Delete mechanism for the Person entity. The Person entity includes IsDeleted property and also implements interface ISoftDeleteEntity.
I also defined index for the Person with constraint of "IsDeleted = False". Because i'm interested to index only "alive" entities.

The db context definition for Person looks as follows :

modelBuilder.Entity<Person>(entity =>
{
    entity.ToTable("Person");
    entity.HasKey(_ => _.Id);
    entity.Property(e => e.Id)
          .HasDefaultValueSql("uuid_generate_v4()");

    entity.OwnsOne(e => e.PersonalDetails,
                   b =>
                   {
                       b.ToJson();
                       b.OwnsMany(e => e.SubDetails);
                   }
    );
});

modelBuilder.Entity<Person>().HasIndex(e => e.Id).HasFilter("\"IsDeleted\" = False").HasDatabaseName("PersonId_Index");

I overrides SaveChange method of DB Context to manipulate with the Person entity when the delete request is coming before actually implementing changes on DB.
What i do :
I change the state of the entity.entry to Modified.
I give 'True' value to IsDeleted property and call base.SaveChanges() .

After that i got an exception that the stack of it i included below.

I look for more information about this kind of exception and i found this link

I can not understand what i do in the code that makes efcore thinks that i try to change the owner of an owned entity.
The exception is about Subdetail.Id where Subdetail is owned by Details and Details is owned by Person.

I uploaded small project to reproduce the bug . Here is a link.
You will find the sln file inside WebApplication1 folder.

The scenario for reproducing after you configured appsettings with Postgres details , applied migration and ran the application.

1. Post the following json {"Id":"f97271fb-96c3-403f-ba62-14f6d5c76bdb","Name":"aaa","PersonalDetails":{"BoolDetail":true,"StrDetail":"asdasdada","SubDetails":[{"IntSubDetail":1,"StrSubDetail":"a","DoubleSubDetail":1},{"IntSubDetail":2,"StrSubDetail":"b","DoubleSubDetail":2},{"IntSubDetail":3,"StrSubDetail":"c","DoubleSubDetail":3}]},"IsDeleted":null,"DeletionTime":null}
2. try to delete the entity by id : f97271fb-96c3-403f-ba62-14f6d5c76bdb

Include stack traces

 An exception occurred in the database while saving changes for context type 'WebApplication1.AppDB.AppDbContext'.
      System.InvalidOperationException: The property 'SubDetail.Id' is part of a key and so cannot be modified or marked as modified. To change the principal of an existing entity with an identifying foreign key, first delete the dependent and invoke 'SaveChanges', and then associate the dependent with the new principal.
         at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.ChangeDetector.ThrowIfKeyChanged(InternalEntityEntry entry, IProperty property)
         at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.ChangeDetector.PropertyChanged(InternalEntityEntry entry, IPropertyBase propertyBase, Boolean setModified)
         at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.InternalEntityEntryNotifier.PropertyChanged(InternalEntityEntry entry, IPropertyBase property, Boolean setModified)
         at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.InternalEntityEntry.SetProperty(IPropertyBase propertyBase, Object value, Boolean isMaterialization, Boolean setModified, Boolean isCascadeDelete, CurrentValueType valueType)         at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.InternalEntityEntry.SetStoreGeneratedValue(IProperty property, Object value, Boolean setModified)
         at Microsoft.EntityFrameworkCore.Update.ModificationCommand.WriteJson(Utf8JsonWriter writer, Object navigationValue, IUpdateEntry parentEntry, IEntityType entityType, Nullable`1 ordinal, Boolean isCollection, Boolean isTopLevel)
         at Microsoft.EntityFrameworkCore.Update.ModificationCommand.WriteJson(Utf8JsonWriter writer, Object navigationValue, IUpdateEntry parentEntry, IEntityType entityType, Nullable`1 ordinal, Boolean isCollection, Boolean isTopLevel)
         at Microsoft.EntityFrameworkCore.Update.ModificationCommand.WriteJson(Utf8JsonWriter writer, Object navigationValue, IUpdateEntry parentEntry, IEntityType entityType, Nullable`1 ordinal, Boolean isCollection, Boolean isTopLevel)
         at Microsoft.EntityFrameworkCore.Update.ModificationCommand.<GenerateColumnModifications>g__HandleJson|41_4(List`1 columnModifications, <>c__DisplayClass41_0&)
         at Microsoft.EntityFrameworkCore.Update.ModificationCommand.GenerateColumnModifications()
         at Microsoft.EntityFrameworkCore.Update.ModificationCommand.<>c.<get_ColumnModifications>b__33_0(ModificationCommand command)
         at Microsoft.EntityFrameworkCore.Internal.NonCapturingLazyInitializer.EnsureInitialized[TParam,TValue](TValue& target, TParam param, Func`2 valueFactory)
         at Microsoft.EntityFrameworkCore.Update.ModificationCommand.get_ColumnModifications()
         at Microsoft.EntityFrameworkCore.Update.Internal.CommandBatchPreparer.CreateCommandBatches(IEnumerable`1 commandSet, Boolean moreCommandSets, Boolean assertColumnModification, ParameterNameGenerator parameterNameGenerator)+MoveNext()
         at Microsoft.EntityFrameworkCore.Update.Internal.CommandBatchPreparer.BatchCommands(IList`1 entries, IUpdateAdapter updateAdapter)+MoveNext()
         at Microsoft.EntityFrameworkCore.Update.Internal.BatchExecutor.Execute(IEnumerable`1 commandBatches, IRelationalConnection connection)
         at Microsoft.EntityFrameworkCore.Storage.RelationalDatabase.SaveChanges(IList`1 entries)
         at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChanges(IList`1 entriesToSave)
         at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChanges(StateManager stateManager, Boolean acceptAllChangesOnSuccess)
         at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.<>c.<SaveChanges>b__112_0(DbContext _, ValueTuple`2 t)
         at Npgsql.EntityFrameworkCore.PostgreSQL.Storage.Internal.NpgsqlExecutionStrategy.Execute[TState,TResult](TState state, Func`3 operation, Func`3 verifySucceeded)
         at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChanges(Boolean acceptAllChangesOnSuccess)
         at Microsoft.EntityFrameworkCore.DbContext.SaveChanges(Boolean acceptAllChangesOnSuccess)
      System.InvalidOperationException: The property 'SubDetail.Id' is part of a key and so cannot be modified or marked as modified. To change the principal of an existing entity with an identifying foreign key, first delete the dependent and invoke 'SaveChanges', and then associate the dependent with the new principal.
         at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.ChangeDetector.ThrowIfKeyChanged(InternalEntityEntry entry, IProperty property)
         at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.ChangeDetector.PropertyChanged(InternalEntityEntry entry, IPropertyBase propertyBase, Boolean setModified)
         at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.InternalEntityEntryNotifier.PropertyChanged(InternalEntityEntry entry, IPropertyBase property, Boolean setModified)
         at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.InternalEntityEntry.SetProperty(IPropertyBase propertyBase, Object value, Boolean isMaterialization, Boolean setModified, Boolean isCascadeDelete, CurrentValueType valueType)         at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.InternalEntityEntry.SetStoreGeneratedValue(IProperty property, Object value, Boolean setModified)
         at Microsoft.EntityFrameworkCore.Update.ModificationCommand.WriteJson(Utf8JsonWriter writer, Object navigationValue, IUpdateEntry parentEntry, IEntityType entityType, Nullable`1 ordinal, Boolean isCollection, Boolean isTopLevel)
         at Microsoft.EntityFrameworkCore.Update.ModificationCommand.WriteJson(Utf8JsonWriter writer, Object navigationValue, IUpdateEntry parentEntry, IEntityType entityType, Nullable`1 ordinal, Boolean isCollection, Boolean isTopLevel)
         at Microsoft.EntityFrameworkCore.Update.ModificationCommand.WriteJson(Utf8JsonWriter writer, Object navigationValue, IUpdateEntry parentEntry, IEntityType entityType, Nullable`1 ordinal, Boolean isCollection, Boolean isTopLevel)
         at Microsoft.EntityFrameworkCore.Update.ModificationCommand.<GenerateColumnModifications>g__HandleJson|41_4(List`1 columnModifications, <>c__DisplayClass41_0&)
         at Microsoft.EntityFrameworkCore.Update.ModificationCommand.GenerateColumnModifications()
         at Microsoft.EntityFrameworkCore.Update.ModificationCommand.<>c.<get_ColumnModifications>b__33_0(ModificationCommand command)
         at Microsoft.EntityFrameworkCore.Internal.NonCapturingLazyInitializer.EnsureInitialized[TParam,TValue](TValue& target, TParam param, Func`2 valueFactory)
         at Microsoft.EntityFrameworkCore.Update.ModificationCommand.get_ColumnModifications()
         at Microsoft.EntityFrameworkCore.Update.Internal.CommandBatchPreparer.CreateCommandBatches(IEnumerable`1 commandSet, Boolean moreCommandSets, Boolean assertColumnModification, ParameterNameGenerator parameterNameGenerator)+MoveNext()
         at Microsoft.EntityFrameworkCore.Update.Internal.CommandBatchPreparer.BatchCommands(IList`1 entries, IUpdateAdapter updateAdapter)+MoveNext()
         at Microsoft.EntityFrameworkCore.Update.Internal.BatchExecutor.Execute(IEnumerable`1 commandBatches, IRelationalConnection connection)
         at Microsoft.EntityFrameworkCore.Storage.RelationalDatabase.SaveChanges(IList`1 entries)
         at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChanges(IList`1 entriesToSave)
         at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChanges(StateManager stateManager, Boolean acceptAllChangesOnSuccess)
         at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.<>c.<SaveChanges>b__112_0(DbContext _, ValueTuple`2 t)
         at Npgsql.EntityFrameworkCore.PostgreSQL.Storage.Internal.NpgsqlExecutionStrategy.Execute[TState,TResult](TState state, Func`3 operation, Func`3 verifySucceeded)
         at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChanges(Boolean acceptAllChangesOnSuccess)
         at Microsoft.EntityFrameworkCore.DbContext.SaveChanges(Boolean acceptAllChangesOnSuccess)
fail: Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware[1]
      An unhandled exception has occurred while executing the request.
      System.InvalidOperationException: The property 'SubDetail.Id' is part of a key and so cannot be modified or marked as modified. To change the principal of an existing entity with an identifying foreign key, first delete the dependent and invoke 'SaveChanges', and then associate the dependent with the new principal.
         at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.ChangeDetector.ThrowIfKeyChanged(InternalEntityEntry entry, IProperty property)
         at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.ChangeDetector.PropertyChanged(InternalEntityEntry entry, IPropertyBase propertyBase, Boolean setModified)
         at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.InternalEntityEntryNotifier.PropertyChanged(InternalEntityEntry entry, IPropertyBase property, Boolean setModified)
         at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.InternalEntityEntry.SetProperty(IPropertyBase propertyBase, Object value, Boolean isMaterialization, Boolean setModified, Boolean isCascadeDelete, CurrentValueType valueType)         at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.InternalEntityEntry.SetStoreGeneratedValue(IProperty property, Object value, Boolean setModified)
         at Microsoft.EntityFrameworkCore.Update.ModificationCommand.WriteJson(Utf8JsonWriter writer, Object navigationValue, IUpdateEntry parentEntry, IEntityType entityType, Nullable`1 ordinal, Boolean isCollection, Boolean isTopLevel)
         at Microsoft.EntityFrameworkCore.Update.ModificationCommand.WriteJson(Utf8JsonWriter writer, Object navigationValue, IUpdateEntry parentEntry, IEntityType entityType, Nullable`1 ordinal, Boolean isCollection, Boolean isTopLevel)
         at Microsoft.EntityFrameworkCore.Update.ModificationCommand.WriteJson(Utf8JsonWriter writer, Object navigationValue, IUpdateEntry parentEntry, IEntityType entityType, Nullable`1 ordinal, Boolean isCollection, Boolean isTopLevel)
         at Microsoft.EntityFrameworkCore.Update.ModificationCommand.<GenerateColumnModifications>g__HandleJson|41_4(List`1 columnModifications, <>c__DisplayClass41_0&)
         at Microsoft.EntityFrameworkCore.Update.ModificationCommand.GenerateColumnModifications()
         at Microsoft.EntityFrameworkCore.Update.ModificationCommand.<>c.<get_ColumnModifications>b__33_0(ModificationCommand command)
         at Microsoft.EntityFrameworkCore.Internal.NonCapturingLazyInitializer.EnsureInitialized[TParam,TValue](TValue& target, TParam param, Func`2 valueFactory)
         at Microsoft.EntityFrameworkCore.Update.ModificationCommand.get_ColumnModifications()
         at Microsoft.EntityFrameworkCore.Update.Internal.CommandBatchPreparer.CreateCommandBatches(IEnumerable`1 commandSet, Boolean moreCommandSets, Boolean assertColumnModification, ParameterNameGenerator parameterNameGenerator)+MoveNext()
         at Microsoft.EntityFrameworkCore.Update.Internal.CommandBatchPreparer.BatchCommands(IList`1 entries, IUpdateAdapter updateAdapter)+MoveNext()
         at Microsoft.EntityFrameworkCore.Update.Internal.BatchExecutor.Execute(IEnumerable`1 commandBatches, IRelationalConnection connection)
         at Microsoft.EntityFrameworkCore.Storage.RelationalDatabase.SaveChanges(IList`1 entries)
         at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChanges(IList`1 entriesToSave)
         at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChanges(StateManager stateManager, Boolean acceptAllChangesOnSuccess)
         at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.<>c.<SaveChanges>b__112_0(DbContext _, ValueTuple`2 t)
         at Npgsql.EntityFrameworkCore.PostgreSQL.Storage.Internal.NpgsqlExecutionStrategy.Execute[TState,TResult](TState state, Func`3 operation, Func`3 verifySucceeded)
         at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChanges(Boolean acceptAllChangesOnSuccess)
         at Microsoft.EntityFrameworkCore.DbContext.SaveChanges(Boolean acceptAllChangesOnSuccess)
         at Microsoft.EntityFrameworkCore.DbContext.SaveChanges()
         at WebApplication1.AppDB.AppDbContext.SaveChanges() in C:\GIT\EfCore.Owns.Json.Exception\WebApplication1\AppDB\AppDBContext.cs:line 53
         at WebApplication1.Controllers.PersonController.DeletePerson(Guid id) in C:\GIT\EfCore.Owns.Json.Exception\WebApplication1\Controllers\PersonController.cs:line 53
         at lambda_method139(Closure, Object, Object[])
         at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.SyncActionResultExecutor.Execute(ActionContext actionContext, IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments)
         at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeActionMethodAsync()
         at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
         at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeNextActionFilterAsync()
      --- End of stack trace from previous location ---
         at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context)         at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
         at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeInnerFilterAsync()
      --- End of stack trace from previous location ---
         at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeFilterPipelineAsync>g__Awaited|20_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
         at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
         at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
         at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
         at Swashbuckle.AspNetCore.SwaggerUI.SwaggerUIMiddleware.Invoke(HttpContext httpContext)
         at Swashbuckle.AspNetCore.Swagger.SwaggerMiddleware.Invoke(HttpContext httpContext, ISwaggerProvider swaggerProvider)
         at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
         at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddlewareImpl.Invoke(HttpContext context)

###My Debug
I called ChangeTracker.ToDebugString() method and this is what i got after changing the state of the Person entry to Modified :
Details {PersonId: aab817dd-262e-46cd-a3a5-9c39dbf6b166} Deleted
PersonId: 'aab817dd-262e-46cd-a3a5-9c39dbf6b166' PK FK
BoolDetail: 'True'
StrDetail: 'asdasdada'
SubDetails: [{DetailsPersonId: aab817dd-262e-46cd-a3a5-9c39dbf6b166, Id: 1}, {DetailsPersonId: aab817dd-262e-46cd-a3a5-9c39dbf6b166, Id: 2}, {DetailsPersonId: aab817dd-262e-46cd-a3a5-9c39dbf6b166, Id: 3}]
Person {Id: aab817dd-262e-46cd-a3a5-9c39dbf6b166} Modified
Id: 'aab817dd-262e-46cd-a3a5-9c39dbf6b166' PK
DeletionTime: '29/08/2024 15:16:10' Modified Originally
IsDeleted: 'True' Modified Originally
Name: 'aaa' Modified
PersonalDetails: {PersonId: aab817dd-262e-46cd-a3a5-9c39dbf6b166}
SubDetail {DetailsPersonId: aab817dd-262e-46cd-a3a5-9c39dbf6b166, Id: 1} Deleted
DetailsPersonId: 'aab817dd-262e-46cd-a3a5-9c39dbf6b166' PK FK
Id: 1 PK
DoubleSubDetail: 1
IntSubDetail: 1
StrSubDetail: 'a'
SubDetail {DetailsPersonId: aab817dd-262e-46cd-a3a5-9c39dbf6b166, Id: 2} Deleted
DetailsPersonId: 'aab817dd-262e-46cd-a3a5-9c39dbf6b166' PK FK
Id: 2 PK
DoubleSubDetail: 2
IntSubDetail: 2
StrSubDetail: 'b'
SubDetail {DetailsPersonId: aab817dd-262e-46cd-a3a5-9c39dbf6b166, Id: 3} Deleted
DetailsPersonId: 'aab817dd-262e-46cd-a3a5-9c39dbf6b166' PK FK
Id: 3 PK
DoubleSubDetail: 3
IntSubDetail: 3
StrSubDetail: 'c'

So i guess that the issue is that Subdetail and Details entries states remained in Deleted State?
But actually i don't need that ChangeTracker will track these objects in such way. They are going to be stored as json.

I found a solution. I don't know if it's a good one or walk-around.
I defined some interface IDetachEntity and just change the state of all of these entities from Deleted to Detached.
Now Details and Subdetail class inherit from this interface and i do the following in my code :
` var detachDeleteEntities = this.ChangeTracker
.Entries()
.Where(e => e.State == Microsoft.EntityFrameworkCore.EntityState.Deleted);

foreach (var entity in detachDeleteEntities)
{
entity.State = Microsoft.EntityFrameworkCore.EntityState.Detached;
}
`
After that i don't get an exception.. and things work as expected.

Include provider and version information

EF Core version: 8.0.8
Database provider: Npgsql.EntityFrameworkCore.PostgreSQL 8.0.4
Target framework: NET 8.0
Operating system: Windows 11
IDE: ( Visual Studio 2022 17.9.4)

@roji , i will appreciate your comment for this issue.

[ajcvickers]

@goodstas Sorry for being so slow responding here.

I found a solution. I don't know if it's a good one or walk-around. I defined some interface IDetachEntity and just change the state of all of these entities from Deleted to Detached.

This is a reasonable solution. Owned types are not a great fit for this, and JSON mapping from complex types should not have this issue. But for now, state of the owned types also needs to be changed to match that of the owner.

[goodstas]

@ajcvickers So if you say that "Owned types are not a great fit for this" then what i should use?
Should i define JSONB column for POCO class?
I did what i did because i read here that
"Version 8.0 of the Npgsql provider introduced support for EF's JSON columns, using ToJson(). That is the recommended way to map POCOs going forward."
I will appreciate your response to my question.

[ajcvickers]

@goodstas As far as EF Core as a whole is concerned, nothing that is a good fit exists at this point. @roji can comment about whether there are any additional options available when using Npgsql.

[goodstas]

@ajcvickers , @roji : if there is noithing what fits good, may be you can suggest what to change to achieve the same result?
Thank u!