Fix to #30604 - Implement JSON serialization/deserialization via Utf8JsonReader/Utf8JsonWriter #31160
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@roji @ajcvickers this is ready for review now, there are still some minor cleanups to be done but overall logic is complete. Good luck ;) |
maumar
commented
Jun 29, 2023
maumar
commented
Jun 29, 2023
test/EFCore.Relational.Specification.Tests/Query/JsonQueryAdHocTestBase.cs
Outdated
Show resolved
Hide resolved
maumar
commented
Jun 29, 2023
...l/Query/RelationalShapedQueryCompilingExpressionVisitor.ShaperProcessingExpressionVisitor.cs
Outdated
Show resolved
Hide resolved
maumar
commented
Jun 29, 2023
...l/Query/RelationalShapedQueryCompilingExpressionVisitor.ShaperProcessingExpressionVisitor.cs
Outdated
Show resolved
Hide resolved
maumar
force-pushed
the
fix31159
branch
3 times, most recently
from
4102a94 to
928ed3a
Compare
maumar
changed the title
maumar
commented
Jul 1, 2023
|
I'll be reviewing this in the coming days for sure.
These would actually be useful as doc comments in the source code too (where necessary/reasonable). |
|
I added fragments that are generated here and there - will add more. But I think the entire output is not feasible as comments in the code - it's 3+ pages of code for the simplest nested scenario - will add some samples for various scenarios on the pr tho |
|
most basic scenario, no tracking ss.Set<JsonEntityBasic>().Select(x => x.OwnedReferenceRoot.OwnedReferenceBranch.OwnedReferenceLeaf).AsNoTracking()(queryContext, dataReader, resultContext, resultCoordinator) =>
{
Stream namelessParameter{0};
JsonReaderData namelessParameter{1};
Utf8JsonReaderManager namelessParameter{2};
object[] namelessParameter{3};
JsonOwnedLeaf namelessParameter{4};
namelessParameter{0} = dataReader.IsDBNull(0) ? default(MemoryStream) : new MemoryStream(Encoding.UTF8.GetBytes((string)dataReader.GetFieldValue<object>(0)));
namelessParameter{1} = namelessParameter{0} == default(MemoryStream) ? default(JsonReaderData) : new JsonReaderData(namelessParameter{0});
namelessParameter{1} != default(JsonReaderData) ?
{
namelessParameter{2} = new Utf8JsonReaderManager(namelessParameter{1});
namelessParameter{2}.MoveNext();
namelessParameter{2}.CaptureState();
} : default(void);
namelessParameter{3} = new object[]{ (object)dataReader.GetInt32(1) };
namelessParameter{4} = ShaperProcessingExpressionVisitor.MaterializeJsonEntity<JsonOwnedLeaf>(
queryContext: queryContext,
keyPropertyValues: namelessParameter{3},
jsonReaderData: namelessParameter{1},
nullable: False,
shaper: (queryContext, namelessParameter{5}, namelessParameter{6}) =>
{
JsonOwnedLeaf namelessParameter{7};
return namelessParameter{7} =
{
MaterializationContext materializationContext1;
IEntityType entityType1;
JsonOwnedLeaf instance1;
materializationContext1 = new MaterializationContext(
ValueBuffer,
queryContext.Context
);
instance1 = null;
namelessParameter{5}[0] != null ?
{
ValueBuffer shadowValueBuffer1;
shadowValueBuffer1 = ValueBuffer;
entityType1 = EntityType: JsonEntityBasic.OwnedReferenceRoot#JsonOwnedRoot.OwnedReferenceBranch#JsonOwnedBranch.OwnedReferenceLeaf#JsonOwnedLeaf CLR Type: JsonOwnedLeaf Owned;
instance1 =
{
Utf8JsonReaderManager namelessParameter{8};
JsonTokenType tokenType;
JsonOwnedLeaf instance;
string namelessParameter{9};
namelessParameter{8} = new Utf8JsonReaderManager(namelessParameter{6});
tokenType = namelessParameter{8}.CurrentReader.TokenType;
Loop(Break: done Continue: )
{
{
tokenType = namelessParameter{8}.MoveNext();
tokenType != EndObject ? switch (tokenType)
{
case PropertyName:
namelessParameter{8}.CurrentReader.ValueTextEquals(SomethingSomething.EncodedUtf8Bytes) ?
{
namelessParameter{8}.MoveNext();
namelessParameter{9} = namelessParameter{8}.CurrentReader.TokenType == Null ? default(string) : (string)JsonStringReaderWriter.FromJson(namelessParameter{8});
} : default(void)
default:
{
namelessParameter{8}.CurrentReader.TrySkip();
}
}
: Goto(break done)
;
}}
namelessParameter{8}.CaptureState();
instance = new JsonOwnedLeaf();
instance.<SomethingSomething>k__BackingField = namelessParameter{9};
(instance is IInjectableService) ? ((IInjectableService)instance).Injected(
context: materializationContext1.Context,
entity: instance,
bindingInfo: ParameterBindingInfo) : default(void);
return instance;
};
return instance1;
} :
{
object[] keyValues1;
keyValues1 = new object[]{ namelessParameter{5}[0] };
return EntityMaterializerInjectingExpressionVisitor.CreateNullKeyValueInNoTrackingQuery(
entityType: EntityType: JsonEntityBasic.OwnedReferenceRoot#JsonOwnedRoot.OwnedReferenceBranch#JsonOwnedBranch.OwnedReferenceLeaf#JsonOwnedLeaf CLR Type: JsonOwnedLeaf Owned,
properties: List<RuntimeProperty> { RuntimeProperty },
keyValues: keyValues1);
};
return instance1;
};
});
return namelessParameter{4};
} |
|
nested scenario (still no-tracking) ss.Set<JsonEntityBasic>().Select(x => x.OwnedReferenceRoot.OwnedCollectionBranch).AsNoTracking()(queryContext, dataReader, resultContext, resultCoordinator) =>
{
Stream namelessParameter{0};
JsonReaderData namelessParameter{1};
Utf8JsonReaderManager namelessParameter{2};
object[] namelessParameter{3};
List<JsonOwnedBranch> namelessParameter{4};
namelessParameter{0} = dataReader.IsDBNull(0) ? default(MemoryStream) : new MemoryStream(Encoding.UTF8.GetBytes((string)dataReader.GetFieldValue<object>(0)));
namelessParameter{1} = namelessParameter{0} == default(MemoryStream) ? default(JsonReaderData) : new JsonReaderData(namelessParameter{0});
namelessParameter{1} != default(JsonReaderData) ?
{
namelessParameter{2} = new Utf8JsonReaderManager(namelessParameter{1});
namelessParameter{2}.MoveNext();
namelessParameter{2}.CaptureState();
} : default(void);
namelessParameter{3} = new object[]{ (object)dataReader.GetInt32(1) };
namelessParameter{4} = ShaperProcessingExpressionVisitor.MaterializeJsonEntityCollection<JsonOwnedBranch, List<JsonOwnedBranch>>(
queryContext: queryContext,
keyPropertyValues: namelessParameter{3},
jsonReaderData: namelessParameter{1},
navigation: Navigation: JsonEntityBasic.OwnedReferenceRoot#JsonOwnedRoot.OwnedCollectionBranch (List<JsonOwnedBranch>) Collection ToDependent JsonEntityBasic.OwnedReferenceRoot#JsonOwnedRoot.OwnedCollectionBranch#JsonOwnedBranch,
innerShaper: (queryContext, namelessParameter{5}, namelessParameter{6}) =>
{
JsonOwnedBranch namelessParameter{7};
return namelessParameter{7} =
{
MaterializationContext materializationContext1;
IEntityType entityType1;
JsonOwnedBranch instance1;
materializationContext1 = new MaterializationContext(
ValueBuffer,
queryContext.Context
);
instance1 = null;
namelessParameter{5}[0] != null && namelessParameter{5}[1] != null ?
{
ValueBuffer shadowValueBuffer1;
shadowValueBuffer1 = ValueBuffer;
entityType1 = EntityType: JsonEntityBasic.OwnedReferenceRoot#JsonOwnedRoot.OwnedCollectionBranch#JsonOwnedBranch CLR Type: JsonOwnedBranch Owned;
instance1 =
{
Utf8JsonReaderManager namelessParameter{8};
JsonTokenType tokenType;
JsonOwnedBranch instance;
DateTime namelessParameter{9};
JsonEnum namelessParameter{10};
decimal namelessParameter{11};
JsonEnum? namelessParameter{12};
List<JsonOwnedLeaf> namelessParameter{13};
JsonOwnedLeaf namelessParameter{14};
namelessParameter{8} = new Utf8JsonReaderManager(namelessParameter{6});
tokenType = namelessParameter{8}.CurrentReader.TokenType;
Loop(Break: done Continue: )
{
{
tokenType = namelessParameter{8}.MoveNext();
tokenType != EndObject ? switch (tokenType)
{
case PropertyName:
namelessParameter{8}.CurrentReader.ValueTextEquals(Date.EncodedUtf8Bytes) ?
{
namelessParameter{8}.MoveNext();
namelessParameter{9} = (DateTime)JsonDateTimeReaderWriter.FromJson(namelessParameter{8});
} : namelessParameter{8}.CurrentReader.ValueTextEquals(Enum.EncodedUtf8Bytes) ?
{
namelessParameter{8}.MoveNext();
namelessParameter{10} = StringEnumConverter<JsonEnum, string, JsonEnum>.ConvertToEnum((string)JsonStringReaderWriter.FromJson(namelessParameter{8}));
} : namelessParameter{8}.CurrentReader.ValueTextEquals(Fraction.EncodedUtf8Bytes) ?
{
namelessParameter{8}.MoveNext();
namelessParameter{11} = (decimal)JsonDecimalReaderWriter.FromJson(namelessParameter{8});
} : namelessParameter{8}.CurrentReader.ValueTextEquals(NullableEnum.EncodedUtf8Bytes) ?
{
namelessParameter{8}.MoveNext();
namelessParameter{12} = namelessParameter{8}.CurrentReader.TokenType == Null ? default(JsonEnum?) : (JsonEnum?)StringEnumConverter<JsonEnum, string, JsonEnum>.ConvertToEnum((string)JsonStringReaderWriter.FromJson(namelessParameter{8}));
} : namelessParameter{8}.CurrentReader.ValueTextEquals(OwnedCollectionLeaf.EncodedUtf8Bytes) ?
{
namelessParameter{8}.MoveNext();
namelessParameter{8}.CaptureState();
namelessParameter{13} = ShaperProcessingExpressionVisitor.MaterializeJsonEntityCollection<JsonOwnedLeaf, List<JsonOwnedLeaf>>(
queryContext: queryContext,
keyPropertyValues: namelessParameter{5},
jsonReaderData: namelessParameter{6},
navigation: Navigation: JsonEntityBasic.OwnedReferenceRoot#JsonOwnedRoot.OwnedCollectionBranch#JsonOwnedBranch.OwnedCollectionLeaf (List<JsonOwnedLeaf>) Collection ToDependent JsonEntityBasic.OwnedReferenceRoot#JsonOwnedRoot.OwnedCollectionBranch#JsonOwnedBranch.OwnedCollectionLeaf#JsonOwnedLeaf Inverse: Parent,
innerShaper: (queryContext, namelessParameter{15}, namelessParameter{16}) =>
{
JsonOwnedLeaf namelessParameter{17};
return namelessParameter{17} =
{
MaterializationContext materializationContext2;
IEntityType entityType2;
JsonOwnedLeaf instance2;
materializationContext2 = new MaterializationContext(
ValueBuffer,
queryContext.Context
);
instance2 = null;
namelessParameter{15}[0] != null && namelessParameter{15}[1] != null && namelessParameter{15}[2] != null ?
{
ValueBuffer shadowValueBuffer2;
shadowValueBuffer2 = ValueBuffer;
entityType2 = EntityType: JsonEntityBasic.OwnedReferenceRoot#JsonOwnedRoot.OwnedCollectionBranch#JsonOwnedBranch.OwnedCollectionLeaf#JsonOwnedLeaf CLR Type: JsonOwnedLeaf Owned;
instance2 =
{
Utf8JsonReaderManager namelessParameter{18};
JsonTokenType tokenType;
JsonOwnedLeaf instance;
string namelessParameter{19};
namelessParameter{18} = new Utf8JsonReaderManager(namelessParameter{16});
tokenType = namelessParameter{18}.CurrentReader.TokenType;
Loop(Break: done Continue: )
{
{
tokenType = namelessParameter{18}.MoveNext();
tokenType != EndObject ? switch (tokenType)
{
case PropertyName:
namelessParameter{18}.CurrentReader.ValueTextEquals(SomethingSomething.EncodedUtf8Bytes) ?
{
namelessParameter{18}.MoveNext();
namelessParameter{19} = namelessParameter{18}.CurrentReader.TokenType == Null ? default(string) : (string)JsonStringReaderWriter.FromJson(namelessParameter{18});
} : default(void)
default:
{
namelessParameter{18}.CurrentReader.TrySkip();
}
}
: Goto(break done)
;
}}
namelessParameter{18}.CaptureState();
instance = new JsonOwnedLeaf();
instance.<SomethingSomething>k__BackingField = namelessParameter{19};
(instance is IInjectableService) ? ((IInjectableService)instance).Injected(
context: materializationContext2.Context,
entity: instance,
bindingInfo: ParameterBindingInfo) : default(void);
return instance;
};
return instance2;
} : default(void);
return instance2;
};
});
namelessParameter{8} = new Utf8JsonReaderManager(namelessParameter{6});
} : namelessParameter{8}.CurrentReader.ValueTextEquals(OwnedReferenceLeaf.EncodedUtf8Bytes) ?
{
namelessParameter{8}.MoveNext();
namelessParameter{8}.CaptureState();
namelessParameter{14} = ShaperProcessingExpressionVisitor.MaterializeJsonEntity<JsonOwnedLeaf>(
queryContext: queryContext,
keyPropertyValues: namelessParameter{5},
jsonReaderData: namelessParameter{6},
nullable: True,
shaper: (queryContext, namelessParameter{20}, namelessParameter{21}) =>
{
JsonOwnedLeaf namelessParameter{22};
return namelessParameter{22} =
{
MaterializationContext materializationContext3;
IEntityType entityType3;
JsonOwnedLeaf instance3;
materializationContext3 = new MaterializationContext(
ValueBuffer,
queryContext.Context
);
instance3 = null;
namelessParameter{20}[0] != null && namelessParameter{20}[1] != null ?
{
ValueBuffer shadowValueBuffer3;
shadowValueBuffer3 = ValueBuffer;
entityType3 = EntityType: JsonEntityBasic.OwnedReferenceRoot#JsonOwnedRoot.OwnedCollectionBranch#JsonOwnedBranch.OwnedReferenceLeaf#JsonOwnedLeaf CLR Type: JsonOwnedLeaf Owned;
instance3 =
{
Utf8JsonReaderManager namelessParameter{23};
JsonTokenType tokenType;
JsonOwnedLeaf instance;
string namelessParameter{24};
namelessParameter{23} = new Utf8JsonReaderManager(namelessParameter{21});
tokenType = namelessParameter{23}.CurrentReader.TokenType;
Loop(Break: done Continue: )
{
{
tokenType = namelessParameter{23}.MoveNext();
tokenType != EndObject ? switch (tokenType)
{
case PropertyName:
namelessParameter{23}.CurrentReader.ValueTextEquals(SomethingSomething.EncodedUtf8Bytes) ?
{
namelessParameter{23}.MoveNext();
namelessParameter{24} = namelessParameter{23}.CurrentReader.TokenType == Null ? default(string) : (string)JsonStringReaderWriter.FromJson(namelessParameter{23});
} : default(void)
default:
{
namelessParameter{23}.CurrentReader.TrySkip();
}
}
: Goto(break done)
;
}}
namelessParameter{23}.CaptureState();
instance = new JsonOwnedLeaf();
instance.<SomethingSomething>k__BackingField = namelessParameter{24};
(instance is IInjectableService) ? ((IInjectableService)instance).Injected(
context: materializationContext3.Context,
entity: instance,
bindingInfo: ParameterBindingInfo) : default(void);
return instance;
};
return instance3;
} : default(void);
return instance3;
};
});
namelessParameter{8} = new Utf8JsonReaderManager(namelessParameter{6});
} : default(void)
default:
{
namelessParameter{8}.CurrentReader.TrySkip();
}
}
: Goto(break done)
;
}}
namelessParameter{8}.CaptureState();
instance = new JsonOwnedBranch();
instance.<Date>k__BackingField = namelessParameter{9};
instance.<Enum>k__BackingField = namelessParameter{10};
instance.<Fraction>k__BackingField = namelessParameter{11};
instance.<NullableEnum>k__BackingField = namelessParameter{12};
(instance is IInjectableService) ? ((IInjectableService)instance).Injected(
context: materializationContext1.Context,
entity: instance,
bindingInfo: ParameterBindingInfo) : default(void);
instance != null && namelessParameter{13} != null ? Invoke((namelessParameter{25}, namelessParameter{26}) =>
{
namelessParameter{25}.<OwnedCollectionLeaf>k__BackingField = namelessParameter{26};
ShaperProcessingExpressionVisitor.InverseCollectionFixup<JsonOwnedLeaf, JsonOwnedBranch>(
collection: namelessParameter{26},
entity: namelessParameter{25},
elementFixup: (namelessParameter{27}, namelessParameter{28}) =>
{
return namelessParameter{27}.<Parent>k__BackingField = namelessParameter{28};
});
}, instance, namelessParameter{13}) : default(void);
instance != null && namelessParameter{14} != null ? Invoke((namelessParameter{29}, namelessParameter{30}) =>
{
namelessParameter{29}.<OwnedReferenceLeaf>k__BackingField = namelessParameter{30};
return namelessParameter{30}.<Parent>k__BackingField = namelessParameter{29};
}, instance, namelessParameter{14}) : default(void);
return instance;
};
return instance1;
} : default(void);
return instance1;
};
});
return namelessParameter{4};
} |
|
tracking query with shadow properties, specifically note the differences between how we built materializer for tracked json entity vs non-json entity - there was a lot of rewrite there (queryContext, dataReader, resultContext, resultCoordinator) =>
{
MyEntityShadowProperties namelessParameter{0};
Stream namelessParameter{1};
JsonReaderData namelessParameter{2};
Utf8JsonReaderManager namelessParameter{3};
object[] namelessParameter{4};
Stream namelessParameter{5};
JsonReaderData namelessParameter{6};
Utf8JsonReaderManager namelessParameter{7};
object[] namelessParameter{8};
Stream namelessParameter{9};
JsonReaderData namelessParameter{10};
Utf8JsonReaderManager namelessParameter{11};
object[] namelessParameter{12};
Stream namelessParameter{13};
JsonReaderData namelessParameter{14};
Utf8JsonReaderManager namelessParameter{15};
object[] namelessParameter{16};
namelessParameter{0} =
{
MaterializationContext materializationContext1;
IEntityType entityType1;
MyEntityShadowProperties instance1;
InternalEntityEntry entry1;
bool hasNullKey1;
materializationContext1 = new MaterializationContext(
ValueBuffer,
queryContext.Context
);
instance1 = null;
entry1 = queryContext.TryGetEntry(
key: Key: MyEntityShadowProperties.Id PK,
keyValues: new object[]{ (object)dataReader.GetInt32(0) },
throwOnNullKey: True,
hasNullKey: hasNullKey1);
!(hasNullKey1) ? entry1 != default(InternalEntityEntry) ?
{
entityType1 = entry1.EntityType;
return instance1 = (MyEntityShadowProperties)entry1.Entity;
} :
{
ValueBuffer shadowValueBuffer1;
shadowValueBuffer1 = ValueBuffer;
entityType1 = EntityType: MyEntityShadowProperties;
instance1 = switch (entityType1)
{
case EntityType: MyEntityShadowProperties:
{
return
{
MyEntityShadowProperties instance;
instance = new MyEntityShadowProperties();
instance.<Id>k__BackingField = dataReader.GetInt32(0);
instance.<Name>k__BackingField = dataReader.IsDBNull(1) ? default(string) : (string)dataReader.GetFieldValue<object>(1);
(instance is IInjectableService) ? ((IInjectableService)instance).Injected(
context: materializationContext1.Context,
entity: instance,
bindingInfo: ParameterBindingInfo) : default(void);
return instance;
}}
default:
null
}
;
entry1 = entityType1 == default(IEntityType) ? default(InternalEntityEntry) : queryContext.StartTracking(
entityType: entityType1,
entity: instance1,
valueBuffer: shadowValueBuffer1);
return instance1;
} : default(void);
return instance1;
};
namelessParameter{1} = dataReader.IsDBNull(2) ? default(MemoryStream) : new MemoryStream(Encoding.UTF8.GetBytes((string)dataReader.GetFieldValue<object>(2)));
namelessParameter{2} = namelessParameter{1} == default(MemoryStream) ? default(JsonReaderData) : new JsonReaderData(namelessParameter{1});
namelessParameter{2} != default(JsonReaderData) ?
{
namelessParameter{3} = new Utf8JsonReaderManager(namelessParameter{2});
namelessParameter{3}.MoveNext();
namelessParameter{3}.CaptureState();
} : default(void);
namelessParameter{4} = new object[]{ (object)dataReader.GetInt32(0) };
namelessParameter{5} = dataReader.IsDBNull(3) ? default(MemoryStream) : new MemoryStream(Encoding.UTF8.GetBytes((string)dataReader.GetFieldValue<object>(3)));
namelessParameter{6} = namelessParameter{5} == default(MemoryStream) ? default(JsonReaderData) : new JsonReaderData(namelessParameter{5});
namelessParameter{6} != default(JsonReaderData) ?
{
namelessParameter{7} = new Utf8JsonReaderManager(namelessParameter{6});
namelessParameter{7}.MoveNext();
namelessParameter{7}.CaptureState();
} : default(void);
namelessParameter{8} = new object[]{ (object)dataReader.GetInt32(0) };
namelessParameter{9} = dataReader.IsDBNull(4) ? default(MemoryStream) : new MemoryStream(Encoding.UTF8.GetBytes((string)dataReader.GetFieldValue<object>(4)));
namelessParameter{10} = namelessParameter{9} == default(MemoryStream) ? default(JsonReaderData) : new JsonReaderData(namelessParameter{9});
namelessParameter{10} != default(JsonReaderData) ?
{
namelessParameter{11} = new Utf8JsonReaderManager(namelessParameter{10});
namelessParameter{11}.MoveNext();
namelessParameter{11}.CaptureState();
} : default(void);
namelessParameter{12} = new object[]{ (object)dataReader.GetInt32(0) };
namelessParameter{13} = dataReader.IsDBNull(5) ? default(MemoryStream) : new MemoryStream(Encoding.UTF8.GetBytes((string)dataReader.GetFieldValue<object>(5)));
namelessParameter{14} = namelessParameter{13} == default(MemoryStream) ? default(JsonReaderData) : new JsonReaderData(namelessParameter{13});
namelessParameter{14} != default(JsonReaderData) ?
{
namelessParameter{15} = new Utf8JsonReaderManager(namelessParameter{14});
namelessParameter{15}.MoveNext();
namelessParameter{15}.CaptureState();
} : default(void);
namelessParameter{16} = new object[]{ (object)dataReader.GetInt32(0) };
ShaperProcessingExpressionVisitor.IncludeJsonEntityCollection<MyEntityShadowProperties, MyJsonEntityShadowProperties>(
queryContext: queryContext,
keyPropertyValues: namelessParameter{4},
jsonReaderData: namelessParameter{2},
entity: namelessParameter{0},
innerShaper: (queryContext, namelessParameter{17}, namelessParameter{18}) =>
{
MyJsonEntityShadowProperties namelessParameter{19};
return namelessParameter{19} =
{
MaterializationContext materializationContext2;
IEntityType entityType2;
MyJsonEntityShadowProperties instance2;
InternalEntityEntry entry2;
bool hasNullKey2;
materializationContext2 = new MaterializationContext(
ValueBuffer,
queryContext.Context
);
instance2 = null;
entry2 = queryContext.TryGetEntry(
key: Key: MyEntityShadowProperties.Collection#MyJsonEntityShadowProperties.MyEntityShadowPropertiesId, MyEntityShadowProperties.Collection#MyJsonEntityShadowProperties.Id PK,
keyValues: new object[]
{
namelessParameter{17}[0],
namelessParameter{17}[1]
},
throwOnNullKey: False,
hasNullKey: hasNullKey2);
!(hasNullKey2) ?
{
bool namelessParameter{20};
ValueBuffer shadowValueBuffer2;
namelessParameter{20} = False;
shadowValueBuffer2 = ValueBuffer;
entityType2 = EntityType: MyEntityShadowProperties.Collection#MyJsonEntityShadowProperties CLR Type: MyJsonEntityShadowProperties Owned;
entry2 != default(InternalEntityEntry) ?
{
entityType2 = entry2.EntityType;
instance2 = (MyJsonEntityShadowProperties)entry2.Entity;
namelessParameter{20} = True;
} : default(void);
instance2 =
{
Utf8JsonReaderManager namelessParameter{21};
JsonTokenType tokenType;
MyJsonEntityShadowProperties instance;
object namelessParameter{22};
string namelessParameter{23};
namelessParameter{21} = new Utf8JsonReaderManager(namelessParameter{18});
tokenType = namelessParameter{21}.CurrentReader.TokenType;
Loop(Break: done Continue: )
{
{
tokenType = namelessParameter{21}.MoveNext();
tokenType != EndObject ? switch (tokenType)
{
case PropertyName:
namelessParameter{21}.CurrentReader.ValueTextEquals(ShadowDouble.EncodedUtf8Bytes) ?
{
namelessParameter{21}.MoveNext();
namelessParameter{22} = (object)(double)JsonDoubleReaderWriter.FromJson(namelessParameter{21});
} : namelessParameter{21}.CurrentReader.ValueTextEquals(Name.EncodedUtf8Bytes) ?
{
namelessParameter{21}.MoveNext();
namelessParameter{23} = namelessParameter{21}.CurrentReader.TokenType == Null ? default(string) : (string)JsonStringReaderWriter.FromJson(namelessParameter{21});
} : default(void)
default:
{
namelessParameter{21}.CurrentReader.TrySkip();
}
}
: Goto(break done)
;
}}
namelessParameter{21}.CaptureState();
!(namelessParameter{20}) ? shadowValueBuffer2 = new ValueBuffer(new object[]
{
namelessParameter{17}[0],
namelessParameter{17}[1],
namelessParameter{22}
}) : default(void);
namelessParameter{20} ?
{
instance = instance2;
} :
{
instance = new MyJsonEntityShadowProperties();
instance.<Name>k__BackingField = namelessParameter{23};
(instance is IInjectableService) ? ((IInjectableService)instance).Injected(
context: materializationContext2.Context,
entity: instance,
bindingInfo: ParameterBindingInfo) : default(void);
};
return instance;
};
!(namelessParameter{20} || entityType2 == default(IEntityType)) ?
{
queryContext.StartTracking(
entityType: entityType2,
entity: instance2,
valueBuffer: shadowValueBuffer2);
} : default(void);
} : default(void);
return instance2;
};
},
fixup: (namelessParameter{24}, namelessParameter{25}) =>
{
return ClrICollectionAccessor<MyEntityShadowProperties, List<MyJsonEntityShadowProperties>, MyJsonEntityShadowProperties>.Add(
entity: namelessParameter{24},
value: namelessParameter{25},
forMaterialization: True);
});
ShaperProcessingExpressionVisitor.IncludeJsonEntityCollection<MyEntityShadowProperties, MyJsonEntityShadowPropertiesWithCtor>(
queryContext: queryContext,
keyPropertyValues: namelessParameter{8},
jsonReaderData: namelessParameter{6},
entity: namelessParameter{0},
innerShaper: (queryContext, namelessParameter{26}, namelessParameter{27}) =>
{
MyJsonEntityShadowPropertiesWithCtor namelessParameter{28};
return namelessParameter{28} =
{
MaterializationContext materializationContext3;
IEntityType entityType3;
MyJsonEntityShadowPropertiesWithCtor instance3;
InternalEntityEntry entry3;
bool hasNullKey3;
materializationContext3 = new MaterializationContext(
ValueBuffer,
queryContext.Context
);
instance3 = null;
entry3 = queryContext.TryGetEntry(
key: Key: MyEntityShadowProperties.CollectionWithCtor#MyJsonEntityShadowPropertiesWithCtor.MyEntityShadowPropertiesId, MyEntityShadowProperties.CollectionWithCtor#MyJsonEntityShadowPropertiesWithCtor.Id PK,
keyValues: new object[]
{
namelessParameter{26}[0],
namelessParameter{26}[1]
},
throwOnNullKey: False,
hasNullKey: hasNullKey3);
!(hasNullKey3) ?
{
bool namelessParameter{29};
ValueBuffer shadowValueBuffer3;
namelessParameter{29} = False;
shadowValueBuffer3 = ValueBuffer;
entityType3 = EntityType: MyEntityShadowProperties.CollectionWithCtor#MyJsonEntityShadowPropertiesWithCtor CLR Type: MyJsonEntityShadowPropertiesWithCtor Owned;
entry3 != default(InternalEntityEntry) ?
{
entityType3 = entry3.EntityType;
instance3 = (MyJsonEntityShadowPropertiesWithCtor)entry3.Entity;
namelessParameter{29} = True;
} : default(void);
instance3 =
{
Utf8JsonReaderManager namelessParameter{30};
JsonTokenType tokenType;
MyJsonEntityShadowPropertiesWithCtor instance;
object namelessParameter{31};
string namelessParameter{32};
namelessParameter{30} = new Utf8JsonReaderManager(namelessParameter{27});
tokenType = namelessParameter{30}.CurrentReader.TokenType;
Loop(Break: done Continue: )
{
{
tokenType = namelessParameter{30}.MoveNext();
tokenType != EndObject ? switch (tokenType)
{
case PropertyName:
namelessParameter{30}.CurrentReader.ValueTextEquals(ShadowNullableByte.EncodedUtf8Bytes) ?
{
namelessParameter{30}.MoveNext();
namelessParameter{31} = (object)namelessParameter{30}.CurrentReader.TokenType == Null ? default(byte?) : (byte?)(byte)JsonByteReaderWriter.FromJson(namelessParameter{30});
} : namelessParameter{30}.CurrentReader.ValueTextEquals(Name.EncodedUtf8Bytes) ?
{
namelessParameter{30}.MoveNext();
namelessParameter{32} = namelessParameter{30}.CurrentReader.TokenType == Null ? default(string) : (string)JsonStringReaderWriter.FromJson(namelessParameter{30});
} : default(void)
default:
{
namelessParameter{30}.CurrentReader.TrySkip();
}
}
: Goto(break done)
;
}}
namelessParameter{30}.CaptureState();
!(namelessParameter{29}) ? shadowValueBuffer3 = new ValueBuffer(new object[]
{
namelessParameter{26}[0],
namelessParameter{26}[1],
namelessParameter{31}
}) : default(void);
namelessParameter{29} ?
{
instance = instance3;
} :
{
instance = new MyJsonEntityShadowPropertiesWithCtor(namelessParameter{32});
};
return instance;
};
!(namelessParameter{29} || entityType3 == default(IEntityType)) ?
{
queryContext.StartTracking(
entityType: entityType3,
entity: instance3,
valueBuffer: shadowValueBuffer3);
} : default(void);
} : default(void);
return instance3;
};
},
fixup: (namelessParameter{33}, namelessParameter{34}) =>
{
return ClrICollectionAccessor<MyEntityShadowProperties, List<MyJsonEntityShadowPropertiesWithCtor>, MyJsonEntityShadowPropertiesWithCtor>.Add(
entity: namelessParameter{33},
value: namelessParameter{34},
forMaterialization: True);
});
ShaperProcessingExpressionVisitor.IncludeJsonEntityReference<MyEntityShadowProperties, MyJsonEntityShadowProperties>(
queryContext: queryContext,
keyPropertyValues: namelessParameter{12},
jsonReaderData: namelessParameter{10},
entity: namelessParameter{0},
innerShaper: (queryContext, namelessParameter{35}, namelessParameter{36}) =>
{
MyJsonEntityShadowProperties namelessParameter{37};
return namelessParameter{37} =
{
MaterializationContext materializationContext4;
IEntityType entityType4;
MyJsonEntityShadowProperties instance4;
InternalEntityEntry entry4;
bool hasNullKey4;
materializationContext4 = new MaterializationContext(
ValueBuffer,
queryContext.Context
);
instance4 = null;
entry4 = queryContext.TryGetEntry(
key: Key: MyEntityShadowProperties.Reference#MyJsonEntityShadowProperties.MyEntityShadowPropertiesId PK,
keyValues: new object[]{ namelessParameter{35}[0] },
throwOnNullKey: False,
hasNullKey: hasNullKey4);
!(hasNullKey4) ?
{
bool namelessParameter{38};
ValueBuffer shadowValueBuffer4;
namelessParameter{38} = False;
shadowValueBuffer4 = ValueBuffer;
entityType4 = EntityType: MyEntityShadowProperties.Reference#MyJsonEntityShadowProperties CLR Type: MyJsonEntityShadowProperties Owned;
entry4 != default(InternalEntityEntry) ?
{
entityType4 = entry4.EntityType;
instance4 = (MyJsonEntityShadowProperties)entry4.Entity;
namelessParameter{38} = True;
} : default(void);
instance4 =
{
Utf8JsonReaderManager namelessParameter{39};
JsonTokenType tokenType;
MyJsonEntityShadowProperties instance;
object namelessParameter{40};
string namelessParameter{41};
namelessParameter{39} = new Utf8JsonReaderManager(namelessParameter{36});
tokenType = namelessParameter{39}.CurrentReader.TokenType;
Loop(Break: done Continue: )
{
{
tokenType = namelessParameter{39}.MoveNext();
tokenType != EndObject ? switch (tokenType)
{
case PropertyName:
namelessParameter{39}.CurrentReader.ValueTextEquals(ShadowString.EncodedUtf8Bytes) ?
{
namelessParameter{39}.MoveNext();
namelessParameter{40} = (object)namelessParameter{39}.CurrentReader.TokenType == Null ? default(string) : (string)JsonStringReaderWriter.FromJson(namelessParameter{39});
} : namelessParameter{39}.CurrentReader.ValueTextEquals(Name.EncodedUtf8Bytes) ?
{
namelessParameter{39}.MoveNext();
namelessParameter{41} = namelessParameter{39}.CurrentReader.TokenType == Null ? default(string) : (string)JsonStringReaderWriter.FromJson(namelessParameter{39});
} : default(void)
default:
{
namelessParameter{39}.CurrentReader.TrySkip();
}
}
: Goto(break done)
;
}}
namelessParameter{39}.CaptureState();
!(namelessParameter{38}) ? shadowValueBuffer4 = new ValueBuffer(new object[]
{
namelessParameter{35}[0],
namelessParameter{40}
}) : default(void);
namelessParameter{38} ?
{
instance = instance4;
} :
{
instance = new MyJsonEntityShadowProperties();
instance.<Name>k__BackingField = namelessParameter{41};
(instance is IInjectableService) ? ((IInjectableService)instance).Injected(
context: materializationContext4.Context,
entity: instance,
bindingInfo: ParameterBindingInfo) : default(void);
};
return instance;
};
!(namelessParameter{38} || entityType4 == default(IEntityType)) ?
{
queryContext.StartTracking(
entityType: entityType4,
entity: instance4,
valueBuffer: shadowValueBuffer4);
} : default(void);
} : default(void);
return instance4;
};
},
fixup: (namelessParameter{42}, namelessParameter{43}) =>
{
return namelessParameter{42}.<Reference>k__BackingField = namelessParameter{43};
});
ShaperProcessingExpressionVisitor.IncludeJsonEntityReference<MyEntityShadowProperties, MyJsonEntityShadowPropertiesWithCtor>(
queryContext: queryContext,
keyPropertyValues: namelessParameter{16},
jsonReaderData: namelessParameter{14},
entity: namelessParameter{0},
innerShaper: (queryContext, namelessParameter{44}, namelessParameter{45}) =>
{
MyJsonEntityShadowPropertiesWithCtor namelessParameter{46};
return namelessParameter{46} =
{
MaterializationContext materializationContext5;
IEntityType entityType5;
MyJsonEntityShadowPropertiesWithCtor instance5;
InternalEntityEntry entry5;
bool hasNullKey5;
materializationContext5 = new MaterializationContext(
ValueBuffer,
queryContext.Context
);
instance5 = null;
entry5 = queryContext.TryGetEntry(
key: Key: MyEntityShadowProperties.ReferenceWithCtor#MyJsonEntityShadowPropertiesWithCtor.MyEntityShadowPropertiesId PK,
keyValues: new object[]{ namelessParameter{44}[0] },
throwOnNullKey: False,
hasNullKey: hasNullKey5);
!(hasNullKey5) ?
{
bool namelessParameter{47};
ValueBuffer shadowValueBuffer5;
namelessParameter{47} = False;
shadowValueBuffer5 = ValueBuffer;
entityType5 = EntityType: MyEntityShadowProperties.ReferenceWithCtor#MyJsonEntityShadowPropertiesWithCtor CLR Type: MyJsonEntityShadowPropertiesWithCtor Owned;
entry5 != default(InternalEntityEntry) ?
{
entityType5 = entry5.EntityType;
instance5 = (MyJsonEntityShadowPropertiesWithCtor)entry5.Entity;
namelessParameter{47} = True;
} : default(void);
instance5 =
{
Utf8JsonReaderManager namelessParameter{48};
JsonTokenType tokenType;
MyJsonEntityShadowPropertiesWithCtor instance;
object namelessParameter{49};
string namelessParameter{50};
namelessParameter{48} = new Utf8JsonReaderManager(namelessParameter{45});
tokenType = namelessParameter{48}.CurrentReader.TokenType;
Loop(Break: done Continue: )
{
{
tokenType = namelessParameter{48}.MoveNext();
tokenType != EndObject ? switch (tokenType)
{
case PropertyName:
namelessParameter{48}.CurrentReader.ValueTextEquals(ShadowInt.EncodedUtf8Bytes) ?
{
namelessParameter{48}.MoveNext();
namelessParameter{49} = (object)(int)JsonInt32ReaderWriter.FromJson(namelessParameter{48});
} : namelessParameter{48}.CurrentReader.ValueTextEquals(Name.EncodedUtf8Bytes) ?
{
namelessParameter{48}.MoveNext();
namelessParameter{50} = namelessParameter{48}.CurrentReader.TokenType == Null ? default(string) : (string)JsonStringReaderWriter.FromJson(namelessParameter{48});
} : default(void)
default:
{
namelessParameter{48}.CurrentReader.TrySkip();
}
}
: Goto(break done)
;
}}
namelessParameter{48}.CaptureState();
!(namelessParameter{47}) ? shadowValueBuffer5 = new ValueBuffer(new object[]
{
namelessParameter{44}[0],
namelessParameter{49}
}) : default(void);
namelessParameter{47} ?
{
instance = instance5;
} :
{
instance = new MyJsonEntityShadowPropertiesWithCtor(namelessParameter{50});
};
return instance;
};
!(namelessParameter{47} || entityType5 == default(IEntityType)) ?
{
queryContext.StartTracking(
entityType: entityType5,
entity: instance5,
valueBuffer: shadowValueBuffer5);
} : default(void);
} : default(void);
return instance5;
};
},
fixup: (namelessParameter{51}, namelessParameter{52}) =>
{
return namelessParameter{51}.<ReferenceWithCtor>k__BackingField = namelessParameter{52};
});
return namelessParameter{0};
} |
maumar
force-pushed
the
fix31159
branch
2 times, most recently
from
f23c8cd to
1fc77a3
Compare
roji
approved these changes
Jul 10, 2023
There was a problem hiding this comment.
Spent some time reviewing this; I'm definitely far from grokking everything that goes on :) But overall it looks good, a a good basis to build on for better perf in the future.
See comments which are almost only nits. I do think we should start systematically assigning names to ParameterExpressions we create, Having to figure out what namelessParameter{7} means in the generated code isn't easy...
...l/Query/RelationalShapedQueryCompilingExpressionVisitor.ShaperProcessingExpressionVisitor.cs
Show resolved
Hide resolved
...l/Query/RelationalShapedQueryCompilingExpressionVisitor.ShaperProcessingExpressionVisitor.cs
Outdated
Show resolved
Hide resolved
...l/Query/RelationalShapedQueryCompilingExpressionVisitor.ShaperProcessingExpressionVisitor.cs
Outdated
Show resolved
Hide resolved
...l/Query/RelationalShapedQueryCompilingExpressionVisitor.ShaperProcessingExpressionVisitor.cs
Outdated
Show resolved
Hide resolved
...l/Query/RelationalShapedQueryCompilingExpressionVisitor.ShaperProcessingExpressionVisitor.cs
Show resolved
Hide resolved
|
|
||
| if (value is not null) | ||
| { | ||
| property.GetJsonValueReaderWriter()!.ToJson(writer, value); |
There was a problem hiding this comment.
@ajcvickers naming nit: should this simply be called a converter?
...l/Query/RelationalShapedQueryCompilingExpressionVisitor.ShaperProcessingExpressionVisitor.cs
Show resolved
Hide resolved
…JsonReader/Utf8JsonWriter Using Utf8JsonReader to read JSON data rather than caching it using DOM. This should reduce allocations significantly. Tricky part is that entity materializers are build in a way that assumes we have random access to all the data we need. This is not the case here. We read JSON data sequentially and can only do it once, and we don't know the order in which we get the data. This is somewhat problematic in case where entity takes argument in the constructor. Those could be at the very end of the JSON string, so we must read all the data before we can instantiate the object, and populate it's properties and do navigation fixup. This requires us reading all the JSON data, store them in local variables, and only when we are done reading we instantiate the entity and populate all the properties with data stored in those variables. This adds some allocations (specifically navigations). We also have to disable de-duplication logic - we can't always safely re-read the JSON string, and definitely can't start reading it from arbitrary position, so now we have to add JSON string for every aggregate projected, even if we already project it's parent. Serialization implementation (i.e. Utf8JsonWriter) is pretty straighforward. Also fix to #30993 - Query/Json: data corruption for tracking queries with nested json entities, then updating nested entities outside EF and re-querying Fix is to recognize and modify shaper in case of tracking query, so that nav expansions are not skipped when parent entity is found in Change Tracker. This is necessary to fix alongside streaming, because now we throw exception from reader (unexpected token) if we don't process the entire stream correctly. Before it would be silently ignored apart from the edge case described in the bug. Fixes #30604 Fixes #30993
| @@ -654,7 +649,6 @@ private ProjectionBindingExpression AddClientProjection(Expression expression, T | |||
| return new ProjectionBindingExpression(_selectExpression, existingIndex, type); | |||
| } | |||
|
|
|||
| #pragma warning disable IDE0052 // Remove unread private members | |||
| private static T GetParameterValue<T>(QueryContext queryContext, string parameterName) | |||
| #pragma warning restore IDE0052 // Remove unread private members | |||
There was a problem hiding this comment.
Is this line still needed?
| } | ||
|
|
||
| private static TResult? MaterializeJsonEntityCollection<TEntity, TResult>( | ||
| private static void IncludeJsonEntityCollection<TIncludingEntity, TIncludedCollectionElement>( |
There was a problem hiding this comment.
This can be refactored to share implementation with MaterializeJsonEntityCollection
This comment was marked as off-topic.
This comment was marked as off-topic.
Sorry, something went wrong.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Using Utf8JsonReader to read JSON data rather than caching it using DOM. This should reduce allocations significantly. Tricky part is that entity materializers are build in a way that assumes we have random access to all the data we need. This is not the case here.
We read JSON data sequentially and can only do it once, and we don't know the order in which we get the data. This is somewhat problematic in case where entity takes argument in the constructor. Those could be at the very end of the JSON string, so we must read all the data before we can instantiate the object, and populate it's properties and do navigation fixup.
This requires us reading all the JSON data, store them in local variables, and only when we are done reading we instantiate the entity and populate all the properties with data stored in those variables. This adds some allocations (specifically navigations).
We also have to disable de-duplication logic - we can't always safely re-read the JSON string, and definitely can't start reading it from arbitrary position, so now we have to add JSON string for every aggregate projected, even if we already project it's parent.
Serialization implementation (i.e. Utf8JsonWriter) is pretty straighforward.
Also fix to #30993 - Query/Json: data corruption for tracking queries with nested json entities, then updating nested entities outside EF and re-querying
Fix is to recognize and modify shaper in case of tracking query, so that nav expansions are not skipped when parent entity is found in Change Tracker. This is necessary to fix alongside streaming, because now we throw exception from reader (unexpected token) if we don't process the entire stream correctly. Before it would be silently ignored apart from the edge case described in the bug.
Fixes #30604
Fixes #30993