JSON: Allow weakly-typed mapping via Dictionary

[roji]

#28871 tracks weakly-typed JSON mapping via JsonDocument/JsonElement, where the JSON document schema varies and so a strongly-typed model isn't appropriate.

We can also allow simply mapping arbitrary Dictionary types - this corresponds to how many people use

• This intersects with #28688 (primitive collections in JSON).
• Both keys and values should support any valid JSON data type (string, int...).
• However we could also support Dictionary<string, object>, to map an entire hierarchy. See #26903 on this (currently seemed blocked by property bag detection).
• This should be supported at the top-level, so that the JSON object as a whole is mapped as a single dictionary (see e.g. this issue). This would mean supporting OwnsOne(x => x.Json, builder => { builder.ToJson(); }) where x.Json is a Dictionary.

[roji]

Duplicate of #29427

[ajcvickers]

Re-opening, since dictionary mapping was not covered by "primitive collections".

[vchirikov]

Link to the "issue" should be in the docs at least, was difficult to find..
Is it planned for 9.0 release?

[roji]

@vchirikov the milestone generally expresses whether an issue is planned for a given release or not.

[onionhammer]

I assume this would also meant support for ExtensionData and Dictionary<string, JsonElement> ?

[roji]

@onionhammer that would be a combination of this and #28871.

[onionhammer]

@roji putting querying aside for now, JsonElement can already be saved/serialized as an entity property, but Dictionary<string, JsonElement> cannot;

[aleksvujic]

@roji You wrote in this comment that mappings like Dictionary<string, string> aren't supported yet and suggested using value converters instead.

I configured serialization and deserialization with HasConversion like this:

modelBuilder.Entity<Result>(entity =>
{
    entity.OwnsOne(x => x.OutputJSON, x =>
    {
        x.ToJson("output");

        x.Property(y => y.Data)
            .HasConversion(
                v => JsonSerializer.Serialize(v, (JsonSerializerOptions?)null),
                v => JsonSerializer.Deserialize<Dictionary<string, string?>>(v, (JsonSerializerOptions?)null)
            );
    });
});

However, when I perform fitDbContext.Results.ToList(), the following exception is thrown:

System.InvalidOperationException: 'Cannot get the value of a token type 'StartObject' as a string.'

I even tried to perform dummy serialization and deserialization like this, but the exception is still the same:

x.Property(y => y.Data)
    .HasConversion(
        v => "{}",
        v => new Dictionary<string, string?>()
    );

Am I using value conversion wrong?

[roji]

@aleksvujic I'm a bit confused by the above - above you have an entity type Result, containing a JSON-mapped owned entity (OutputJSON), and inside that you want to have a Data property that's another JSON sub-document mapped to a dictionary? So a strongly-typed JSON document (OutputJson) containing a weakler-typed JSON document (Data)? Is that the intention here?

If so, note that the entire weakly-typed dictionary (Data) would be serialized as a single string inside the larger JSON document. So you wouldn't get { "Data": { "x" : 3 } }, but rather { "Data": "{\"x\": 3 }" } (note that quotes around Data value. I'm not sure that's what you want.

[aleksvujic]

@roji I think you summarized it correctly. JSON in the output column in PostgreSQL table looks like this:

{
  "data": {
    "john": "doe",
    "hello": "world"
  }
}

The data field in the JSON is static and is always there, its content is dynamic (dictionary - (string, string) pairs).

That's why I wanted to model data in C# like this with Dictionary<string, string>:

// scaffolded DB model
[Table("results", Schema = "fit3")]
public partial class Result
{
    [Key]
    [Column("id")]
    public int Id { get; set; }

    [Column("output", TypeName = "jsonb")]
    public string? Output { get; set; }
}

// custom class with additional strongly-typed OutputJSON property
public partial class Result
{
    public Output? OutputJSON { get; set; }
}

// content of the "data" JSON field (dictionary)
public class Output
{
    [JsonPropertyName("data")]
    public Dictionary<string, string?>? Data { get; set; }
}

If I understand correctly, this can't work - this exact problem is tracked by this issue. Correct?

If I format the nested JSON document in the PostgreSQL table as string (shown below), then EF successfuly maps it to C# Dictionary<string, string> when I perform fitDbContext.Results.ToList().

{"data": "{ \"john\": \"doe\",  \"hello\": \"world\" }"}

[roji]

If I understand correctly, this can't work - this exact problem is tracked by this issue. Correct?

No. EF already supports mapping anything via value converters - whether it be a dictionary or something else. This issue tracks allowing mapping a Dictionary to JSON without value converters, which would mean that EF manages the conversion and knows what's going on; that would notably mean that you can e.g. use dictionary lookups inside queries, which is something that you can't do when value-converting a value.

Now, there may be an EF bug specifically when nesting your value-converted dictionary within another JSON owned entity; but that wouldn't be related to this issue.

Can you please open a new issue and include a minimal repro for the problem?