Introduction
Technically, this is a continuation of How to prevent the user from falling asleep while loading a large dataset.I am doing "major refactoring" of a corporate system that has been in use for 20 years. I am presenting some of my solutions here in the hope that they will be useful to someone, but also to learn something new from the comments.
Problem #1.
We have an old database, in which for 20 years there are all sorts of things. It also has primary keys of 2 fields, once it was relevant. The new system has to be built on it to be able to start using it earlier, even in parallel for some time, gradually transferring the functionality. At some point, perhaps, this database structure will be replaced too. We want models neither on the application server nor on the client to know nothing about the database structure. The obvious solution is to load objects with database keys from the database and pass them to the client, and use them through interfaces, without keys.
Problem #2.
Objects for the table for the table should not contain a complete structure, a few displayed fields for visual contact, search and sorting is enough. On the other hand, you would not want to have different classes to display in the table and load a single object. It makes sense to use one class, but populate its entities depending on needs, using different interfaces for access.
What can be done by the regular means of System.Text.Json ?
Let's look at a few options.
public interface IPreCat
{
string Breed { get; }
}
public interface ICatForListing
{
string Breed { get; }
string Name { get; }
}
public interface IPaw
{
Longitude Longitude { get; }
Latitude Latitude { get; }
List Claws { get; }
}
public interface IClaw
{
double Sharpness { get; }
}
public interface IMustache
{
double Length { get; }
}
public interface ITail
{
double Length { get; }
double Thickness { get; }
}
public class StringIntId
{
public string StringId { get; set; }
public int IntId { get; set; }
}
public class Cat: PreCat, ICat, ICatForListing
{
...
public StringIntId Id { get; set; }
public string Name { get; set; }
public List Paws { get; init; } = new();
public IMustache? Mustache { get; set; } = null;
public ITail? Tail { get; set; } = null;
public override string ToString()
{
return $"{{{GetType().Name}:ntbreed: {Breed},ntname: {Name},ntpaws: [ntt{string.Join(",ntt", Paws)}nt],ntmustache: {Mustache},nttail: {Tail}n}}";
}
}
...
[Test]
public void Test1()
{
// (1)
Cat cat = CreateCat() as Cat;
Console.WriteLine(cat);
// (2)
string json = JsonSerializer.Serialize(cat);
Console.WriteLine(json);
// (3)
json = JsonSerializer.Serialize(cat);
Console.WriteLine(json);
// (4)
json = JsonSerializer.Serialize(cat);
Console.WriteLine(json);
// (5)
json = JsonSerializer.Serialize(cat);
Console.WriteLine(json);
}
We build a cat (1), see how it is displayed as a line:
{Cat:
breed: Havana,
name: Murka,
paws: [
{Paw: longitude: Front, latitude: Left, claws: 5},
{Paw: longitude: Rear, latitude: Left, claws: 4},
{Paw: longitude: Front, latitude: Right, claws: 5},
{Paw: longitude: Rear, latitude: Right, claws: 3}
],
mustache: ,
tail: {Tail: length:25, thickness: 3}
}
We write it in JSON using class (2):
{"Id":{"StringId":"weadsfdfadsgsag","IntId":1},"Name":"Murka",
"Paws":[{"Longitude":1,"Latitude":1,
"Claws":[{"Sharpness":1},{"Sharpness":1},
{"Sharpness":1},{"Sharpness":1},{"Sharpness":1}]},
{"Longitude":2,"Latitude":1,
"Claws":[{"Sharpness":2},{"Sharpness":2},
{"Sharpness":2},{"Sharpness":2}]},
{"Longitude":1,"Latitude":2,
"Claws":[{"Sharpness":1},{"Sharpness":1},
{"Sharpness":1},{"Sharpness":1},{"Sharpness":1}]},
{"Longitude":2,"Latitude":2,
"Claws":[{"Sharpness":2},{"Sharpness":2},{"Sharpness":2}]}],
"Mustache":null,"Tail":{"Length":25,"Thickness":3},"Breed":"Havana"}
We write it in JSON using object (3):
{"Id":{"StringId":"weadsfdfadsgsag","IntId":1},"Name":"Murka",
"Paws":[{"Longitude":1,"Latitude":1,
"Claws":[{"Sharpness":1},{"Sharpness":1},
{"Sharpness":1},{"Sharpness":1},{"Sharpness":1}]},
{"Longitude":2,"Latitude":1,
"Claws":[{"Sharpness":2},{"Sharpness":2},
{"Sharpness":2},{"Sharpness":2}]},
{"Longitude":1,"Latitude":2,
"Claws":[{"Sharpness":1},{"Sharpness":1},
{"Sharpness":1},{"Sharpness":1},{"Sharpness":1}]},
{"Longitude":2,"Latitude":2,
"Claws":[{"Sharpness":2},{"Sharpness":2},{"Sharpness":2}]}],
"Mustache":null,"Tail":{"Length":25,"Thickness":3},"Breed":"Havana"}
We see that the results (1) and (2) are the same. Even Id got here, but only in the cat itself, since the whiskers, paws, claws, and tail are represented by interfaces. If we represented them by implementations, we would not be able to either refer through the interfaces to the cat itself, or those interfaces would depend on the implementations of the cat's parts. Both of these options don't work for us. Also, we would have to drag a lot of extra properties into the table. And it's also not good, in my opinion, that the enum comes in as a number (e.g. ... "Longitude":1, "Latitude":1..., here they mean "front-back" and "left-right"). In principle, you can set not to pass default values, but, for example, our list of paws is created in the constructor, and in general, objects could be loaded earlier and completely, and then suddenly it was necessary to pass them to the table on the client.
Let's write in JSON the simplified cat for table (3):
public interface ICatForListing
{
string Breed { get; }
string Name { get; }
}
{"Breed":"Havana","Name":"Murka"}
Well, it's short but keyless.
Finally, let's write down the complete cat (4):
{"Name":"Murka",
"Paws":[{"Longitude":1,"Latitude":1,
"Claws":[{"Sharpness":1},{"Sharpness":1},
{"Sharpness":1},{"Sharpness":1},{"Sharpness":1}]},
{"Longitude":2,"Latitude":1,
"Claws":[{"Sharpness":2},{"Sharpness":2},
{"Sharpness":2},{"Sharpness":2}]},
{"Longitude":1,"Latitude":2,
"Claws":[{"Sharpness":1},{"Sharpness":1},
{"Sharpness":1},{"Sharpness":1},{"Sharpness":1}]},
{"Longitude":2,"Latitude":2,
"Claws":[{"Sharpness":2},{"Sharpness":2},
{"Sharpness":2}]}],
"Mustache":null,
"Tail":{"Length":25,"Thickness":3}}
Of course, there are no keys, missing breed (Breed), because we did not include it in the ICat for some reason.
It turns out that neither option satisfied us, and we have nothing else to do but write ...
Custom converter
Just in case, let me remind you how the custom converter is embedded in the JSON serialization classes provided in the System.Text.Json namespace. In terms of design patterns, "Strategy" applies here. Our converter object (or converter factory, what we'll actually use) is added to the Converters list of the JsonSerializerOptions class object, which is passed to the JsonSerializer.Serialize(...) and JsonSerializer.Deserialize(...) methods. Our converter should be able to answer the question whether it converts objects of requested type. If yes, then such objects will be further passed to it.
OurCuctomConverter converter = new();
JsonSerializerOptions options = new();
options.Converters.Add(converter);
string json = JsonSerializer.Serialize(cat, options);
Let's think about what we would like to get.
During serialization:
- So that you can register any number of interfaces and classes, and if a class or its ancestor or any of their interfaces are registered, the properties from that registered type go into JSON.
- So that a property marked with a special [Key] attribute also goes into JSON.
During deserializing:
- Have the ability to provide a ready object to populate from JSON.
- If some property of the target object is itself and already assigned, populate it rather than assigning a new object.
- If you still need to create a new object, then use the dependency injection mechanism.
- For the top-level array, that is, when we get JSON: [{...}, {...}, ..., {...}], we want to be able to fill an existing list/collection, and in one of two ways: by filling it again or by appending it to the tail. The second option can be used, for example, to load data if there are a lot of them
In both cases:
- Since we need to convert several different types, our converter should not be a JsonConverter, but a JsonConverterFactory.
So, we inherit from System.Text.Json.Serialization.JsonConverterFactory:
public class TransferJsonConverterFactory : JsonConverterFactory{}
We need to implement abstract methods:
public abstract bool CanConvert(Type typeToConvert);
public abstract JsonConverter?
CreateConverter(Type typeToConvert, JsonSerializerOptions options);
We'll come back to the implementation later when we look at how to register types and implement dependencies.
Dependency implementation and type registration
Let's try to combine the two. The reason for this may be that some types may already be registered on the host as services, and others may not. Since we can't register anything in the system IServiceProvider, we will create our own, and use the system one, if it is available. To do this, let's create a class that implements this interface:
internal class ServiceProviderImpl : IServiceProvider
{
private readonly IServiceProvider? _parentServiceProvider;
private readonly Dictionary?> _services = new();
public ServiceProviderImpl(IServiceProvider? parentServiceProvider = null)
{
_parentServiceProvider = parentServiceProvider;
}
public void AddTransient(Type key, Func? func)
{
_services[key] = func;
}
public bool IsRegistered()
{
return IsRegistered(typeof(T));
}
public bool IsRegistered(Type serviceType)
{
return _services.ContainsKey(serviceType);
}
public List GetRegistered()
{
return _services.Keys.ToList();
}
#region Реализация IServiceProvider
public object? GetService(Type serviceType)
{
if (_services.ContainsKey(serviceType))
{
if (_services[serviceType] is {} service)
{
return service.Invoke(this);
}
if (serviceType.IsClass
&& serviceType.GetConstructor(new Type[] { }) is {})
{
object? result = _parentServiceProvider?
.GetService(serviceType);
if (result is {})
{
return result;
}
return Activator.CreateInstance(serviceType);
}
}
return _parentServiceProvider?.GetService(serviceType);
}
#endregion
}
We have associated some external service provider. We can register the type with one of the overloaded AddTransient(...) methods. The name of the method kind of reminds us that the object should be created every time we call GetService(...) or GetRequiredService(...). We can pass either a registerable type or a factory method, then that type or factory method will be created, regardless of the external service provider. If we pass only a registerable type, then we try to get a new object from the external service provider, and if it's not made there, then call the public constructor without parameters. Also, our implementation answers the question if the type is registered.
Our implementation of the service provider we include a composition relation:
internal ServiceProviderImpl ServiceProvider { get; init; }
public TransferJsonConverterFactory(IServiceProvider? serviceProvider)
{
ServiceProvider = new ServiceProviderImpl(serviceProvider);
}
And here we have the implementation of the first abstract method:
public override bool CanConvert(Type typeToConvert)
{
// If deserialization is called for one of the stub types:
// AppendableListStub<> or RewritableListStub<>,
if (ServiceProvider.GetRegistered().Any(t => typeof(ListStub<>)
.MakeGenericType(new Type[] { t })
.IsAssignableFrom(typeToConvert))
)
{
return true;
}
return ServiceProvider.IsRegistered(typeToConvert);
}
The stubs, which are checked first, are used to deserialize the JSON array, as we wanted above. That is, if we deserialize to a new list, we just call the deserializer with the right type, and our custom converter is not involved at all. For example:
List cats = JsonSerializer.Deserialize>(json);
In the case where we provided our list, we do things differently. For example, to fill out the list anew:
ObservableCollection cats;
...
TransferJsonConverterFactory serializer =
new TransferJsonConverterFactory(null)
.AddTransient()
;
JsonSerializerOptions options = new();
options.Converters.Add(serializer);
serializer.Target = cats;
JsonSerializer.Deserialize>(
jsonString, options);
At the same time, if there are objects in the list that we deserialize with our converter, their carcasses are reused. Such is the resettlement of souls.
Let's pay attention to the property:
public object? Target{ ... }
Just here we clamp an existing object to fill it.
And here is the implementation of the second abstract method:
public override JsonConverter? CreateConverter(Type typeToConvert,
JsonSerializerOptions options)
{
JsonConverter converter;
Type? type = ServiceProvider.GetRegistered().Where(
t => typeof(ListStub<>).MakeGenericType(new Type[] { t })
.IsAssignableFrom(typeToConvert)
).FirstOrDefault((Type?)null);
if (type is not null)
{
converter = (JsonConverter)Activator.CreateInstance(
typeof(ListDeserializer<>)
.MakeGenericType(new Type[] { type }),
args: new object[] { this,
typeToConvert == typeof(AppendableListStub<>)
.MakeGenericType(new Type[] { type }) }
)!;
}
else
{
converter = (JsonConverter)Activator.CreateInstance(
typeof(DtoConverter<>).MakeGenericType(
new Type[] { typeToConvert }),
args: new object[] { this }
)!;
}
return converter;
}
Here we act almost the same way as in the case of CanConvert(...): if one of the stub types for lists is requested, we create a ListDeserializer<> converter, otherwise - DtoConverter<>. Both classes are heirs of JsonConverter<>.
We will not give their code here, because it is quite lengthy. If you want, you can see it in the source code.
Let's just note that our factory is associated with these objects, so although we don't have direct access to them as they are to each other, the registered types and the target object are accessed through the factory.
Conclusion
Custom converters help us build and live.
Useful Links
