Deserialize hal+json to complex model

I have the following HAL+JSON sample:

{
    "id": "4a17d6fe-a617-4cf8-a850-0fb6bc8576fd",
    "country": "DE",
    "_embedded": {
      "company": {
        "name": "Apple",
        "industrySector": "IT",
      "owner": "Klaus Kleber",
      "_embedded": {
        "emailAddresses": [
          {
            "id": "4a17d6fe-a617-4cf8-a850-0fb6bc8576fd",
            "value": "test2@consoto.com",
            "type": "Business",
            "_links": {
              "self": {
                "href": "https://any-host.com/api/v1/customers/1234"
              }
            }
          }
        ],
        "phoneNumbers": [
          {
            "id": "4a17d6fe-a617-4cf8-a850-0fb6bc8576fd",
            "value": "01670000000",
            "type": "Business",
            "_links": {
              "self": {
                "href": "https://any-host.com/api/v1/customers/1234"
              }
            }
          }
        ],
      },
      "_links": {
        "self": {
          "href": "https://any-host.com/api/v1/customers/1234"
        },
        "phoneNumbers": {
          "href": "https://any-host.com/api/v1/customers/1234"
        },
        "addresses": {
          "href": "https://any-host.com/api/v1/customers/1234"
        },
      }
    },
  },
  "_links": {
    "self": {
      "href": "https://any-host.com/api/v1/customers/1234"
    },
    "legalPerson": {
      "href": "https://any-host.com/api/v1/customers/1234"
    },
    "naturalPerson": {
      "href": "https://any-host.com/api/v1/customers/1234"
    }
  }
}

And the following models:

public class Customer
{
    public Guid Id { get; set; }
    public string Country { get; set; }
    public LegalPerson Company { get; set; }
}
public class LegalPerson
{
    public string Name { get; set; }
    public string IndustrySector { get; set; }
    public string Owner { get; set; }
    public ContactInfo[] EmailAddresses { get; set; }
    public ContactInfo[] PhoneNumbers { get; set; }
}
public class ContactInfo
{
    public Guid Id { get; set; }
    public string Type { get; set; }
    public string Value { get; set; }
}

Now, because of the _embbeded, I can't do an out-of-the-box serialization with Newtonsoft.Json, because then Company will be null;

I was hoping to see a native hal+json support by Json.NET, but it only has one recommendation to use a custom JsonConverter.

I started to create a custom one by myself, but feels like "reinventing the wheel" for me.

So, anyone knows a smart way to get out with this?

UPDATE:

  • It's important to not change the models/classes. I can add attributes, but never change it's structures.

The most likely solution is as suggested that you create a custom converter to parse the desired models.

In this case the custom converter would need to be able to read nested paths.

This should provide a simple workaround.

public class NestedJsonPathConverter : JsonConverter {

    public override object ReadJson(JsonReader reader, Type objectType,
                                    object existingValue, JsonSerializer serializer) {
        JObject jo = JObject.Load(reader);
        var properties = jo.Properties();
        object targetObj = existingValue ?? Activator.CreateInstance(objectType);
        var resolver = serializer.ContractResolver as DefaultContractResolver;

        foreach (PropertyInfo propertyInfo in objectType.GetProperties()
                                                .Where(p => p.CanRead && p.CanWrite)) {

            var attributes = propertyInfo.GetCustomAttributes(true).ToArray();

            if (attributes.OfType<JsonIgnoreAttribute>().Any())
                continue;

            var jsonProperty = attributes.OfType<JsonPropertyAttribute>().FirstOrDefault();

            var jsonPath = (jsonProperty != null ? jsonProperty.PropertyName : propertyInfo.Name);

            if (resolver != null) {
                jsonPath = resolver.GetResolvedPropertyName(jsonPath);
            }

            JToken token = jo.SelectToken(jsonPath) ?? GetTokenCaseInsensitive(properties, jsonPath);

            if (token != null && token.Type != JTokenType.Null) {
                object value = token.ToObject(propertyInfo.PropertyType, serializer);
                propertyInfo.SetValue(targetObj, value, null);
            }
        }
        return targetObj;
    }

    JToken GetTokenCaseInsensitive(IEnumerable<JProperty> properties, string jsonPath) {
        var parts = jsonPath.Split('.');

        var property = properties.FirstOrDefault(p =>
            string.Equals(p.Name, parts[0], StringComparison.OrdinalIgnoreCase)
        );

        for (var i = 1; i < parts.Length && property != null && property.Value is JObject; i++) {
            var jo = property.Value as JObject;
            property = jo.Properties().FirstOrDefault(p =>
                string.Equals(p.Name, parts[i], StringComparison.OrdinalIgnoreCase)
            );
        }

        if (property != null && property.Type != JTokenType.Null) {
            return property.Value;
        }

        return null;
    }

    public override bool CanConvert(Type objectType) {
         //Check if any JsonPropertyAttribute has a nested property name {name}.{sub}
        return objectType
            .GetProperties()
            .Any(p =>
                p.CanRead
                && p.CanWrite
                && p.GetCustomAttributes(true)
                    .OfType<JsonPropertyAttribute>()
                    .Any(jp => (jp.PropertyName ?? p.Name).Contains('.'))
            );
    }

    public override bool CanWrite {
        get { return false; }
    }

    public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer) {
        throw new NotImplementedException();
    }
}

The original class structure now does not need to change, with only the properties that require custom paths needing to be decorated with JsonPropertyAttribute that indicates the path to populate the property.

In this example

public class Customer {
    public Guid Id { get; set; }
    public string Country { get; set; }
    [JsonProperty("_embedded.company")]
    public LegalPerson Company { get; set; }
}
public class LegalPerson {
    public string Name { get; set; }
    public string IndustrySector { get; set; }
    public string Owner { get; set; }
    [JsonProperty("_embedded.emailAddresses")]
    public ContactInfo[] EmailAddresses { get; set; }
    [JsonProperty("_embedded.phoneNumbers")]
    public ContactInfo[] PhoneNumbers { get; set; }
}

Just include the converter as needed.

var settings = new JsonSerializerSettings {
    ContractResolver = new DefaultContractResolver {
        NamingStrategy = new CamelCaseNamingStrategy()
    }
};
settings.Converters.Add(new NestedJsonPathConverter());

var customer = JsonConvert.DeserializeObject<Customer>(json, settings);

The two important parts of the code are the GetTokenCaseInsensitive method that searches for the requested token and allows for nested paths that can be case-insensitive.

JToken GetTokenCaseInsensitive(IEnumerable<JProperty> properties, string jsonPath) {
    var parts = jsonPath.Split('.');

    var property = properties.FirstOrDefault(p =>
        string.Equals(p.Name, parts[0], StringComparison.OrdinalIgnoreCase)
    );

    for (var i = 1; i < parts.Length && property != null && property.Value is JObject; i++) {
        var jo = property.Value as JObject;
        property = jo.Properties().FirstOrDefault(p =>
            string.Equals(p.Name, parts[i], StringComparison.OrdinalIgnoreCase)
        );
    }

    if (property != null && property.Type != JTokenType.Null) {
        return property.Value;
    }

    return null;
}

and the overridden CanConvert which will check of any properties have nested paths

public override bool CanConvert(Type objectType) {
     //Check if any JsonPropertyAttribute has a nested property name {name}.{sub}
    return objectType
        .GetProperties()
        .Any(p => 
            p.CanRead 
            && p.CanWrite
            && p.GetCustomAttributes(true)
                .OfType<JsonPropertyAttribute>()
                .Any(jp => (jp.PropertyName ?? p.Name).Contains('.'))
        );
}

Deserialize HAL responses · Issue #30 · cognitect-labs/aws-api , I don't know the proper place/way to do deserialization, but here might be a pointer do get the embedded data of the HAL response:  HAL has a media type for both the JSON and XML variants, whos names are application/hal+json and application/hal+xml respectively. When serving HAL over HTTP, the Content-Type of the response should contain the relevant media type name. The structure of a HAL document Minimum valid document. A HAL document must at least contain an empty resource.

Possible solution is to use custom JsonConverter but not implement all converting logic from scratch.

Some time ago I found and updated JsonPathConverter which allows to use property path for JsonProperty attribute. For example in your case

 [JsonProperty("_embedded.company")]
 public LegalPerson Company { get; set; }

So your models with attributes will look like:

[JsonConverter(typeof(JsonPathConverter))]
public class Customer
{
    [JsonProperty("id")]
    public Guid Id { get; set; }

    [JsonProperty("country")]
    public string Country { get; set; }

    [JsonProperty("_embedded.company")]
    public LegalPerson Company { get; set; }
}

[JsonConverter(typeof(JsonPathConverter))]
public class LegalPerson
{
    [JsonProperty("name")]
    public string Name { get; set; }

    [JsonProperty("industrySector")]
    public string IndustrySector { get; set; }

    [JsonProperty("owner")]
    public string Owner { get; set; }

    [JsonProperty("_embedded.emailAddresses")]
    public ContactInfo[] EmailAddresses { get; set; }

    [JsonProperty("_embedded.phoneNumbers")]
    public ContactInfo[] PhoneNumbers { get; set; }
}

public class ContactInfo
{
    [JsonProperty("id")]
    public Guid Id { get; set; }

    [JsonProperty("value")]
    public string Type { get; set; }

    [JsonProperty("type")]
    public string Value { get; set; }
}

The code of JsonPathConverter is this. But I believe you can improve it.

  public class JsonPathConverter : JsonConverter
{
    public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
    {
        var properties = value.GetType().GetRuntimeProperties().Where(p => p.CanRead && p.CanWrite);
        JObject main = new JObject();

        foreach (PropertyInfo prop in properties)
        {
            JsonPropertyAttribute att = prop.GetCustomAttributes(true)
                .OfType<JsonPropertyAttribute>()
                .FirstOrDefault();

            string jsonPath = att != null ? att.PropertyName : prop.Name;

            if (serializer.ContractResolver is DefaultContractResolver resolver)
                jsonPath = resolver.GetResolvedPropertyName(jsonPath);

            var nesting = jsonPath.Split('.');
            JObject lastLevel = main;

            for (int i = 0; i < nesting.Length; ++i)
            {
                if (i == (nesting.Length - 1))
                {
                    lastLevel[nesting[i]] = new JValue(prop.GetValue(value));
                }
                else
                {
                    if (lastLevel[nesting[i]] == null)
                        lastLevel[nesting[i]] = new JObject();

                    lastLevel = (JObject) lastLevel[nesting[i]];
                }
            }
        }

        serializer.Serialize(writer, main);
    }

    public override object ReadJson(JsonReader reader, Type objectType, object existingValue,
        JsonSerializer serializer)
    {
        var jo = JToken.Load(reader);
        object targetObj = Activator.CreateInstance(objectType);

        foreach (PropertyInfo prop in objectType.GetRuntimeProperties().Where(p => p.CanRead && p.CanWrite))
        {
            var attributes = prop.GetCustomAttributes(true).ToArray();

            JsonIgnoreAttribute ignoreAttribute = attributes.OfType<JsonIgnoreAttribute>().FirstOrDefault();

            if (ignoreAttribute != null)
                continue;

            JsonPropertyAttribute att = attributes.OfType<JsonPropertyAttribute>().FirstOrDefault();

            string jsonPath = att != null ? att.PropertyName : prop.Name;

            if (serializer.ContractResolver is DefaultContractResolver resolver)
                jsonPath = resolver.GetResolvedPropertyName(jsonPath);

            if (!Regex.IsMatch(jsonPath, @"^[a-zA-Z0-9_.-]+$"))
                throw new InvalidOperationException(
                    $"JProperties of JsonPathConverter can have only letters, numbers, underscores, hyphens and dots but name was ${jsonPath}."); // Array operations not permitted

            JToken token = jo.SelectToken(jsonPath);

            if (token != null && token.Type != JTokenType.Null)
            {
                object value;
                var jsonConverterAttr = attributes.OfType<JsonConverterAttribute>().FirstOrDefault();
                if (jsonConverterAttr == null)
                {
                    value = token.ToObject(prop.PropertyType, serializer);
                }
                else
                {
                    var converter = (JsonConverter) Activator.CreateInstance(jsonConverterAttr.ConverterType,
                        jsonConverterAttr.ConverterParameters);

                    var r = token.CreateReader();
                    r.Read();
                    value = converter.ReadJson(r, prop.PropertyType, prop.GetValue(targetObj),
                        new JsonSerializer());
                }

                prop.SetValue(targetObj, value, null);
            }
        }

        return targetObj;
    }

    public override bool CanConvert(Type objectType)
    {
        // CanConvert is not called when [JsonConverter] attribute is used
        return false;
    }
}

And finally you can use it like this:

var json = "*your json string here*";
var customer = JsonConvert.DeserializeObject<Customer>(json);

Jackson2HalModule.HalLinkListDeserializer (Spring HATEOAS , org.springframework.hateoas.hal. deserialize, getDelegatee, getEmptyValue, getEmptyValue, public List<Link> deserialize(com.fasterxml.jackson.core. The documentation for JSON.NET shows you how to deserialize. The only bit you need to tweak is to account for the property names with colons in them. You can overcome this by tagging each affected property with a JsonPropertyAttribute, passing in the name expected to be found in the JSON (i.e. the name with the colons).

The company object will be under Embedded _embedded object.

like

    class Program
    {
        static void Main(string[] args)
        {
            string json = "{\"id\": \"4a17d6fe-a617-4cf8-a850-0fb6bc8576fd\",\"country\": \"DE\",\"_embedded\": {\"company\": {\"name\": \"Apple\",\"industrySector\": \"IT\",\"owner\": \"Klaus Kleber\",\"_embedded\": {\"emailAddresses\": [{\"id\": \"4a17d6fe-a617-4cf8-a850-0fb6bc8576fd\",\"value\": \"test2@consoto.com\",\"type\": \"Business\",\"_links\": {\"self\": {\"href\": \"https://any-host.com/api/v1/customers/1234\"}}}],\"phoneNumbers\": [{\"id\": \"4a17d6fe-a617-4cf8-a850-0fb6bc8576fd\",\"value\": \"01670000000\",\"type\": \"Business\",\"_links\": {\"self\": {\"href\": \"https://any-host.com/api/v1/customers/1234\"}}}],},\"_links\": {\"self\": {\"href\": \"https://any-host.com/api/v1/customers/1234\"},\"phoneNumbers\": {\"href\": \"https://any-host.com/api/v1/customers/1234\"},\"addresses\": {\"href\": \"https://any-host.com/api/v1/customers/1234\"},}},},\"_links\": {\"self\": {\"href\": \"https://any-host.com/api/v1/customers/1234\"},\"legalPerson\": {\"href\": \"https://any-host.com/api/v1/customers/1234\"},\"naturalPerson\": {\"href\": \"https://any-host.com/api/v1/customers/1234\"}}}";

            CustomerJson results = JsonConvert.DeserializeObject<CustomerJson>(json);
            Customer customer = new Customer()
            {
                Id = results.id,
                Country = results.country,
                Company = new LegalPerson()
                {
                    EmailAddresses = results._embedded.company._embedded.emailAddresses,
                    PhoneNumbers = results._embedded.company._embedded.phoneNumbers,
                    IndustrySector = results._embedded.company.industrySector,
                    Name = results._embedded.company.name,
                    Owner = results._embedded.company.owner
                }
            };

        }

    }

    public class EmbeddedContactInfoJson
    {
        public ContactInfo[] emailAddresses { get; set; }
        public ContactInfo[] phoneNumbers { get; set; }
    }
    public class CompanyJson
    {
        public string name { get; set; }
        public string industrySector { get; set; }
        public string owner { get; set; }
        public EmbeddedContactInfoJson _embedded { get; set; }
        public EmbeddedLinksJson _links { get; set; }
    }

    public class EmbeddedJson
    {
        public CompanyJson company { get; set; }
    }
    public class HrefJson
    {
        public string href { get; set; }
    }

    public class EmbeddedLinksJson
    {
        public HrefJson self { get; set; }
        public HrefJson phoneNumbers { get; set; }
        public HrefJson addresses { get; set; }
    }
    public class LinksJson
    {
        public HrefJson self { get; set; }
        public HrefJson legalPerson { get; set; }
        public HrefJson naturalPerson { get; set; }
    }
    public class CustomerJson
    {
        public Guid id { get; set; }
        public string country { get; set; }
        public EmbeddedJson _embedded { get; set; }
        public LinksJson _links { get; set; }
    }

    public class Customer
    {
        public Guid Id { get; set; }
        public string Country { get; set; }
        public LegalPerson Company { get; set; }
    }
    public class LegalPerson
    {
        public string Name { get; set; }
        public string IndustrySector { get; set; }
        public string Owner { get; set; }
        public ContactInfo[] EmailAddresses { get; set; }
        public ContactInfo[] PhoneNumbers { get; set; }
    }
    public class ContactInfo
    {
        public Guid Id { get; set; }
        public string Type { get; set; }
        public string Value { get; set; }
    }

Spring HATEOAS, If you returned such an instance from a Spring MVC or WebFlux controller and the client sent an Accept header set to application/hal+json , the  hal-serializer. A Node.js framework agnostic library for serializing your data to HAL compliant responses (a specification for building APIs in JSON).. Installation

Deserialize JSON containing (_links and _embedded) using spring , Finally, I found a better way to use these application / hal + json APIs. Spring hateoas actually provides a client that's almost ready to use:  Stack Overflow for Teams is a private, secure spot for you and your coworkers to find and share information. Learn more Deserialize JSON containing (_links and _embedded) using spring-hateoas

What is the convenient way to deserialize JSON(links + embedded , So we have the following HAL+JSON representation and we want to deserialize it using spring-hateoas : { "id": "1", "title": "album title", "artistId": "1", "stockLevel":  In case the schema is used for both directions to serialize and to deserialize the halogen.schema.Accessor can be passed with both getter and setter specified. Getter is a string or callable in order to get the value from a model, and setter is a string or callable that knows where the deserialized value should be stored.

Support deserialization for hal+json (needs more language handling , Support deserialization for hal+json (needs more language handling tests). Needs work. Project: Drupal core. Version: 8.9.x-dev. Component:. How to Use the Serializer¶. Symfony provides a serializer to serialize/deserialize to and from objects and different formats (e.g. JSON or XML). Before using it, read the Serializer component docs to get familiar with its philosophy and the normalizers and encoders terminology.

Comments
  • Did you take a look into hal serialize: npmjs.com/package/hal-serializer ?
  • @AhmedBinGamal I will do, but that's a node solution, I'm looking for a c# permanent one.
  • You said you can't change your models, how about extending or overriding them?
  • It's going to be a problem. The scenario I have here is, full working and on a production system, and the source of data schema was changed to hal+json response. I can't go and change the struct of, or extend/override, a model - otherwise, I will need to touch that whole system. I need a pinpoint approach. Add some attributes - which I can do at runtime - or add JsonConverter, or other options that I did not figure out.
  • maybe just use manual LINQ to JSON -> objects version without any deserialization? It might be easier than writing deserializer
  • Also worked like a charm. You may see it working here.
  • @ThiagoLunardi note that with this example you can exclude the JsonProperty attributes as once they match the json naming they will be mapped automatically
  • I upvoted your answer because it works with less intervention than @Ivan one. I will wait until the bounty expiration for more answers if any. Thanks a lot, for now, kudos!
  • This worked as expected with very low intervention to models. Great! You can see working here. I will try @Nkosi version now.
  • @Nkosi I reviewed the code of JsonPathConverter and realized that it could be implemented better. And here it is. According to my Benchmarks it works about 2 times faster and supports more Newtonsoft Json attributes like JsonIgnore and JsonConverter.