jeudi 8 août 2019

Create RavenDB index based on dynamic properties

In RavenDB 4.2, I want to create an Index/Map based on a dynamic object. Better put, on dynamic properties which are not known at compile-time.

Here is an example of the raw JSON I'm ingesting:

{
    "id": "A",
    "detections": 
    [
        {
            "steps": [
                {
                    "object": {
                        "id": "A1",
                        "target": {
                            "domain_name": "foobar.com" 
                        }
                    },
                    "object": {
                        "id": "A2",
                        "target": {
                            "ipv4": "127.0.0.1"
                        }
                    }
                }
            ]
        }

    ]
}

The above sample is ingested from a 3rd party and stored in a RavenDB collection. Roughly translated, the following model has the challenge:

public class Step
{
    public string Id { get; set; }
    public DateTime When {get; set;}
    public dynamic Object { get; set; } // aware that it's not handy naming
}

The pickle in this is that the object.target.X property name is dynamic. They cannot be strong-typed and can be a lot of things, like: domain_name, ipv4, ipv6, dns, shoe_size, hair_colour etc. This is why the entire steps.object is ingested and stored as either System.Object or dynamic.

My objective is to basically do a SelectMany() on each object.target and extract the property name (key) and value. This would make my RavenDB Index something like this:

public class StepsIndex : AbstractIndexCreationTask<Models.Step, StepsIndex.Result>
{
    public class Result
    {
        public DateTime When { get; set; }
        public string TargetKey { get; set; }
        public string TargetValue { get; set; }
        // ... removed other properties for brevity
    }

    public StepsIndex()
    {
        Map = steps => 
            from block in blocks
            from detection in blocks.Detections
            from step in detection.Steps

            select new Result
            {
                // extract property name (key), like 'domain_name'
                TargetKey = step.Object.target.GetType().GetProperties()[0].Name,

                // extract property value, like 'foobar.com'
                TargetValue = step.Object.target.GetType().GetProperty(s.Object.target.GetType().GetProperties()[0].Name).GetValue(s.Object.target, null)
            };
    }
}

Unfortunately this doesn't work due to step.Object being dynamic and resulting in the following error during compile-time:

Error [CS1963] An expression tree may not contain a dynamic operation

Second option I've tried is to cast it to JSON in the expression, which also fails because Raven's projection is not aware of Newtonsoft.Json during runtime:

// Error CS0103: The name 'JObject' does not exist in the current context
// Error CS0103: The name 'JsonConvert' does not exist in the current context
TargetKey = JObject.Parse(JsonConvert.SerializeObject(ass.Object))["target"][0].Value<string>(),

A third option I thought of was perhaps changing the dynamic Object to System.Object Object, but haven't found a neat way to extract the property key/values without knowning the property.

The question: how can I extract these dynamic property keys and values and Map them to a RavenDB index?





Aucun commentaire:

Enregistrer un commentaire