Forum BigDB Serializing DatabaseObject as JSON

Discussion and help relating to the PlayerIO database solution, BigDB.

Serializing DatabaseObject as JSON

Postby HVStudios » September 11th, 2013, 4:24 am

We had a need to send a DatabaseObject from Server->Client (didn't want client to directly access a DB), and since Messages don't support the DatabaseObject type, I went ahead and created a little helper routine to serialize a DatabaseObject as JSON and send it across the network, where it can be deserialized and used as a Dictionary.

I wanted to share this code in case anyone might find it helpful, but I also wanted any thoughts on the efficiency or the wisdom of doing this operation. I have to admit that I'm very new to using LINQ, so I'm unsure what the performance implications are for the "ToDictionary" operation, or doing it recursively for that matter.

Code: Select all
using System;
using System.Collections.Generic;
using System.Linq;
using PlayerIO.GameLibrary;
using MiniJSON;

public static class DatabaseObjectExtensions
{
   public static string ToSerializedJson(this DatabaseObject dbObj)
   {
      Dictionary<string, object> dict = dbObj.ToDictionary(KeySelector, ValueSelector);
      return Json.Serialize(dict);
   }

   private static string KeySelector(KeyValuePair<string, object> entry)
   {
      return entry.Key;
   }

   private static object ValueSelector(KeyValuePair<string, object> entry)
   {
      if(entry.Value is DatabaseObject)
      {
         DatabaseObject temp = (DatabaseObject)entry.Value;
         return temp.ToDictionary(KeySelector, ValueSelector);
      }
      else
      {
         return entry.Value;
      }
   }   
}


The only external dependency for this code is MiniJSON, which is a single .cs file with JSON serialize and deserialize functionality. Maybe you can use C#'s built-in JSON parsers, but I think MiniJSON is a bit faster.

Anyway, if anyone has any comments or criticisms, please let me know. My main concern is that I am somehow doing something horribly inefficient by transferring DatabaseObjects in this way.

Thanks!
HVStudios
Paid Member
 
Posts: 20
Joined: July 30th, 2013, 7:15 pm

Re: Serializing DatabaseObject as JSON

Postby groomi » September 29th, 2013, 6:13 pm

So if the client can (presumably) modify this object and post it back to the server to be de-serialised and written to the DB, why is this any more secure? Or do I misunderstand your implementation? Surely a client (via a man in the middle or packet spoofing type of attack) can just send back a malformed JSON which will result in the database being edited anyway.

If it is just a read operation though, it seems to make equally as little sense. Give the client a connection type which is read-only and allow it to get the information directly from the DB?

Basically, I can think of no reason where this would ever be needed
groomi
Paid Member
 
Posts: 46
Joined: January 15th, 2013, 5:14 pm

Re: Serializing DatabaseObject as JSON

Postby HVStudios » September 30th, 2013, 7:07 pm

Hey groomi,

This is being used for read-only operations, which takes aways the DB modification concerns. As-is, we'd have to write some API call that takes JSON as input, deserializes it, and puts it in the correct DB location, which doesn't seem easily doable anyway.

The reason we found this useful over a direct DB access from the client was that using the client-side BigDB API requires us to hardcode database, record, and key values into our client, which we felt would be better to avoid. We decided it would be better to access data using a messaging API, which we could ensure stays consistent for all versions of the client (or, if we must introduce a breaking change, we can send a signal to older clients requesting an update).

For example, take retrieving store data from the server. Using BigDB, we can request the store data record, but we'd have to hardcode "GameSettings" DB, and "ProductData" record. Using the messaging API, we can have an API call of "RetrieveProductData", and then the client receives the data without having to know the internal representation of that data in our DB. While changes to the DB structure can break the code, the fix requires updating server-side code and not client code.

And so, the problem with the Message class: there doesn't seem to be a way to send entire DatabaseObjects as a result of API calls. Therefore, it was valuable to have a way to serialize the DatabaseObject in a generic way and send it over the network.
HVStudios
Paid Member
 
Posts: 20
Joined: July 30th, 2013, 7:15 pm

Re: Serializing DatabaseObject as JSON

Postby groomi » October 1st, 2013, 1:06 pm

Very smart answer. Now I just wish I could help you more productively :p
groomi
Paid Member
 
Posts: 46
Joined: January 15th, 2013, 5:14 pm

Re: Serializing DatabaseObject as JSON

Postby vaices » February 21st, 2014, 4:59 pm

Hi, does somebody has developed reverse Method ?
JSON to DatabaseObject ?
vaices
 
Posts: 4
Joined: September 23rd, 2013, 8:08 am

Re: Serializing DatabaseObject as JSON

Postby joh » November 28th, 2014, 1:11 pm

I have an alternative way to serialize the DabatabaseObject and its content...

Code: Select all
using MiniJSON;
using PlayerIO.GameLibrary;
using System.Collections.Generic;

public static class DatabaseObjectExtensions
{
    public static string ToSerializedJson(this DatabaseObject dbObj)
    {
        Dictionary<string, object> dict = new Dictionary<string, object>();

        visitDBO(ref dict, dbObj);

        return Json.Serialize(dict);
    }

    private static void visitDBO(ref Dictionary<string, object> dict, DatabaseObject dbObj)
    {
        foreach (KeyValuePair<string, object> entry in dbObj)
        {
            if (entry.Value is DatabaseObject)
            {
                Dictionary<string, object> dbodict = new Dictionary<string, object>();
                dict.Add(entry.Key, dbodict);
                visitDBO(ref dbodict, (DatabaseObject)entry.Value);
            }
            else if (entry.Value is DatabaseArray)
            {
                Dictionary<string, object> dbadict = new Dictionary<string, object>();
                dict.Add(entry.Key, dbadict);
                visitDBA(ref dbadict, (DatabaseArray)entry.Value);
            }
            else
            {
                dict.Add(entry.Key, entry.Value);
            }
        }
    }

    private static void visitDBA(ref Dictionary<string, object> dict, DatabaseArray dbArr)
    {
        foreach (KeyValuePair<int, object> entry in dbArr.IndexesAndValues)
        {
            if (entry.Value is DatabaseObject)
            {
                Dictionary<string, object> dbodict = new Dictionary<string, object>();
                dict.Add(entry.Key.ToString(), dbodict);
                visitDBO(ref dbodict, (DatabaseObject)entry.Value);
            }
            else if (entry.Value is DatabaseArray)
            {
                Dictionary<string, object> dbadict = new Dictionary<string, object>();
                dict.Add(entry.Key.ToString(), dbadict);
                visitDBA(ref dbadict, (DatabaseArray)entry.Value);
            }
            else
            {
                dict.Add(entry.Key.ToString(), entry);
            }
        }
    }
}


I'm using this MiniJSON version: https://gist.github.com/darktable/1411710
joh
 
Posts: 1
Joined: April 2nd, 2012, 5:19 pm

Re: Serializing DatabaseObject as JSON

Postby HVStudios » December 1st, 2014, 8:09 pm

Yeah, that looks great! It's nice to have LINQ do the heavy lifting, but then again, I'm not sure how fast it is...good to have a way to do it without relying on LINQ.

Regarding JSON to DatabaseObject, it seems somewhat tricky. On server-side, you can probably deserialize the JSON as a C# dictionary and then iterate over the dictionary, creating a DB object in the process. You know, if this entry is a string, add a string to the DB object, if it's an int add an int, etc.
HVStudios
Paid Member
 
Posts: 20
Joined: July 30th, 2013, 7:15 pm

Re: Serializing DatabaseObject as JSON

Postby Zwick » January 25th, 2017, 9:42 pm

Hey. Just sharing my simple and dummy methods Convert2DatabaseObject and Convert2DatabaseArray that are converting any IList to DatabaseArray and IEnumerable to DatabaseObject. Feel free to use it. :idea:

Code: Select all
        private DatabaseObject Convert2DatabaseObject(IEnumerable o)
        {
            var dbo = new DatabaseObject();

            foreach (KeyValuePair<string, object> entry in o)
            {
                var value = entry.Value;
                string key = entry.Key;
                if (value is Dictionary<string, object>)
                {
                    dbo.Set(key, Convert2DatabaseObject(value as IEnumerable));
                }
                else
                    if (value is IList)
                    {
                        dbo.Set(key, Convert2DatabaseArray(value as IList));
                    }
                    else
                        if (value is string)
                        {
                            dbo.Set(key, (string)value);
                        }
                        else
                            if (value is double)
                            {
                                dbo.Set(key, (double)value);
                            }
                            else
                                if (value is float)
                                {
                                    dbo.Set(key, (float)value);
                                }
                                else
                                    if (value is int)
                                    {
                                        dbo.Set(key, (int)value);
                                    }
                                    else if (value is bool)
                                    {
                                        dbo.Set(key, (bool)value);
                                    }
                                    else
                                        if (value is byte[])
                                        {
                                            dbo.Set(key, (byte[])value);
                                        }
                                        else
                                            if (value is DateTime)
                                            {
                                                dbo.Set(key, (DateTime)value);
                                            }
                                            else
                                                if (value is long)
                                                {
                                                    dbo.Set(key, (long)value);
                                                }
                                                else
                                                    if (value is uint)
                                                    {
                                                        dbo.Set(key, (uint)value);
                                                    }
            }

            return dbo;
        }


        private DatabaseArray Convert2DatabaseArray(IList o)
        {
            DatabaseArray dbo = new DatabaseArray();

            for (int i = 0; i < o.Count; i++)
            {
                var value = o[i];
                if (value is IList)
                {
                    dbo.Set(i, Convert2DatabaseArray(value as IList));
                }
                else
                    if (value is string)
                    {
                        dbo.Set(i, (string)value);
                    }
                    else
                        if (value is double)
                        {
                            dbo.Set(i, (double)value);
                        }
                        else
                            if (value is float)
                            {
                                dbo.Set(i, (float)value);
                            }
                            else
                                if (value is int)
                                {
                                    dbo.Set(i, (int)value);
                                }
                                else if (value is bool)
                                {
                                    dbo.Set(i, (bool)value);
                                }
                                else
                                    if (value is byte[])
                                    {
                                        dbo.Set(i, (byte[])value);
                                    }
                                    else
                                        if (value is DateTime)
                                        {
                                            dbo.Set(i, (DateTime)value);
                                        }
                                        else
                                            if (value is long)
                                            {
                                                dbo.Set(i, (long)value);
                                            }
                                            else
                                                if (value is uint)
                                                {
                                                    dbo.Set(i, (uint)value);
                                                }
            }

            return dbo;
        }
    }
Zwick
 
Posts: 19
Joined: December 2nd, 2012, 3:35 pm


Return to BigDB



cron