Home > Algorithm, C-Sharp, Code, Microsoft > C# and Serializing Structs to byte arrays

C# and Serializing Structs to byte arrays

So I have a application in C# that is communicating to something else via UDP messages. The messages are byte packed arrays of data. There are several bytes of header and then N bytes of data whos format is described in the header. I might send several Ints, or a string, or whatever.

This is pretty easy to handle in C/C++ as you can get raw pointers to everything. So if each message type is a struct, I can get a char* to the address of the struct and just read N bytes and send that out.

The problem on the C# end is this isnt so easy. If I want to pull messages apart and put them back into structs I have to do this manually each multi byte field at a time. So I was left with a function call for each message type to serialize it, and then another to deserialize it, each one having to know exactly what was in each message, so if I changed a message I’d have to edit all the handling functions. What I needed was a way to jsut get the raw bytes the in the received array into a struct of the right type, and vice versa. Fortunately there are some tricky ways to make this super easy.

First, we have to make sure the structs we declare in C# and packed. Here is an example:

using System.Runtime.InteropServices;

[StructLayout(LayoutKind.Sequential, Pack=1)]
struct msg1{
     int a;
     int b
     short c;
}

This will ensure there are no gaps between elements in the struct.

Next, we need a way to take an array of Bytes and copy it directly into the struct:

public static T DeserializeMsg< T >(Byte[] data) where T : struct
        {
            int objsize = Marshal.SizeOf(typeof(T));
            IntPtr buff = Marshal.AllocHGlobal(objsize);

            Marshal.Copy(data, 0, buff, objsize);

            T retStruct = (T)Marshal.PtrToStructure(buff, typeof(T));

            Marshal.FreeHGlobal(buff);

            return retStruct;
        }

Lets say we have a msg1 struct populated and we want to send that out over UDP. We need to serialize it back to raw bytes. This function will do just that:

public static Byte[] SerializeMessage< T >(T msg) where T : struct
        {
            int objsize = Marshal.SizeOf(typeof(T));
            Byte[] ret = new Byte[objsize];

            IntPtr buff = Marshal.AllocHGlobal(objsize);

            Marshal.StructureToPtr(msg, buff, true);

            Marshal.Copy(buff, ret, 0, objsize);

            Marshal.FreeHGlobal(buff);

            return ret;
        }

Thats it!

qrcode

  1. December 30th, 2009 at 07:25 | #1

    hi,

    First of all. Thanks very much for your useful post.

    I just came across your blog and wanted to drop you a note telling you how impressed I was with the information you have posted here.

    Please let me introduce you some info related to this post and I hope that it is useful for .Net community.

    There is a good C# resource site, Have alook

    http://www.csharptalk.com/2009/09/c-array.html
    http://www.csharptalk.com/2009/10/creating-arrays.html

    simi

  2. March 18th, 2010 at 11:59 | #2

    Hi,

    Can my structure contain string inside or string array? But having string is not fixed length, does this method still work?

  3. Mike
    April 26th, 2010 at 03:22 | #3

    Thanks so much for posting this. I’ve been trying to make this work in a generic way for ages until I found your blog. Awesome! Well impressed…

  4. David
    April 26th, 2010 at 04:26 | #4

    @ssteo

    Strings present more interesting problems. Given that they are dynamic in size, you can’t just include them in the the struct and serialize like above. In C# a “string” type is a reference (aka pointer), so the actual string characters aren’t in your struct anyway, just the reference.

    I would expect when you call sizeof(T) on a struct with strings you would get the size of the various ints, etc, and the size of the reference type of string.

    You would need some clever way of holding the space in the struct for your string and then going back and doing a memcpy into that during serialize.

    I’m not sure I can think of an easy solution off hand for strings…

  5. David
    April 26th, 2010 at 04:27 | #5

    @Mike

    Thanks Mike! Glad we can be of help :)

  6. huynhdt
    May 20th, 2010 at 03:50 | #6

    @David
    Could you give a pie of example. Thanks.

  7. May 4th, 2011 at 17:49 | #7

    THANK YOU SO MUCH!!! I’ve been stuck on this problem for a couple of days…

    PD: Sorry my shitty English…

  8. Jan de Jager
    October 2nd, 2011 at 10:18 | #8

    This is an awesome post. But my question if how would you go about using this solution on a on variable bit length structure. i.e.

    field 1 = bit 15 to 13
    field 2 = bit 12 to 10
    field 3 = bit 9 to 0

    etc…

  9. Jimmy JoeBob Alooba
    December 6th, 2011 at 15:04 | #9

    Actually, strings are no problem.

    byte[] ToBytes(string ThisString)
    {
    byte[] StringBytes = System.Text.Encoding.UTF8.GetBytes(ThisString);
    return StringBytes;
    }

    string FromBytes(byte[] TheseBytes)
    {
    string ThisString = new String(System.Text.Encoding.UTF8.GetChars(TheseBytes));
    return ThisString;
    }

    ..Of course, you may have to do some byte reversals depending on your data transmission format (such as over networks).

  10. David
    December 13th, 2011 at 05:09 | #10

    The problem is not converting strings to byte arrays and back. When you have a struct with the string variable in it, the struct itself only stores a reference to the string. The actual string data is stored in some memory location allocated for that string. So if you do a SizeOf(MyStruct), it does not include the variable length of the string, just the size in bytes of the reference to the string. So when you serialize the struct you just get the address of the string, which is meaningless on the system you send the message to.

    You could do something like append the real string as a byte[ ] to the end of the message and then use the bytes where the string reference was in the struct’s byte[ ] to store the length of the string. Then on the receiving end you would deserialize back the struct, set the MyStruct.someSrtring = new string(bytes appended to the message).

  1. No trackbacks yet.

Powered by WP Hashcash