C# and Serializing Structs to byte arrays
So I have a application in C# that is communicating to something else via UDP messages. The messages are byte packed arrays of data. There are several bytes of header and then N bytes of data whos format is described in the header. I might send several Ints, or a string, or whatever.
This is pretty easy to handle in C/C++ as you can get raw pointers to everything. So if each message type is a struct, I can get a char* to the address of the struct and just read N bytes and send that out.
The problem on the C# end is this isnt so easy. If I want to pull messages apart and put them back into structs I have to do this manually each multi byte field at a time. So I was left with a function call for each message type to serialize it, and then another to deserialize it, each one having to know exactly what was in each message, so if I changed a message I’d have to edit all the handling functions. What I needed was a way to jsut get the raw bytes the in the received array into a struct of the right type, and vice versa. Fortunately there are some tricky ways to make this super easy.
First, we have to make sure the structs we declare in C# and packed. Here is an example:
using System.Runtime.InteropServices;
[StructLayout(LayoutKind.Sequential, Pack=1)]
struct msg1{
int a;
int b
short c;
}
This will ensure there are no gaps between elements in the struct.
Next, we need a way to take an array of Bytes and copy it directly into the struct:
public static T DeserializeMsg< T >(Byte[] data) where T : struct
{
int objsize = Marshal.SizeOf(typeof(T));
IntPtr buff = Marshal.AllocHGlobal(objsize);
Marshal.Copy(data, 0, buff, objsize);
T retStruct = (T)Marshal.PtrToStructure(buff, typeof(T));
Marshal.FreeHGlobal(buff);
return retStruct;
}
Lets say we have a msg1 struct populated and we want to send that out over UDP. We need to serialize it back to raw bytes. This function will do just that:
public static Byte[] SerializeMessage< T >(T msg) where T : struct
{
int objsize = Marshal.SizeOf(typeof(T));
Byte[] ret = new Byte[objsize];
IntPtr buff = Marshal.AllocHGlobal(objsize);
Marshal.StructureToPtr(msg, buff, true);
Marshal.Copy(buff, ret, 0, objsize);
Marshal.FreeHGlobal(buff);
return ret;
}
Thats it!
hi,
First of all. Thanks very much for your useful post.
I just came across your blog and wanted to drop you a note telling you how impressed I was with the information you have posted here.
Please let me introduce you some info related to this post and I hope that it is useful for .Net community.
There is a good C# resource site, Have alook
http://www.csharptalk.com/2009/09/c-array.html
http://www.csharptalk.com/2009/10/creating-arrays.html
simi
Hi,
Can my structure contain string inside or string array? But having string is not fixed length, does this method still work?
Thanks so much for posting this. I’ve been trying to make this work in a generic way for ages until I found your blog. Awesome! Well impressed…
@ssteo
Strings present more interesting problems. Given that they are dynamic in size, you can’t just include them in the the struct and serialize like above. In C# a “string” type is a reference (aka pointer), so the actual string characters aren’t in your struct anyway, just the reference.
I would expect when you call sizeof(T) on a struct with strings you would get the size of the various ints, etc, and the size of the reference type of string.
You would need some clever way of holding the space in the struct for your string and then going back and doing a memcpy into that during serialize.
I’m not sure I can think of an easy solution off hand for strings…
@Mike
Thanks Mike! Glad we can be of help
@David
Could you give a pie of example. Thanks.
THANK YOU SO MUCH!!! I’ve been stuck on this problem for a couple of days…
PD: Sorry my shitty English…
This is an awesome post. But my question if how would you go about using this solution on a on variable bit length structure. i.e.
field 1 = bit 15 to 13
field 2 = bit 12 to 10
field 3 = bit 9 to 0
etc…
Actually, strings are no problem.
byte[] ToBytes(string ThisString)
{
byte[] StringBytes = System.Text.Encoding.UTF8.GetBytes(ThisString);
return StringBytes;
}
string FromBytes(byte[] TheseBytes)
{
string ThisString = new String(System.Text.Encoding.UTF8.GetChars(TheseBytes));
return ThisString;
}
..Of course, you may have to do some byte reversals depending on your data transmission format (such as over networks).
The problem is not converting strings to byte arrays and back. When you have a struct with the string variable in it, the struct itself only stores a reference to the string. The actual string data is stored in some memory location allocated for that string. So if you do a SizeOf(MyStruct), it does not include the variable length of the string, just the size in bytes of the reference to the string. So when you serialize the struct you just get the address of the string, which is meaningless on the system you send the message to.
You could do something like append the real string as a byte[ ] to the end of the message and then use the bytes where the string reference was in the struct’s byte[ ] to store the length of the string. Then on the receiving end you would deserialize back the struct, set the MyStruct.someSrtring = new string(bytes appended to the message).