Richard Boyd Richard Boyd - 10 months ago 37
Vb.net Question

Improving performance when Marshaling to a complex structure

This is a follow up from question
.net file random access recoard locking

I have been able to make the reading and marshalling work ok but the performance is a lot slower then using Fileget().

There is a few articles around e.g. https://msdn.microsoft.com/en-us/library/ff647812.aspx

However we do have to use Non-Blittable entities unfortunately and they don't provide working examples on how to make performance gains.

Examples of current code being used:

<StructLayout(LayoutKind.Sequential, _
CharSet:=Runtime.InteropServices.CharSet.Ansi, Pack:=1)>
Structure SALbchCX 'SAL555
<MarshalAs(UnmanagedType.ByValArray, SizeConst:=1)>
<VBFixedString(1)> Public XJ1s As Char() 'H/D
<MarshalAs(UnmanagedType.ByValArray, SizeConst:=8)>
<VBFixedString(8)> Public XJ2s As Char() '
<MarshalAs(UnmanagedType.ByValArray, SizeConst:=8)>
<VBFixedString(8)> Public XJ3s As Char() '
<MarshalAs(UnmanagedType.R4, SizeConst:=4)>
Public XJ4 As Single
Public XJ5 As Single
<MarshalAs(UnmanagedType.R8, SizeConst:=8)>
Public XJ6d As Double
<MarshalAs(UnmanagedType.R4, SizeConst:=4)>
Public XJ7 As Single
<MarshalAs(UnmanagedType.ByValArray, SizeConst:=61)>
<VBFixedString(61)> Public XJ8s As Char()
<MarshalAs(UnmanagedType.ByValArray, SizeConst:=1)>
<VBFixedString(1)> Public XJ8bs As Char()
<MarshalAs(UnmanagedType.ByValArray, SizeConst:=1)>
<VBFixedString(1)> Public XJ8cs As Char()
<MarshalAs(UnmanagedType.ByValArray, SizeConst:=1)>
<VBFixedString(1)> Public XJ8ds As Char()
<MarshalAs(UnmanagedType.ByValArray, SizeConst:=1)>
<VBFixedString(1)> Public XJ8es As Char()
<MarshalAs(UnmanagedType.ByValArray, SizeConst:=1)>
<VBFixedString(1)> Public XJ8fs As Char()
<MarshalAs(UnmanagedType.I4, SizeConst:=4)>
Public XJ8Al As Integer
<MarshalAs(UnmanagedType.R4, SizeConst:=4)>
Public XJ9 As Single
<MarshalAs(UnmanagedType.ByValArray, SizeConst:=11)>
<VBFixedString(11)> Public XJAs As Char()
<MarshalAs(UnmanagedType.ByValArray, SizeConst:=30)>
<VBFixedString(30)> Public XJAAs As Char()
<MarshalAs(UnmanagedType.ByValArray, SizeConst:=9)>
<VBFixedString(9)> Public XJBs As Char()
<MarshalAs(UnmanagedType.ByValArray, SizeConst:=35)>
<VBFixedString(35)> Public XJBBs As Char()
<MarshalAs(UnmanagedType.ByValArray, SizeConst:=5)>
<VBFixedString(5)> Public XJCs As Char()
<MarshalAs(UnmanagedType.ByValArray, SizeConst:=7)>
<VBFixedString(7)> Public XJDs As Char()
<MarshalAs(UnmanagedType.ByValArray, SizeConst:=8)>
<VBFixedString(8)> Public XJDas As Char()
<MarshalAs(UnmanagedType.R4, SizeConst:=4)>
Public XJE As Single
<MarshalAs(UnmanagedType.ByValArray, SizeConst:=1)>
<VBFixedString(1)> Public XJFs As Char()
<MarshalAs(UnmanagedType.ByValArray, SizeConst:=6)>
<VBFixedString(6)> Public XJGs As Char()
<MarshalAs(UnmanagedType.ByValArray, SizeConst:=30)>
<VBFixedString(30)> Public XJHs As Char()
<MarshalAs(UnmanagedType.ByValArray, SizeConst:=30)>
<VBFixedString(30)> Public XJIs As Char()
<MarshalAs(UnmanagedType.ByValArray, SizeConst:=30)>
<VBFixedString(30)> Public XJJs As Char()
<MarshalAs(UnmanagedType.ByValArray, SizeConst:=30)>
<VBFixedString(30)> Public XJKs As Char()
<MarshalAs(UnmanagedType.ByValArray, SizeConst:=35)>
<VBFixedString(35)> Public XJKKs As Char()
<MarshalAs(UnmanagedType.R4, SizeConst:=4)>
Public XJL As Single
<MarshalAs(UnmanagedType.ByValArray, SizeConst:=1)>
<VBFixedString(1)> Public XJMs As Char()
<MarshalAs(UnmanagedType.ByValArray, SizeConst:=2)>
<VBFixedString(2)> Public XJNs As Char()
<MarshalAs(UnmanagedType.ByValArray, SizeConst:=70)>
<VBFixedString(70)> Public XJOs As Char()
<MarshalAs(UnmanagedType.R4, SizeConst:=4)>
Public XJP As Single
Public XJQ As Single
Public XJR As Single
<MarshalAs(UnmanagedType.ByValArray, SizeConst:=1)>
<VBFixedString(1)> Public XJSs As Char()
<MarshalAs(UnmanagedType.ByValArray, SizeConst:=128)>
<VBFixedString(128)> Public XJTs As Char()
End Structure

....

Public SAL555 As SALbchCX

....
Dim f5 As New FileStream(coUC, FileMode.Open, _
FileAccess.ReadWrite, FileShare.ReadWrite, Marshal.SizeOf(SAL555))


For X = 1 To 10
GetBatchRec(X, f5) 'populates SAL555
Next X
....


Public Sub GetBatchRec(RecNumber As Integer, File As FileStream)

Dim b() As Byte
ReDim b(Marshal.SizeOf(SAL555) - 1)
File.Seek((RecNumber - 1) * 600, SeekOrigin.Begin) 'Marshal.Size(SAL555)
File.Read(b, 0, b.Length)
Dim h = GCHandle.Alloc(b, GCHandleType.Pinned)
SAL555 = Marshal.PtrToStructure(Of SALbchCX)(h.AddrOfPinnedObject())
h.Free()

End Sub


We are using ByValArray rather then ByValTStr as ByValTStr did not capture the last char of the string due to it expecting a ending return char. We copy the char() to strings using SAL5.XJIs = New String(SAL555.XJIs) which again takes up processing as its around 2ms to do.

The main two parts of the code which seem to be taking up most of the time are the following.

File.Seek((RecNumber - 1) * 600, SeekOrigin.Begin) 'Marshal.Size(SAL555)
SAL555 = Marshal.PtrToStructure(Of SALbchCX)(h.AddrOfPinnedObject())


Each operation takes around 10ms and just using fileget() uses around 8ms so we are taking around twice as slow even before to copy the char() to strings.

I was hopping that someone had a better idea of how to read/marshal the data to a complex structure which is a bit more efficient.

Any suggestions welcome, thanks Richard

FYI: going to ignore answers related to "Stop using flat files" as this is not an option as present :)

Answer Source

After extensive performance testing we discovered that the solution of Marshaling is actually faster then vb6 ported fileget().

The misleading thing was that the development environment was taking an age to run through the loops however when running the compiled EXE or the visual studio inbuilt performance profiler the performance of the marshalling was better then fileget() and we are also not hitting locked records prematurely, win/win!!

Learn something new every day!

Thanks, Richard.

Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download