N/Direct - The .NET Interoperability Resource Center 

FAQ

 

 

 

Most Valuable Professional

How do I deal with variable length structures?
Written: 6/19/2002 Updated: 6/20/2002

The .NET marshaler doesn't support structures with variable length. Every type must have a well-defined layout.

Sometimes this isn't a problem, because you rarely or never look at the structure content, you just pass around pointers to it. In this case, just use an IntPtr as the structure pointer and don't bother trying to define the structure itself. Examples of this are the SID structure used in Win32 security APIs, and the ITEMIDLIST structure used in Shell APIs (where the pointer is often referred to as a PIDL).

When you do need to look at the structure content, it's usually best to work with a dynamically allocated chunk of memory from the unmanaged heap. You can use the methods of the Marshal class to allocate the memory, and then read and write data to it as needed. For small, simple structures, you can probably skip defining the structure altogether, and work directly against the right member offsets in the memory blob. As an example, look at the FILE_NOTIFY_INFORMATION structure used in calls to the ReadDirectoryChangesW API. FILE_NOTIFY_INFORMATION contains only three 32 bit integers, followed by a Unicode string of variable length. Here's an example of how it's used.

C#
[DllImport("kernel32.dll", CharSet=CharSet.Auto, SetLastError=true)]
static extern IntPtr CreateFile(string lpFileName, uint dwDesiredAccess, uint dwShareMode,
  IntPtr lpSecurityAttributes, uint dwCreationDisposition, uint dwFlagsAndAttributes,
  IntPtr hTemplateFile);
[DllImport("kernel32.dll")]
static extern bool CloseHandle(IntPtr hObject);
[DllImport("kernel32.dll", CharSet=CharSet.Unicode, ExactSpelling=true, SetLastError=true)]
static extern bool ReadDirectoryChangesW(IntPtr hDirectory, IntPtr lpBuffer, uint nBufferLength,
  bool bWatchSubtree, uint dwNotifyFilter, out uint lpBytesReturned, IntPtr lpOverlapped,
  IntPtr lpCompletionRoutine);
// Here's how the struct FILE_NOTIFY_INFORMATION would look in C# code,
// but we don't actually use this definition, just remember the field offsets.
/*
struct FILE_NOTIFY_INFORMATION
{
  public uint NextEntryOffset;
  public uint Action;
  public uint FileNameLength;
  public char[] FileName;
}
*/
const uint FILE_LIST_DIRECTORY = 0x1;
const uint FILE_SHARE_READ = 0x1;
const uint FILE_SHARE_WRITE = 0x2;
const uint FILE_SHARE_DELETE = 0x4;
const uint OPEN_EXISTING = 3;
const uint FILE_FLAG_BACKUP_SEMANTICS = 0x2000000;
const uint FILE_NOTIFY_CHANGE_FILE_NAME = 0x1;
const uint FILE_NOTIFY_CHANGE_DIR_NAME = 0x2;
const uint FILE_NOTIFY_CHANGE_LAST_WRITE = 0x10;
  
// ...
const uint BUFSIZE = 2048;
string myDocs = Environment.GetFolderPath( Environment.SpecialFolder.Personal );
Console.WriteLine( "Monitoring name changes in {0} and subdirectories.", myDocs );
IntPtr hDir = CreateFile( myDocs, FILE_LIST_DIRECTORY, FILE_SHARE_READ | FILE_SHARE_WRITE |  FILE_SHARE_DELETE, IntPtr.Zero, OPEN_EXISTING, FILE_FLAG_BACKUP_SEMANTICS, IntPtr.Zero );
if ( hDir == IntPtr.Zero ) {
  Console.WriteLine( "CreateFile failed. " + Marshal.GetLastWin32Error() );
  return;
}
IntPtr pBuf = IntPtr.Zero;
try {
  pBuf = Marshal.AllocHGlobal( (int)BUFSIZE );
  uint bytesReturned;
  if ( ReadDirectoryChangesW( hDir, pBuf, BUFSIZE, true, FILE_NOTIFY_CHANGE_FILE_NAME | 
         FILE_NOTIFY_CHANGE_DIR_NAME | FILE_NOTIFY_CHANGE_LAST_WRITE, out bytesReturned,
         IntPtr.Zero, IntPtr.Zero ) ) {
    string[] actions = new string[] { "(unknown action) ", "Added ", "Removed ",
                                      "Modified ", "Old name ", "New name " };
    IntPtr pCurrent = pBuf;
    while ( pCurrent != IntPtr.Zero ) {
      // Read file length (in bytes) at offset 8
      int fileLen = Marshal.ReadInt32( pCurrent, 8 );
      // Read file name (fileLen/2 characters) from offset 12
      string file = Marshal.PtrToStringUni( (IntPtr)(12 + (int)pCurrent), fileLen/2 );
      // Read action at offset 4
      int action = Marshal.ReadInt32( pCurrent, 4 );
      if ( action < 1 || action >= actions.Length ) action = 0;
      Console.WriteLine( actions[action] + file );
      // Read NextEntryOffset at offset 0 and move pointer to next structure if needed
      int inc = Marshal.ReadInt32( pCurrent );
      pCurrent = inc != 0 ? (IntPtr)(inc + (int)pCurrent) : IntPtr.Zero;
    }
  }
  else
    Console.WriteLine( "ReadDirectoryChangesW failed. " + Marshal.GetLastWin32Error() );
}
finally {
  if ( pBuf != IntPtr.Zero ) Marshal.FreeHGlobal( pBuf );
  CloseHandle( hDir );
}
    

VB.NET
Declare Auto Function CreateFile Lib "kernel32.dll" (ByVal lpFileName As String, _
  ByVal dwDesiredAccess As Integer, ByVal dwShareMode As Integer, _
  ByVal lpSecurityAttributes As IntPtr, ByVal dwCreationDisposition As Integer, _
  ByVal dwFlagsAndAttributes As Integer, ByVal hTemplateFile As IntPtr) As IntPtr
Declare Function CloseHandle Lib "kernel32.dll" (ByVal hObject As IntPtr) As Boolean
Declare Function ReadDirectoryChangesW Lib "kernel32.dll" (ByVal hDirectory As IntPtr, _
  ByVal lpBuffer As IntPtr, ByVal nBufferLength As Integer, ByVal bWatchSubtree As Boolean, _
  ByVal dwNotifyFilter As Integer, ByRef lpBytesReturned As Integer, ByVal lpOverlapped As IntPtr, _
  ByVal lpCompletionRoutine As IntPtr) As Boolean
' Here's how the structure FILE_NOTIFY_INFORMATION would look in VB.NET code,
' but we don't actually use this definition, just remember the field offsets.
'
'Structure FILE_NOTIFY_INFORMATION
'  Public NextEntryOffset As Integer
'  Public Action As Integer
'  Public FileNameLength As Integer
'  Public FileName() As Char
'End Structure
Const FILE_LIST_DIRECTORY As Integer = &H1
Const FILE_SHARE_READ As Integer = &H1
Const FILE_SHARE_WRITE As Integer = &H2
Const FILE_SHARE_DELETE As Integer = &H4
Const OPEN_EXISTING As Integer = 3
Const FILE_FLAG_BACKUP_SEMANTICS As Integer = &H2000000
Const FILE_NOTIFY_CHANGE_FILE_NAME As Integer = &H1
Const FILE_NOTIFY_CHANGE_DIR_NAME As Integer = &H2
Const FILE_NOTIFY_CHANGE_LAST_WRITE As Integer = &H10
' ...
Const BUFSIZE As Integer = 2048
Dim myDocs As String = Environment.GetFolderPath(Environment.SpecialFolder.Personal)
Console.WriteLine("Monitoring changes in {0} and subdirectories.", myDocs)
Dim hDir As IntPtr = CreateFile(myDocs, FILE_LIST_DIRECTORY, FILE_SHARE_READ Or FILE_SHARE_WRITE Or _
  FILE_SHARE_DELETE, IntPtr.Zero, OPEN_EXISTING, FILE_FLAG_BACKUP_SEMANTICS, IntPtr.Zero)
If hDir.Equals(IntPtr.Zero) Then
  Console.WriteLine("CreateFile failed. " & Marshal.GetLastWin32Error())
  Return
End If
Dim pBuf As IntPtr
Try
  pBuf = Marshal.AllocHGlobal(BUFSIZE)
  Dim bytesReturned As Integer
  If ReadDirectoryChangesW(hDir, pBuf, BUFSIZE, True, FILE_NOTIFY_CHANGE_FILE_NAME Or _
       FILE_NOTIFY_CHANGE_DIR_NAME Or FILE_NOTIFY_CHANGE_LAST_WRITE, bytesReturned, _
       IntPtr.Zero, IntPtr.Zero) Then
    Dim actions() As String = New String() {"(unknown action) ", "Added ", "Removed ", _
                                            "Modified ", "Old name ", "New name "}
    Dim pCurrent As IntPtr = pBuf
    Do Until pCurrent.Equals(IntPtr.Zero)
      ' Read file length (in bytes) at offset 8
      Dim fileLen As Integer = Marshal.ReadInt32(pCurrent, 8)
      ' Read file name (fileLen/2 characters) from offset 12
      Dim file As String = Marshal.PtrToStringUni(New IntPtr(12 + pCurrent.ToInt32()), fileLen\2)
      ' Read action at offset 4
      Dim action As Integer = Marshal.ReadInt32(pCurrent, 4)
      If action < 1 OrElse action >= actions.Length Then action = 0
      Console.WriteLine(actions(action) & file)
      ' Read NextEntryOffset at offset 0 and move pointer to next structure if needed
      Dim inc As Integer = Marshal.ReadInt32(pCurrent)
      If inc = 0 Then pCurrent = IntPtr.Zero Else pCurrent = New IntPtr(inc + pCurrent.ToInt32())
    Loop
  Else
    Console.WriteLine("ReadDirectoryChangesW failed. " & Marshal.GetLastWin32Error())
  End If
Finally
  If Not pBuf.Equals(IntPtr.Zero) Then Marshal.FreeHGlobal(pBuf)
  CloseHandle(hDir)
End Try
    

The code checks for file system changes in the My Documents folder, and prints out the result the first time something changes. To try it, compile and run the code, then go to My Documents and add, rename, modify or delete a file.

For larger, more complex structures, it makes sense to define the fixed-length part of it in managed code, because it makes it easier and faster to read and write the information to and from the unmanaged buffer. The variable-length part, which usually is located at the very end of the structure, can then be located by adding the size of the fixed-length part (Marshal.SizeOf()) to the structure pointer.