Mixing Native and Managed Types in C++

Wow, its turning into a busy month. I just haven’t had any time to blog despite having a number of interesting topics to cover. I’ll try to get a few of them written soon. Here’s a topic from one of my regular readers.

The CLR naturally supports mixing managed and native method calls allowing you to easily call native functions from managed functions (which are of course natively compiled before execution) and visa versa. This is all largely transparent in C++. What’s not as transparent is how to mix managed and native types. The reason is that there is a greater distinction between native and managed types compared with function calls. Beside obvious differences such as those introduced by different calling conventions and virtual invocation, function calls aren’t all that different. Types however require a bit more help from the programmer/compiler since native and managed types can have very different characteristics. This is very evident in C# as you often need to decorate native type definitions with all kinds of attributes to control memory layout and marshalling. Fortunately for the C++ programmer the compiler takes care of much of this when you define or include native type definitions such as those found in the various Windows header files, but the programmer is still responsible for telling the compiler just how those types are to be used.

Visual C++ provides many of the building blocks for mixing native and managed types but in some cases you need to write a little code to help it along. Fortunately C++/CLI is very capable. Let’s consider a few different scenarios.

Embed Simple Managed Type in Native Type

Since the CLR needs to keep track of every single instance of a managed type in a process, storing some kind of reference/pointer/handle to a managed object in a native type is not directly supported since instances of native types can be allocated in any region of memory and cast to all kinds of foreign data types that would be completely opaque to the CLR and its services. Instead you need to register such occurrences with the CLR so that it is aware of these “native” references to managed types. This is achieved with the use of the GCHandle type. Internally GCHandle manages a static table of (native) pointers that are used to lookup the objects in the managed heap. Of course using GCHandle directly from C++ can be quite tedious. It’s a CLS compliant value type which means native pointers are represented by IntPtr values. It also does not preserve static type information so static_casts are inevitable. Fortunately Visual C++ ships with the gcroot native template class that provides a strongly-typed interface over the GCHandle type.

#include <vcclr.h>
 
ref struct ManagedType
{
    void HelloDotNet()
    {
        Console::WriteLine("Hello .NET");
    }
};
 
struct NativeType
{
    ManagedType m1;          // Error!
   
    ManagedType^ m2;         // Error!
   
    gcroot<ManagedType^> m3; // OK
};
 
void main()
{
    NativeType native;
    native.m3 = gcnew ManagedType;
 
    native.m3->HelloDotNet();
}

As you can see, gcroot provides a “smart” pointer for storing handles in native types. It may be smart but it does not provide automatic cleanup of resources. Specifically, the gcroot destructor makes no attempt to dispose of the managed object’s resources.

Embed Managed Resource in Native Type

Enter the auto_gcroot class. This native template class wraps a gcroot and provides transfer-of-ownership semantics for managed objects stored in native types. If you’re looking for a point of reference, think of the auto_ptr template class from the Standard C++ Library which does the same thing for native pointers. The auto_gcroot destructor takes care of “deleting” the handle which results in the object’s IDisposable::Dispose method (if any) being called.

#include <msclr\auto_gcroot.h>
 
ref struct ManagedType
{
    void HelloDotNet()
    {
        Console::WriteLine("Hello .NET");
    }
 
    ~ManagedType()
    {
        Console::WriteLine("dtor");

        // Compiler implements Dispose pattern...
    }
};
 
struct NativeType
{
    msclr::auto_gcroot<ManagedType^> m3; // OK
};
 
void main()
{
    NativeType native;
    native.m3 = gcnew ManagedType;
 
    native.m3->HelloDotNet();
}

The NativeType destructor (provided by the compiler) will automatically call the auto_gcroot destructor which will delete the managed object resulting in its destructor being called through its compiler generated Dispose method.

Embed Native Type in Managed Type

Now let’s turn things around. Let’s say we want to store a native type as a member of a managed type. The challenge is that the only native type the CLR really supports within managed types is a native pointer. C# programmers use IntPtr but that is only because IntPtr is the CLS compliant way of representing a native pointer and C# tries really hard to remain CLS compliant. The CLR fully supports storing native pointers without losing type information.

struct NativeType
{
};
 
ref struct ManagedType
{
    NativeType n1; // Error!

    NativeType* n2; // OK
};

That’s great except that now we have a resource management issue. Recall that C++ does not have the separation of memory and resource management evident in the CLR. The native object pointed to by the ManagedType member needs to be deleted. Here is one solution.

ref struct ManagedType
{
    NativeType* n2; // OK
 
    ~ManagedType()
    {
        if (0 != n2)
        {
            delete n2;
            n2 = 0;
        }
    }
};

Now the ManagedType has a Dispose implementation that will faithfully delete the native object. But this can become tedious and error prone very quickly. A better solution is to use some kind of “automatic” approach. Fortunately C++/CLI support by-value semantics for members so all we need is a managed auto-pointer template class. With such a class the ManagedType becomes really simple.

ref struct ManagedType
{
    AutoPtr<NativeType> n2; // OK
};

ManagedType stores a pointer to a native object and its destructor automatically deletes the object. Woohoo!

The C++ compiler really takes care of a lot of boilerplate code. If you’re not sure just how much code the compiler is taking care of for you then take a look at the compiled assembly in a disassembler.

Although Visual C++ does not provide a managed AutoPtr class, it is reasonably simple to write one. Here is a basic implementation.

template <typename T>
ref struct AutoPtr
{
    AutoPtr() : m_ptr(0)
    {
        // Do nothing
    }
    AutoPtr(T* ptr) : m_ptr(ptr)
    {
        // Do nothing
    }
    AutoPtr(AutoPtr<T>% right) : m_ptr(right.Release())
    {
        // Do nothing
    }
    ~AutoPtr()
    {
        if (0 != m_ptr)
        {
            delete m_ptr;
            m_ptr = 0;
        }
    }
    T& operator*()
    {
        return *m_ptr;
    }
    T* operator->()
    {
        return m_ptr;
    }
    T* Get()
    {
        return m_ptr;
    }
    T* Release()
    {
        T* released = m_ptr;
        m_ptr = 0;
        return released;
    }
    void Reset()
    {
        Reset(0);
    }
    void Reset(T* ptr)
    {
        if (0 != m_ptr)
        {
            delete m_ptr;
        }
        m_ptr = ptr;
    }
private:
    T* m_ptr;
};

In a future post I may provide a few realistic examples of mixing native and managed code, but I hope this introduction has given you a few ideas on how to mix native and managed code and types effectively in C++.


© 2005 Kenny Kerr

 

Published Tuesday, July 12, 2005 9:14 AM by KennyKerr

Comments

# re: Mixing Native and Managed Types in C++

Tuesday, July 12, 2005 9:32 PM by Norman Diamond
Thank you. With regard to the problem I had before, the most important part of the answer seems to be that although a managed class cannot contain a member variable with an unmanaged type, it can contain a pointer to an unmanaged heap variable.

Now I wonder about this:
> AutoPtr(AutoPtr<T>% right) :
> m_ptr(right.Release())
> {
> // Do nothing
> }

If I understand correctly, here's what will happen. If our managed class contains a member AutoPtr variable pointing to unmanaged information that we need, and we pass our AutoPtr as a parameter to a method of another class (and the other class calls the unmanaged Win32 API or whatever that really uses the information), then we lose access to the information from our own AutoPtr. Did I misunderstand? If this is actually what happens, then what was the purpose of defining it this way?

# re: Mixing Native and Managed Types in C++

Wednesday, July 13, 2005 7:49 AM by George Gombos
One thing I'd like to know is why
if (0 != n2)
{
delete n2;
n2 = 0;
}
but not rather
if (nullptr != n2)
{
delete n2;
n2 = nullptr;
}

# re: Mixing Native and Managed Types in C++

Wednesday, July 13, 2005 1:48 PM by Kenny Kerr
nullptr is a constant proposed by Herb Sutter and Bjarne Stroustrup to avoid the ambiguity present in Standard C++ today with the use of 0 as both an integer constant and a null pointer constant. It is not yet part of Standard C++ so you can’t use it in native Visual C++ projects unless you define it yourself. C++/CLI requires nullptr since assigning 0 to some handle types (such as Object^ and int^) will implicitly box the value resulting in a new object. Although you can use nullptr as a null pointer constant in managed code, I prefer to use 0 with pointers and nullptr with handles until such time as it is standardized. I do this because I often still write purely native code where nullptr is not available and it just seems more consistent to me.

At the end of the day it’s up to you. Both nullptr and 0 results in a ldnull IL instruction when used with pointers which is the same instruction used when assigning nullptr to handles.

# re: Mixing Native and Managed Types in C++

Wednesday, July 13, 2005 2:07 PM by Kenny Kerr
The challenge with implementing any smart pointer is figuring out how to handle the ownership issue and specifically how to implement the copy constructor and assignment operator.

A reference counting smart pointer (such as ATL::CComPtr or boost::shared_ptr) solves the ownership issue by sharing the ownership among all the clients. When the last client deletes their smart pointer the shared object is destroyed (although strictly speaking CComPtr delegates to the object itself to manage the reference count).

std::auto_ptr is designed simply to ensure that the object to which it points gets destroyed automatically when the smart pointer goes out of scope. Since there is no reference counting, the auto_ptr “owns” the pointer and implementing as well as using copy construction and assignment needs to be done with care. Instead of simply forbidding copy and assignment (which cannot be done reliably anyway) the auto_ptr implements transfer-of-ownership semantics. When an auto_ptr is copy constructed or assigned another auto_ptr, the right-hand-side releases its ownership of the pointer and ownership is given to the left-hand-side auto_ptr.

My AutoPtr ref class simply employs the auto_ptr smart pointer approach to ownership. Anyway, this topic deserves a blog post of its own...

# re: Mixing Native and Managed Types in C++

Thursday, July 21, 2005 5:40 AM by Nish
Hello Kenny,

Nice entry :-)

One suggestion though. You should add a finalizer to AutoPtr - just for safety.

{
AutoPtr<Native>^ pN = gcnew AutoPtr<Native>();
}

pN goes out of scope and is eventually GC'd but there's no finalizer and so m_ptr leaks.

Regards,
Nish

# re: Mixing Native and Managed Types in C++

Thursday, July 21, 2005 5:41 AM by Nish
Of course, people shouldn't be using AutoPtr that way - I admit that, but this is just a safety precaution :-)

# re: Mixing Native and Managed Types in C++

Wednesday, August 3, 2005 6:16 AM by Andy Sawyer
On the subject of null pointers, I'm curious as to why you write:

if (0 != n2)
{
delete n2;
n2 = 0;
}

rather than just:

delete n2;
n2 = 0;

It's safe to pass a null pointer to delete (the primary motivation being that it obviates the need for the very common pattern above...)

# re: Mixing Native and Managed Types in C++

Wednesday, August 3, 2005 10:07 AM by Kenny Kerr
This pattern is just a habit I have picked up since many of the smart pointers I write (and I tend to write quite a few) tend to have non-trivial destructors or at least destructors that involve custom memory allocators (thus the need for a smart pointer to provide consistent and exception safe behavior). Many other OS and custom allocators aren’t as forgiving as the delete operator.

# re: Mixing Native and Managed Types in C++

Wednesday, August 9, 2006 1:12 PM by Jeremy Went

So the template class constructs a pointer but doesn't allocate space on the native heap for the class (or whatever) the pointer is pointing to? Does AutoPtr<> need an assignment operator such as:

T* operator=(T* rhs) { return (m_ptr = rhs); }

so we can write:

SomeManagedClass->NativePointer = new NativeType

Jeremy

# re: Mixing Native and Managed Types in C++

Wednesday, August 9, 2006 1:39 PM by Jeremy Went

Kenny I know you wrote this a year ago but I am a bit behind the times - forgive me. One other problem - when I close my windows forms program the Finalize calls dispose which calls the !operator which asserts m_ptr = 0 which it isn't because ~AutoPtr is not being called ... why would ~AutoPtr() not be called from C# to the C++/CLI code?

Jeremy

# re: Mixing Native and Managed Types in C++

Wednesday, August 9, 2006 1:59 PM by KennyKerr

Jeremy,

Regarding assignment: since the smart pointer's purpose for being is to safely manage native resources, assignment is managed explicitly using the Reset method. So your assignment example would be written as follows:

SomeManagedClass->NativePointer.Reset(new NativeType);

Regarding dispose: The destructor (Dispose in C#) needs to be called explicitly. I provide an updated version of the AutoPtr class in a later article which calls the destructor from the finalizer as a last resort. I would encourage you to dispose object explicitly but also to use the updated version of the AutoPtr class that you can find here:

http://msdn.com/library/en-us/dnvs05/html/CplusCLIBP.asp

# re: Mixing Native and Managed Types in C++

Thursday, August 24, 2006 3:52 PM by Jeremy Went

Thank you and keep up the good work.

# re: Mixing Native and Managed Types in C++

Friday, September 15, 2006 3:06 PM by Aldo

This question is a little on the C# side:

I'm calling a P/Invoking a Windows function (DMProcessConfigXML) that automatically allocates a string in the heap, and expects the caller to deallocate it with the "delete" operator in C++.

However, I'm working in C#. How do I delete the string in C#?

Thanks!

# re: Mixing Native and Managed Types in C++

Friday, September 15, 2006 3:46 PM by KennyKerr

Aldo: Questions like this are best posted on the MSDN forums (http://forums.microsoft.com/MSDN/).

Since DMProcessConfigXML is part of a C++ library and uses the CRT for allocating and releasing memory, you cannot call it directly from C#. You will need to wrap the call in C++, in a DLL for example, (or directly in C++/CLI) so that you can copy it to a managed buffer before releasing the memory using the delete operator. The delete operator cannot be called directly from C# for a number of reasons not the least of which is that the CRT has no compatibility guarantees across versions. C++/CLI is a great solution to this problem as you can mix native CRT and managed code directly.

# re: Mixing Native and Managed Types in C++

Friday, September 15, 2006 5:53 PM by KennyKerr

Aldo: this may work coincidentally but it may fail in the future without warning (and possibly without you knowing) and corrupt the CLR if the version of the CRT used internally by the CLR (since the CLR is written in C++ using the CRT) differs from the version of the CRT used by the Device Configuration API. If you can guarantee that they will always be the same version then it should be safe.

# re: Mixing Native and Managed Types in C++

Friday, September 15, 2006 6:45 PM by KennyKerr

Aldo: right, I was thinking of the desktop CLR. As I said, as long as they use the same version of the CRT then it is quite safe and it sounds like that is the case here. You can check the version of the CRT that it binds to using the Dependency Walker tool.

# re: Mixing Native and Managed Types in C++

Thursday, November 2, 2006 5:38 PM by Steven H

Can you post a complete examples. I'm having problems I think I'm having a logic error.

What I want is a native app to help make it hard on dissamsblers. and it gives me

'/MT' and '/clr' command-line options are incompatible and all the options under the threading fail has well.

# re: Mixing Native and Managed Types in C++

Friday, November 10, 2006 12:45 PM by KennyKerr

DaveDowd: There’s the problem. AutoPtr is designed to provide deterministic cleanup of native resources in managed code. Your C# code should be explicitly freeing the managed wrapper by calling its Dispose method rather than waiting for the GC to clean it up. That’s what the assertion is for – to ensure that you’re calling Dispose from C#. If you really don’t care when the memory is freed then you can remove the assertion in the finalizer (!AutoPtr) and the native resource will eventually be freed when the GC gets to it. It is however best to free resources deterministically and you do so in C# by calling the object’s Dispose method or by using a “using” block. If you dispose of the object then the finalizer will never be called.