Delphi 10.4 introduced Custom Managed Records. In this post we show how you can use this new language feature to wrap third party C(++) APIs in an easy to use model. We also compare this method to some other ways you may have interfaced with C(++) in the past.

Ways to interface with C(++)

For this discussion we assume that the C(++) API you want to wrap follows some sort of create-use-destroy “object” model. Here, the term “object” can mean a C++ object or any form of structure as used in C. Many external APIs use this model. For example, the Windows API has functions like CreateWindow, ShowWindow and DestroyWindow. From here on, we will use the term C API even if the library is written in C++ (since we ultimately need to interface with flat C functions).

1. Using Classes

There are several ways you can wrap C APIs. Traditionally, these are wrapped in Delphi classes. The VCL is a prime example of this; most VCL classes are wrappers around handles in the Windows API.

A common design pattern here is to create the handle in the constructor of the wrapper and destroy it in the destructor. Other methods and properties can then use the handle:

ype
  TFoo = class
  private
    FHandle: THandle;
  public
    constructor Create;
    destructor Destroy;
    function Calc(const ALeft, ARight: Integer): Integer;
  end;

constructor TFoo.Create;
begin
  inherited;
  FHandle := foo_create();
end;

destructor TFoo.Destroy;
begin
  foo_destroy(FHandle);
  inherited;
end;

function TFoo.Calc(const ALeft, ARight: Integer): Integer;
begin
  Result := foo_calc(FHandle, ALeft, ARight);
end;

Here, the routines that start with the “foo_” prefix are implemented in a third part library.

There can be some challenges with this method though. Suppose the C library keeps a list of Bar “objects” for each Foo object. We would wrap these using a TBar class in Delphi. An implementation could look like this:

type
  TFoo = class
  public
    ...
    function GetItem(const AIndex: Integer): TBar;
  end;

function TFoo.GetItem(const AIndex: Integer): TBar;
begin
  var BarHandle := foo_get_item(FHandle, AIndex);
  Result := TBar.Create(BarHandle); // ??
end;

Here the highlighted line is problematic. We retrieve a handle to a Bar object from the C API and we need to wrap this in a TBar class somehow. We could create a TBar instance on-the-fly as in this example. However, this instance needs to be destroyed at some point. In this example, the user of your wrapper class is responsible for this. This puts the burden of lifetime management on the user. And since it is not obvious that a method called GetItem creates a new instance, the user will probably not destroy it, leading to memory leaks.

This is a problem with all C APIs that return references to existing objects. There are several solutions for this issue. For example, the TFoo class could keep a list of TBar objects and keep these in sync with the C API. Then, the GetItem method would return an object in this list instead of creating a new one. And the TFoo class would make sure the objects get destroyed at the appropriate time. However, depending on the API it is not always easy or possible to keep this list in sync.

Another solution you may see is to use dictionaries that map C handles to their Delphi wrappers. In that case, the GetItem method would check if the dictionary already contains a Bar handle. If so, it would return its corresponding Delphi wrapper. Otherwise, it would create it and add it to the dictionary. However, keeping the dictionary up-to-date can also be problematic sometimes.

In addition, when using references to existing C objects, the Delphi wrapper does not own the handle, and should not destroy it in its destructor. This situation is often handled by using a flag called FOwnsHandle or something similar:

type
  TBar = class
  private
    FHandle: THandle;
    FOwnsHandle: Boolean;
  private
    constructor Create(const AHandle: THandle;
      const AOwnsHandle: Boolean);
  public
    destructor Destroy; override;
  end;

constructor TBar.Create(const AHandle: THandle;
  const AOwnsHandle: Boolean);
begin
  inherited Create;
  FHandle := AHandle;
  FOwnsHandle := AOwnsHandle;
end;

destructor TBar.Destroy;
begin
  if (FOwnsHandle) then
    bar_destroy(FHandle);
  inherited;
end;

Here, the constructor is made private since it should only be used inside the unit that declares the TBar type.

2. Using Object Interfaces

Many of these issues can be avoided by using object interfaces instead of classes. This also simplifies lifetime management since you (and more importantly the users of your wrapper) don’t need to worry about destroying objects at the appropriate time.

The sample class model above could be converted to using object interfaces like this:

type
  IBar = interface
    { Various declarations... }
  end;

type
  IFoo = interface
    function GetItem(const AIndex: Integer): IBar;
  end;

type
  TBar = class(TInterfacedObject, IBar)
    { Similar to TBar shown earlier }
  end;

type
  TFoo = class(TInterfacedObject, IFoo)
  private
    FHandle: THandle;
  protected
    { IFoo }
    function GetItem(const AIndex: Integer): IBar;
  public
    constructor Create;
    destructor Destroy; override;
  end;

constructor TFoo.Create;
begin
  inherited;
  FHandle := foo_create;
end;

destructor TFoo.Destroy;
begin
  foo_destroy(FHandle);
  inherited;
end;

function TFoo.GetItem(const AIndex: Integer): IBar;
begin
  var BarHandle := foo_get_item(FHandle, AIndex);
  Result := TBar.Create(BarHandle, False);
end;

Here, the TFoo.GetItem method returns a newly created IBar interface (that doesn’t own the handle). Since this is an interface, it will be destroyed automatically when the user is done with it. No need for the TFoo class to maintain a list or dictionary of Bar instances (although it may still do so to avoid frequent allocations of TBar objects if that is an issue).

There are some disadvantages too though. Many wrapper methods are very small and only call the corresponding C API. However, since these methods are implemented in interfaces, you cannot inline them. Furthermore, since an interface is basically a virtual method table, all methods in an interface are virtual methods, resulting in an extra level of indirection when calling the method. This is not very cache-friendly since it requires accessing various (probably distant) locations in memory before making the call. When the wrapper function is called a lot, this can have an impact on performance.

Furthermore, as with classes, you are basically wrapping a (C) object inside another (Delphi) object, resulting in an additional dynamic memory allocation. If you create a lot of (temporary) objects this way, this can lead to additional memory fragmentation and performance loss.

Finally, using interfaces requires more code, since you have to write both an interface declaration, and then duplicate that interface declaration in a class. You also have to keep the interface and class declarations in sync. However, I consider this a minor disadvantage compared to all the benefits that interfaces have to offer.

3. Using Records

Sometimes, the C API lends itself for wrapping inside a simple Delphi record. That way, you can avoid dynamic memory allocations and will be able to inline wrapper methods. You cannot use automatic resource management in this case though, so it is usually up to the users of your wrapper to manage object lifetimes.

There may be situations where this can be avoided if every record is owned by some sort of “master” class or record. For example, you could wrap a 3rd party XML DOM library, where each XML node or element is a record that is owned by a master document, and the master document takes care of managing the lifetimes of these records.

For many APIs though, this is not a feasible solution.

4. Using Custom Managed Records

Which brings us to the main purpose of this article. In the remainder of this post I will show you how you can use Custom Managed Records to wrap C APIs. This allows you to avoid many of the disadvantages mentioned above (although it has some disadvantages of its own, which will be discussed at the end of the article).

The reason for this post is that I was looking for a (better) way to wrap the Skia Graphics API. We use this API in our Lumicademy product to provide a consistent way to present (vector) graphics on all platforms. We used to have an interface-based wrapper for this library, which works pretty well. However, it has some of the disadvantages mentioned above such as frequent allocations of small objects and the inability to inline method calls. Since the Skia API underwent some major changes recently, I thought this was a good time to reevaluate our model and see if we could do better.

Custom Managed Records 101

Lets start with a quick introduction of Custom Managed Records for those who aren’t familiar with this new language feature yet.

Managed Records (not Custom ones) are nothing new in Delphi. They have existed since version 1, although you probably haven’t called them this. A Managed Record is just a regular record, but with one or more fields of a managed type. A managed type is a type whose lifetime is managed by the Delphi compiler and runtime (usually through reference counting), and includes strings, dynamic arrays, object interfaces and other managed records.

Before Delphi 10.4, a regular object was also a managed type when compiled on ARC (mobile) platforms. That’s no longer the case though now that we have a uniform memory management model on all platforms (yay!).

When you declare a managed record, Delphi adds code behind the scenes to manage its lifetime, and to handle assigning one record to another. For example, consider this code:

type
  TSomeRecord = record
    S: String; // A managed field
  end;

procedure Something;
var
  A, B: TSomeRecord;
begin
  A.S := 'Foo';
  B := A;
end;

The Delphi compiler actually converts this to the following code (this is not entirely accurate, but suffices for demonstration purposes):

procedure Something;
var
  A, B: TSomeRecord;
begin
  InitializeRecord(A);
  try
    InitializeRecord(B);
    try
      A.S := 'Foo';
      CopyRecord(B, A);
    finally
      FinalizeRecord(B);
    end;
  finally
    FinalizeRecord(A);
  end;
end;

Here, InitializeRecord initializes all managed fields in a record (that is, it clears strings, dynamic arrays, interfaces etc.). Likewise, CopyRecord copies managed fields from one record to another. It does so by updating the reference counts of all managed fields. Finally, FinalizeRecord decreases these reference counts again, which can result in freeing the managed field if its reference count reaches 0.

Note that these three routines use RTTI to enumerate all managed fields in a record, and as such may incur a small performance penalty. Although this is usually negligible, it is something you should be aware of, especially if a record has many managed fields.

A Custom Managed Record is very similar to a regular Managed Record, but instead of the compiler inserting calls to InitializeRecord, CopyRecord and FinalizeRecord, it calls the newly introduced Initialize, Assign and Finalize operators of the record instead. You have to write these operators yourself, hence the “Custom” in Custom Managed Records. The signatures of these methods look like this:

type
  TFoo = record
  public
    class operator Initialize(out ADest: TFoo);
    class operator Finalize(var ADest: TFoo);
    class operator Assign(var ADest: TFoo;
      const [ref] ASrc: TFoo);
  end;

Since the compiler makes sure these operators are called at the appropriate times, this makes Custom Managed Records ideal for light-weight resource lifetime management (and RAII). One obvious use case is for implementing smart pointers to support automatic memory management for regular objects. I am sure there are Delphi developers out there experimenting with this very feature right now…

But it is also very useful for managing the lifetime of wrapped C(++) objects.

Wrapping C(++) Objects in Custom Managed Records

So we finally arrived at the point of this article: how to use this new language feature to wrap C(++) objects. I will use the Skia library as a real-world example of how to do this. Skia exposes two kinds of objects through their C API: reference counted objects and non-reference counted objects. These require two different approaches.

1. Wrapping Reference Counted C(++) Objects

If you are lucky, the C API you are wrapping has support for reference counting of its objects. (This is becoming more common for C APIs nowadays). We can then take advantage of this for our Custom Managed Record implementation. An example is the SKImage class in Skia, which we wrap like this:

type
  TSKImage = record
  private
    FHandle: THandle;
    function GetWidth: Integer; inline;
    ...
  private
    constructor Create(const AHandle: THandle;
      const AOwnsHandle: Boolean); overload;
  public
    class operator Initialize(out ADest: TSKImage);
    class operator Finalize(var ADest: TSKImage);
    class operator Assign(var ADest: TSKImage;
      const [ref] ASrc: TSKImage);
  public
    class function Create(...): TSKImage; overload; static;
    ...
    property Width: Integer read GetWidth;
    ...
  end;

class function TSKImage.Create(...): TSKImage;
begin
  var Handle := sk_image_new_raster(...);
  Result := TSKImage.Create(Handle, True);
end;

constructor TSKImage.Create(const AHandle: THandle;
  const AOwnsHandle: Boolean);
begin
  FHandle := AHandle;
  if (AHandle <> 0) and (not AOwnsHandle) then
    sk_refcnt_safe_ref(AHandle);
end;

class operator TSKImage.Initialize(out ADest: TSKImage);
begin
  ADest.FHandle := 0;
end;

class operator TSKImage.Finalize(var ADest: TSKImage);
begin
  if (ADest.FHandle <> 0) then
    sk_refcnt_safe_unref(ADest.FHandle);
end;

class operator TSKImage.Assign(var ADest: TSKImage;
  const [ref] ASrc: TSKImage);
begin
  if (ASrc.FHandle <> ADest.FHandle) then
  begin
    if (ADest.FHandle <> 0) then
      sk_refcnt_safe_unref(ADest.FHandle);

    ADest.FHandle := ASrc.FHandle;

    if (ADest.FHandle <> 0) then
      sk_refcnt_safe_ref(ADest.FHandle);
  end;
end;

function TSKImage.GetWidth: Integer;
begin
  Assert(FHandle <> 0);
  Result := sk_image_get_width(FHandle);
end;

This is boilerplate code for most wrappers that use library-provided reference counting. It works like this:

• The Create class function creates a new handle to a Skia image object and passes that handle to the constructor (setting AOwnsHandle to True to indicate that the wrapper will own the handle).
• The constructor stores the handle and uses a little track to avoid having to store an FOwnsHandle field: when the AOwnsHandle parameter equals True, the constructor does nothing else. In that case, when the wrapper goes out of scope, the Finalize operator will get called, which decreases the reference count (which may result in the destruction of the underlying C++ object). When the AOwnsHandle parameter is False however, we don’t want to decrease the reference count in Finalize. We could store this flag in a FOwnsHandle field and check this value in Finalize. However, a solution that doesn’t require storing this flag is to just increase the reference count in the constructor. Then when the reference count is decreased again in Finalize, the net result will be as if the reference count hasn’t changed, and the underlying C++ object will not be affected.
• The Initialize operator is very simple: it just clears the handle.
• The Finalize operator isn’t that more complicated: it just calls the C API to decrease the reference count (after checking the validity of the handle). Note that this may result in the destruction of the underlying C++ object if the reference count reaches 0.
• The Assign operator is a bit more involved since we need to do a couple of things here. First, we don’t need to do anything if the two records are the same (that is, wrap the same handle). Next, you need to keep in mind that the record you are assigning to may already wrap another C++ object. In that case, we need to decrease its reference count since it will wrap a different object now. Then we can assign the handle and increase it reference count accordingly.
• Finally, most other methods can be pretty simple and usually just call the underlying C API, as the GetWidth method in this example shows. Depending on how the C API is written, you may have to check if the handle is valid before calling the API. In this example, I use an assertion, but you may also raise an exception or use some other kind of error handling.

This is a bit of boilerplate code that you have to repeat for your wrappers (although in some situations, you can reduce the amount of code with generics). But the result is a very light-weight wrapper that doesn’t use any dynamic memory at all (at least on the Delphi side) and allows for fast inlined methods (like GetWidth in the example).

And importantly, the users of your wrapper don’t have to worry about destroying resources. This is managed automatically by the compiler and the runtime. From a users perspective, a Custom Managed Record can be used like a regular class, without having to worry about lifetime management.

2. Non-Reference Counted C(++) Objects

If the C API you are wrapping does not support reference counting, then you have to implement this yourself. Your initial instinct may be to add an integer reference count field to your record, and then manipulate this in your Initialize, Assign and Finalize operators. However, this will not work if you allow assigning one record to another. In that case, both copies will end up with their own reference counts. And the reference count of one record may reach 0 (and destroy the underlying C++ object) while the other record still has a reference to it.

So when assigning one record to another, you must make sure there is only 1 instance of the reference count field. The solution I used for the Skia wrapper is to store the handle, reference count and some other information inside a dynamically allocated helper record. Then, the Initialize, Assign and Finalize operators of the wrapper use this helper record to manage the lifetime of the wrapped handle.

type
  TSKSharedHandleDeleter = procedure(AHandle: THandle); cdecl;

  PSKSharedHandle = ^TSKSharedHandle;
  TSKSharedHandle = record
  private
    FHandle: THandle;
    FDeleter: TSKSharedHandleDeleter;
    [volatile] FRefCount: Integer;
    FOwnsHandle: Boolean;
  public
    class function Create(const AHandle: THandle; 
       const ADeleter: TSKSharedHandleDeleter;
       const AOwnsHandle: Boolean): PSKSharedHandle; static;
    procedure AddRef; inline;
    procedure Release; inline;
  end;

class function TSKSharedHandle.Create(const AHandle: THandle;
  const ADeleter: TSKSharedHandleDeleter;
  const AOwnsHandle: Boolean): PSKSharedHandle;
begin
  Assert(AHandle <> 0);
  Assert(Assigned(ADeleter));
  GetMem(Result, SizeOf(TSKSharedHandle));
  Result.FHandle := AHandle;
  Result.FDeleter := ADeleter;
  Result.FRefCount := 1;
  Result.FOwnsHandle := AOwnsHandle;
end;

procedure TSKSharedHandle.AddRef;
begin
  if (@Self <> nil) then
    AtomicIncrement(FRefCount);
end;

procedure TSKSharedHandle.Release;
begin
  if (@Self <> nil) and (AtomicDecrement(FRefCount) = 0) then
  begin
    if (FOwnsHandle) then
      FDeleter(FHandle);
    FreeMem(@Self);
  end;
end;

This requires some explanation:

• The record has 4 fields:
  • The wrapped handle.
  • The C API that is used to free the handle when the reference count reaches 0. In the Skia library, all object destruction APIs have the same signature: a cdecl procedure with a single parameter containing the handle of the object to be destroyed.
  • The reference count that we use for lifetime management.
  • A flag indicating whether we own the handle or not.
• The Create class function allocates an instance of this record on the heap. It initializes all fields and sets the reference count to 1.
• There is an AddRef method that is called when we need to retain a reference. It just increments the reference count in a thread-safe manner. The “if (@Self <> nil) then” check is added to allow you to call this method on a nil-pointer. This simplifies some code later on. (The @ operator is needed because we are dealing with a record, not an object.)
• Likewise, the Release method is used to release a reference. It decreases the reference count. When the reference count reaches 0, it calls the C API to destroy the handle, and free the memory that was allocated by the Create function.

Now we can use this helper in our Skia wrappers. An example where we use this is in the SKCanvas class. The Skia API does not provide built-in reference counting for this class, so we do it ourselves with the PSKSharedHandle helper:

type
  TSKCanvas = record
  private
    FShared: PSKSharedHandle;
  private
    constructor Create(const AHandle: THandle;
      const AOwnsHandle: Boolean); overload;
  public
    class operator Initialize(out ADest: TSKCanvas);
    class operator Finalize(var ADest: TSKCanvas);
    class operator Assign(var ADest: TSKCanvas;
      const [ref] ASrc: TSKCanvas);
  public
    class function Create(...): TSKCanvas; overload; static;

    procedure Clear; inline;
    ...
  end;

class function TSKCanvas.Create(...): TSKCanvas;
begin
  var Handle := sk_canvas_new_from_bitmap(...);
  Result := TSKCanvas.Create(Handle, True);
end;

constructor TSKCanvas.Create(const AHandle: THandle;
  const AOwnsHandle: Boolean);
begin
  if (AHandle = 0) then
    FShared := nil
  else
    FShared := TSKSharedHandle.Create(AHandle,
      sk_canvas_destroy, AOwnsHandle);
end;

class operator TSKCanvas.Initialize(out ADest: TSKCanvas);
begin
  ADest.FShared := nil;
end;

class operator TSKCanvas.Finalize(var ADest: TSKCanvas);
begin
  ADest.FShared.Release;
end;

class operator TSKCanvas.Assign(var ADest: TSKCanvas;
  const [ref] ASrc: TSKCanvas);
begin
  if (ADest.FShared <> ASrc.FShared) then
  begin
    ADest.FShared.Release;
    ADest.FShared := ASrc.FShared;
    ADest.FShared.AddRef;
  end;
end;

procedure TSKCanvas.Clear;
begin
  Assert(FShared <> nil);
  sk_canvas_clear(FShared.Handle);
end;

This is again mostly boilerplate:

• Instead of a FHandle field, this record has a FShared field of type PSKSharedHandle. This makes sure that all copies of the record will have a FShared field that points to the same underlying handle and reference count.
• The constructor allocates the shared handle. It passes the sk_canvas_destroy C API to the shared handle. This API will get called to destroy the canvas when the all records that share this handle have gone out of scope.
• All other methods are pretty straight forward. The only thing to keep in mind again is that the Assign operator may be assigning to a record that already wraps another C++ object. So we need to perform a Release on the old wrapper and an AddRef on the new one (remember that it is safe to call these methods on a nil-wrapper).

This way of wrapping a C++ object is still pretty light-weight, although it requires some dynamic memory allocation for the PSKSharedHandle. You could improve this by creating a PSKSharedHandle pool to avoid repeated allocations of small amounts of memory.

Some C APIs have the capability to store arbitrary user data (or a tag) with an object. You may be able to use this to store the reference count so you don’t need a dynamically allocated shared handle.

Things To Keep In Mind

All wrapping methods discussed here have some caveats. For example, how do you compare two wrappers. If you would wrap the C++ objects in classes or interfaces, then you couldn’t just write “if (Image1 = Image2) then...“. This would compare the wrapped Delphi objects to each other and not their wrapped handles. You could add an Equals method to compare their underlying handles instead (or override TObject.Equals if you are using classes for wrapping). However, in this case the burden is on the user of your wrapper to call Equals instead of using an equality operator.

With Custom Managed Records you can overload the equality (and inequality) operator however, resulting in a very natural way for your users to compare two C++ objects:

type
  TSKImage = record
  private
    FHandle: THandle;
  public
    class operator Equal(const ALeft,
      ARight: TSKImage): Boolean; inline; static;
    class operator NotEqual(const ALeft,
      ARight: TSKImage): Boolean; inline; static;
  end;

class operator TSKImage.Equal(const ALeft,
  ARight: TSKImage): Boolean;
begin
  Result := (ALeft.FHandle = ARight.FHandle);
end;

class operator TSKImage.NotEqual(const ALeft,
  ARight: TSKImage): Boolean;
begin
  Result := (ALeft.FHandle <> ARight.FHandle);
end;

Disadvantages

Using Custom Managed Records for wrapping has some disadvantages as well compared to other methods.

1. Compare Against nil

An advantage of using classes or object interfaces is that these can be nil to indicate the absence of an instance. However, records cannot be nil and cannot be compared against nil. So you cannot write something like “if (Image <> nil) then...“. You could solve this by adding a method like IsNil which returns True when the underlying handle is 0. In the Skia wrapper, we use overloaded (in)equality operators again so you compare against nil in a more natural way:

class operator TSKImage.Equal(const ALeft: TSKImage;
  const ARight: Pointer): Boolean;
begin
  Result := (Pointer(ALeft.FHandle) = ARight);
end;

class operator TSKImage.NotEqual(const ALeft: TSKImage;
  const ARight: Pointer): Boolean;
begin
  Result := (Pointer(ALeft.FHandle) <> ARight);
end;

Here, the right-hand side of the operation is a pointer, which allows you to compare against nil.

2. Default Parameters

However, this will not work if you allow nil for default parameters, since you cannot have default parameters for record types:

procedure DrawButton(const ACaption: String;
  const AGlyph: TSKImage = nil); // Does NOT compile

The usual solution for this is to create an overloaded version:

procedure DrawButton(const ACaption: String); overload;
procedure DrawButton(const ACaption: String;
  const AGlyph: TSKImage); overload;

3. Forward Declarations

Sometimes you have two C++ class types that reference each other. When wrapping these in Delphi classes or interfaces, you can use forward declarations, as in:

type
  TBar = class;

  TFoo = class
  public
    procedure Go(const ABar: TBar);
  end;

  TBar = class
  public
    procedure Go(const AFoo: TFoo);
  end;

However, you cannot do this with (Custom Managed) Records. A way to work around this is with record helpers:

type
  TFoo = record
  public
    ...
  end;

type
  TBar = record
  public
    procedure Go(const AFoo: TFoo);
  end;

type
  _TFooHelper = record helper for TFoo
  public
    procedure Go(const ABar: TBar);
  end;

This is not pretty, but it works.

Note that I prefer to start record and class helper names with an underscore (_) to dissuade people from using the type directly.

4. Inheritance

Finally – and this is a big one – you cannot use inheritance with records. If your C API uses a (deep) class hierarchy, then using Custom Managed Records to wrap them is probably not for you.

Fortunately, the Skia library uses a very flat class hierarchy, making it ideal for wrapping using Custom Managed Records. There are a couple of classes that are inherited, but these are not common. (For these cases, I created a subclass “record” as a record with a single field of its base record type. This is again not pretty, but fortunately also not common.)

Fortunately, many recent APIs shy away from deep class hierarchies in favor of a more modern “composition-over-inheritance” approach. These are ideal for wrapping using Custom Managed Records.

Give It a Try

So if you ever find yourself needing to wrap a C API, then consider using Custom Managed Records. If the API fits this model, then I think this is a very elegant and light-weight approach to exposing the API in a user-friendly way.