Python: Who owns the memory in ctypes?

Question

I have a pile of C code that I wrote that I want to interface to via the ctypes module ( https://docs.python.org/3/library/ctypes.html ).

The C code uses the Boehm-Demers-Weiser garbage collector ( http://www.hboehm.info/gc/ ) for all of its memory management. What I want to know is, who owns allocated memory? That is, if my C code allocates memory via GC_MALLOC() (the standard call for allocating memory in the garbage collector), and I access some object via ctypes in python, will the python garbage collector assume that it owns it and attempt to dispose of it when it goes out of scope?

Ideally, the memory is owned by the side that created it, with the other side simply referencing it, but I want to be sure before I invest a lot of time interfacing the two sides together.

Answer 1

ctypes objects own only the memory that they allocate. Inspect the _b_needsfree_ attribute to determine whether a ctypes object owns the referenced memory.

For example:

This array object owns a 12-byte buffer for the array elements:

 >>> arr = (ctypes.c_uint * 3)(0, 1, 2)
 >>> arr._b_needsfree_
 1

This pointer object owns an 8-byte buffer for the 64-bit target address:

 >>> p = ctypes.POINTER(ctypes.c_uint * 3)(arr)
 >>> p._b_needsfree_
 1

The following new array object created by dereferencing the pointer does not own the 12-byte buffer:

 >>> ref = p[0]
 >>> ref[:]
 [0, 1, 2]
 >>> ref._b_needsfree_
 0

However, it does have a reference chain back to the original array:

 >>> ref._b_base_ is p
 True
 >>> p._objects['1'] is arr
 True

On the other hand, if you use the from_address() class method, the resulting object is a dangling reference to memory that it doesn't own and for which there's no supporting reference chain to the owning object.

For example:

 >>> arr = (ctypes.c_uint * 2**24)()
 >>> arr[-1] = 42
 >>> ref = type(arr).from_address(ctypes.addressof(arr))
 >>> ref[-1]
 42

 >>> ref._b_base_ is None
 True
 >>> ref._objects is None
 True

2**24 bytes is a big allocation that uses mmap (Unix) instead of the heap. Thus accessing ref[-1] causes a segfault after arr is
deallocated:

 >>> del arr
 >>> ref[-1]
 Segmentation fault (core dumped)