libslack(mem) - memory module
#include <slack/std.h> #include <slack/mem.h> typedef struct Pool Pool; #define null NULL #define nul '\0' #define mem_new(type) #define mem_create(size, type) #define mem_resize(mem, size) void *mem_resize_fn(void **mem, size_t size); #define mem_release(mem) void *mem_destroy(void **mem); void *mem_create_secure(size_t size); void mem_release_secure(void *mem); void *mem_destroy_secure(void **mem); char *mem_strdup(const char *str); #define mem_create2d(type, x, y) #define mem_create3d(type, x, y, z) #define mem_create4d(type, x, y, z, a) void *mem_create_space(size_t size, ...); size_t mem_space_start(size_t size, ...); #define mem_release2d(space) #define mem_release3d(space) #define mem_release4d(space) #define mem_release_space(space) #define mem_destroy2d(space) #define mem_destroy3d(space) #define mem_destroy4d(space) #define mem_destroy_space(space) Pool *pool_create(size_t size); Pool *pool_create_with_locker(Locker *locker, size_t size); void pool_release(Pool *pool); void *pool_destroy(Pool **pool); Pool *pool_create_secure(size_t size); Pool *pool_create_secure_with_locker(Locker *locker, size_t size); void pool_release_secure(Pool *pool); void *pool_destroy_secure(Pool **pool); void pool_clear_secure(Pool *pool); #define pool_new(pool, type) #define pool_newsz(pool, size, type) void *pool_alloc(Pool *pool, size_t size); void pool_clear(Pool *pool);
This module is mostly just an interface to malloc(3), realloc(3) and
free(3) that tries to ensure that pointers that don't point to anything
get set to null
. It also provides dynamically allocated multi-dimensional
arrays, memory pools and secure memory for the more adventurous.
#define null NULL
Easier to type. Easier to read.
#define nul '\0'
A name for the nul
character.
#define mem_new(type)
Allocates enough memory (with malloc(3)) to store an object of type
type
. It is the caller's responsibility to deallocate the allocated
memory with mem_release(3), mem_destroy(3) or free(3). On success,
returns the address of the allocated memory. On error, returns null
.
#define mem_create(size, type)
Allocates enough memory (with malloc(3)) to store size
objects of type
type
. It is the caller's responsibility to deallocate the allocated
memory with mem_release(3), mem_destroy(3) or free(3). On success,
returns the address of the allocated memory. On error, returns null
.
#define mem_resize(mem, num)
Alters the amount of memory pointed to by *mem
. If *mem
is null
,
new memory is allocated and assigned to *mem
. If size is zero, *mem
is
deallocated and null
is assigned to *mem
. Otherwise, *mem
is
reallocated and assigned back to *mem
. On success, returns *mem
(which
will be null
if size
is zero). On error, returns null
with errno
set appropriately and *mem
is not altered.
void *mem_resize_fn(void **mem, size_t size)
An interface to realloc(3) that also assigns to a pointer variable unless
an error occurred. mem
points to the pointer to be affected. size
is
the requested size in bytes. If size
is zero, *mem
is deallocated and
set to null
. This function is exposed as an implementation side effect.
Don't call it directly. Call mem_resize(3) instead. On error, returns
null
with errno
set appropriately.
#define mem_release(mem)
Releases (deallocates) mem
. Same as free(3). Only to be used in
destructor functions. In other cases, use mem_destroy(3) which also sets
mem
to null
.
void *mem_destroy(void **mem)
Calls free(3) on the pointer, *mem
. Then assigns null
to this
pointer. Returns null
.
void *mem_create_secure(size_t size)
Allocates size
bytes of memory (with malloc(3)) and then locks it into
RAM with mlock(2) so that it can't be paged to disk where some nefarious
local user with root access might read its contents. It is the caller's
responsibility to deallocate the secure memory with mem_release_secure(3)
or mem_destroy_secure(3) which will clear the memory and unlock it before
deallocating it. On success, returns the address of the secure allocated
memory. On error, returns null
with errno
set appropriately.
Note that entire pages are locked by mlock(2) so don't create many, small pieces of secure memory or many entire pages will be locked. Use a secure memory pool instead. Also note that secure memory requires root privileges.
On some systems (e.g. Solaris), memory locks must start on page boundaries.
So we need to malloc(3) enough memory to extend from whatever address
malloc(3) may return to the next page boundary (worst case: pagesize -
sizeof(int)
) and then the actual number of bytes requested. We need an
additional 8 bytes to store the address returned by malloc(3) (so we can
free(3) it later) and the size passed to mlock(2) so we can pass it to
munlock(2) later. Unfortunately, we need to store the address and size
after the page boundary and not before it because malloc(3) may return a
page boundary or an address less than 8 bytes to the left of a page
boundary.
It will look like:
for free() +-------+ +- size+8 for munlock() v | v +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | |* * * *|# # # #| | | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ^ ^ ^ . . . size bytes . . . . . . | +- next page | +- malloc() +- address returned
If your system doesn't require page boundaries (e.g. Linux), the address returned by malloc(3) is locked and returned and only the size is stored.
void mem_release_secure(void *mem)
Sets the memory pointed to by mem
to nul
bytes, then unlocks and
releases (deallocates) mem
. Only to be used on memory returned by
mem_create_secure(3). Only to be used in destructor functions. In other
cases, use mem_destroy(3) which also sets mem
to null
.
void *mem_destroy_secure(void **mem)
Sets the memory pointed to by *mem
to nul
bytes, then unlocks and
destroys (deallocates and sets to null
) *mem
. Only to be used on
memory returned by mem_create_secure(3). Returns null
.
char *mem_strdup(const char *str)
Returns a dynamically allocated copy of str
. It is the caller's
responsibility to deallocate the new string with mem_release(3),
mem_destroy(3) or free(3). This function exists because strdup(3)
is not part of the ISO C standard. On error, returns null
with errno
set appropriately.
#define mem_create2d(i, j, type)
Alias for allocating a 2-dimensional array. See mem_create_space(3).
#define mem_create3d(i, j, k, type)
Alias for allocating a 3-dimensional array. See mem_create_space(3).
#define mem_create4d(i, j, k, l, type)
Alias for allocating a 4-dimensional array. See mem_create_space(3).
void *mem_create_space(size_t size, ...)
Allocates a multi-dimensional array of elements of size size
and sets the
memory to nul
bytes. The remaining arguments specify the sizes of each
dimension. The last argument must be zero. There is an arbitrary limit of 32
dimensions. The memory returned is set to zero. The memory returned needs to
be cast or assigned into the appropriate pointer type. You can then set and
access elements exactly like a real multi-dimensional C array. Finally, it
must be deallocated with mem_destroy_space(3) or mem_release_space(3)
or mem_destroy(3) or mem_release(3) or free(3).
Note: You must not use memset(3) on all of the returned memory because the start of this memory contains pointers into the remainder. The exact amount of this overhead depends on the number and size of dimensions. The memory is allocated with calloc(3) to reduce the need to memset(3) the elements but if you need to know where the elements begin, use mem_space_start(3).
The memory returned looks like (e.g.):
char ***a = mem_create3d(2, 2, 3, char); +-------------------------+ +-------|-------------------+ | a +-------|-------|-------------+ | | | +-------|-------|-------|-------+ | | | v | | | | V V V V +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | a[0] | a[1] |a[0][0]|a[0][1]|a[1][0]|a[1][1]| | | | | | | | | | | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | ^ ^ a a a a a a a a a a a a +-------|-------+ | 0 0 0 0 0 0 1 1 1 1 1 1 +-----------------------+ 0 0 0 1 1 1 0 0 0 1 1 1 0 1 2 0 1 2 0 1 2 0 1 2
size_t mem_space_start(size_t size, ...)
Calculates the amount of overhead required for a multi-dimensional array created by a call to mem_create_space(3) with the same arguments. If you need reset all elements in such an array to zero:
int ****space = mem_create_space(sizeof(int), 2, 3, 4, 5, 0); size_t start = mem_space_start(sizeof(int), 2, 3, 4, 5, 0); memset((char *)space + start, '\0', sizeof(int) * 2 * 3 * 4 * 5);
#define mem_release2d(space)
Alias for releasing (deallocating) a 2-dimensional array. See mem_release_space(3).
#define mem_release3d(space)
Alias for releasing (deallocating) a 3-dimensional array. See mem_release_space(3).
#define mem_release4d(space)
Alias for releasing (deallocating) a 4-dimensional array. See mem_release_space(3).
#define mem_release_space(space)
Releases (deallocates) a multi-dimensional array, space
, allocated with
mem_create_space. Same as free(3). Only to be used in destructor
functions. In other cases, use mem_destroy_space(3) which also sets
space
to null
.
#define mem_destroy2d(space)
Alias for destroying (deallocating and setting to null
) a
2-dimensional array. See mem_destroy_space(3).
#define mem_destroy3d(space)
Alias for destroying (deallocating and setting to null
) a
3-dimensional array. See mem_destroy_space(3).
#define mem_destroy4d(space)
Alias for destroying (deallocating and setting to null
) a
4-dimensional array. See mem_destroy_space(3).
#define mem_destroy_space(mem)
Destroys (deallocates and sets to null
) the multi-dimensional array
pointed to by space
.
Pool *pool_create(size_t size)
Creates a memory pool of size size
from which smaller chunks of memory
may be subsequently allocated (with pool_alloc(3)) without resorting to
the use of malloc(3). Useful when you have many small objects to allocate
but malloc(3) is slowing your program down too much. It is the caller's
responsibility to deallocate the new pool with pool_release(3) or
pool_destroy(3). On success, returns the pool. On error, returns null
.
The size of a pool can't be changed after it is created and the individual chunks of memory allocated from within a pool can't be separately deallocated. The entire pool can be emptied with pool_clear(3).
Pool *pool_create_with_locker(Locker *locker, size_t size)
Equivalent to pool_create(3) except that multiple threads accessing the
new pool will be synchronised by locker
.
void pool_release(Pool *pool)
Releases (deallocates) pool
. Only to be used in destructor functions. In
other cases, use pool_destroy(3) which also sets pool
to null
.
void *pool_destroy(Pool **pool)
Destroys (deallocates and sets to null
) *pool
. Returns null
.
Note: pools shared by multiple threads must not be destroyed until after
all threads have finished with it.
Pool *pool_create_secure(size_t size)
Creates a memory pool of size size
just like pool_create(3) except
that the memory pool is locked into RAM with mlock(2) so that it can't be
paged to disk where some nefarious local user might read its contents. It is
the caller's responsibility to deallocate the new pool with
pool_release_secure(3) or pool_destroy_secure(3) which will clear the
memory pool and unlock it before deallocating it. In all other ways, the
pool returned is exactly like a pool returned by pool_create(3). On
success, returns the pool. On error, returns null
with errno
set
appropriately. Note that secure memory requires root privileges.
Pool *pool_create_secure_with_locker(Locker *locker, size_t size)
Equivalent to pool_create_secure(3) except that multiple threads accessing
the new pool will be synchronised by locker
.
void pool_release_secure(Pool *pool)
Sets the contents of the memory pool to nul
bytes, then unlocks and
releases (deallocates) pool
. Only to be used on pools returned by
pool_create_secure(3). Only to be used in destructor functions. In other
cases, use pool_destroy_secure(3) which also sets pool
to null
.
void *pool_destroy_secure(Pool **pool)
Sets the contents of the memory pool to nul
bytes, then unlocks and
destroys (deallocates and sets to null
) *pool
. Returns null
.
Note: secure pools shared by multiple threads must not be destroyed until
after all threads have finished with it.
void pool_clear_secure(Pool *pool)
Fills the secure pool
with nul
bytes and deallocates all of the chunks
of secure memory previously allocated from pool
so that it can be reused.
Does not use free(3).
#define pool_new(pool, type)
Allocates enough memory from pool
to store an object of type type
. On
success, returns the address of the allocated memory. On error, returns
null
with errno
set appropriately.
#define pool_newsz(pool, size, type)
Allocates enough memory from pool
to store size
objects of type
type
. On success, returns the address of the allocated memory. On error,
returns null
with errno
set appropriately.
void *pool_alloc(Pool *pool, size_t size)
Allocates a chunk of memory of size
bytes from pool
. Does not use
malloc(3). The pointer returned must not be passed to free(3) or
realloc(3). Only the entire pool can be deallocated with
pool_release(3) or pool_destroy(3). All of the chunks can be
deallocated in one go with pool_clear(3) without deallocating the pool
itself.
On success, returns the pointer to the allocated pool memory. On error,
returns null
with errno
set appropriately (i.e. EINVAL
if pool
is null
, ENOSPC
if pool
does not have enough unused memory to
allocate size
bytes).
It is the caller's responsibility to ensure the correct alignment if necessary by allocating the right numbers of bytes. The easiest way to do ensure is to use separate pools for each specific data type that requires specific alignment.
void pool_clear(Pool *pool)
Deallocates all of the chunks of memory previously allocated from pool
so
that it can be reused. Does not use free(3).
On error, errno
is set by underlying functions or as follows:
EINVAL
When arguments are invalid.
ENOSPC
When there is insufficient available space in a pool for pool_alloc(3) to satisfy a request.
ENOSYS
Returned by mem_create_secure(3) and pool_create_secure(3) when mlock(2) is not supported (e.g. Mac OS X).
MT-Safe (mem)
MT-Disciplined (pool) man locker(3) for details.
1D array of longs:
long *mem = mem_create(100, long); mem_resize(&mem, 200); mem_destroy(&mem);
3D array of ints:
int ***space = mem_create3d(10, 20, 30, int); int i, j, k; for (i = 0; i < 10; ++i) for (j = 0; j < 20; ++j) for (k = 0; k < 30; ++k) space[i][j][k] = i + j + j; mem_destroy3d(&space);
A pool of a million integers:
void pool() { Pool *pool; int i, *p; if (!(pool = pool_create(1024 * 1024 * sizeof(int)))) return; for (i = 0; i < 1024 * 1024; ++i) { if (!(p = pool_new(pool, int))) break; *p = i; } pool_destroy(&pool); }
Secure memory:
char *secure_passwd = mem_create_secure(32); if (!secure_passwd) exit(EXIT_FAILURE); get_passwd(secure_passwd, 32); use_passwd(secure_passwd); mem_destroy_secure(&secure_passwd);
Secure memory pool:
Pool *secure_pool; char *secure_passwd; if (!(secure_pool = pool_create_secure(1024 * 1024))) exit(EXIT_FAILURE); secure_passwd = pool_alloc(secure_pool, 32); get_passwd(secure_passwd, 32); use_passwd(secure_passwd); pool_destroy_secure(&secure_pool);
libslack(3), malloc(3), realloc(3), calloc(3), free(3), mlock(2), munlock(2), locker(3)
20100612 raf <raf@raf.org>