21.2 Overriding The Default Memory Management

Flex calls the functions yyalloc, yyrealloc, and yyfree when it needs to allocate or free memory. By default, these functions are wrappers around the standard C functions, malloc, realloc, and free, respectively. You can override the default implementations by telling flex that you will provide your own implementations.

To override the default implementations, you must do two things:

1. Suppress the default implementations by specifying one or more of the following options:
   • %option noyyalloc
   • %option noyyrealloc
   • %option noyyfree.
2. Provide your own implementation of the following functions: ³

   // For a non-reentrant scanner
   void * yyalloc (size_t bytes);
   void * yyrealloc (void * ptr, size_t bytes);
   void   yyfree (void * ptr);
   
   // For a reentrant scanner
   void * yyalloc (size_t bytes, void * yyscanner);
   void * yyrealloc (void * ptr, size_t bytes, void * yyscanner);
   void   yyfree (void * ptr, void * yyscanner);
   

In the following example, we will override all three memory routines. We assume that there is a custom allocator with garbage collection. In order to make this example interesting, we will use a reentrant scanner, passing a pointer to the custom allocator through yyextra.

%{
#include "some_allocator.h"
%}

/* Suppress the default implementations. */
%option noyyalloc noyyrealloc noyyfree
%option reentrant

/* Initialize the allocator. */
%{
#define YY_EXTRA_TYPE  struct allocator*
#define YY_USER_INIT  yyextra = allocator_create();
%}

%%
.|\n   ;
%%

/* Provide our own implementations. */
void * yyalloc (size_t bytes, void* yyscanner) {
    return allocator_alloc (yyextra, bytes);
}

void * yyrealloc (void * ptr, size_t bytes, void* yyscanner) {
    return allocator_realloc (yyextra, bytes);
}

void yyfree (void * ptr, void * yyscanner) {      
    /* Do nothing -- we leave it to the garbage collector. */
}