Any program that uses the CFITSIO interface must include the fitsio.h header file with the statement
#include "fitsio.h"
This header file contains the prototypes for all the CFITSIO user interface routines as well as the definitions of various constants used in the interface. It also defines a C structure of type ‘fitsfile’ that is used by CFITSIO to store the relevant parameters that define the format of a particular FITS file. Application programs must define a pointer to this structure for each FITS file that is to be opened. This structure is initialized (i.e., memory is allocated for the structure) when the FITS file is first opened or created with the fits_open_file or fits_create_file routines. This fitsfile pointer is then passed as the first argument to every other CFITSIO routine that operates on the FITS file. Application programs must not directly read or write elements in this fitsfile structure because the definition of the structure may change in future versions of CFITSIO.
A number of symbolic constants are also defined in fitsio.h for the convenience of application programmers. Use of these symbolic constants rather than the actual numeric value will help to make the source code more readable and easier for others to understand.
String Lengths, for use when allocating character arrays:
#define FLEN_FILENAME 1025 /* max length of a filename */
#define FLEN_KEYWORD 72 /* max length of a keyword */
#define FLEN_CARD 81 /* max length of a FITS header card */
#define FLEN_VALUE 71 /* max length of a keyword value string */
#define FLEN_COMMENT 73 /* max length of a keyword comment string */
#define FLEN_ERRMSG 81 /* max length of a CFITSIO error message */
#define FLEN_STATUS 31 /* max length of a CFITSIO status text string */
Note that FLEN_KEYWORD is longer than the nominal 8-character keyword
name length because the HIERARCH convention supports longer keyword names.
Access modes when opening a FITS file:
#define READONLY 0
#define READWRITE 1
BITPIX data type code values for FITS images:
#define BYTE_IMG 8 /* 8-bit unsigned integers */
#define SHORT_IMG 16 /* 16-bit signed integers */
#define LONG_IMG 32 /* 32-bit signed integers */
#define LONGLONG_IMG 64 /* 64-bit signed integers */
#define FLOAT_IMG -32 /* 32-bit single precision floating point */
#define DOUBLE_IMG -64 /* 64-bit double precision floating point */
The following 4 data type codes are also supported by CFITSIO:
#define SBYTE_IMG 10 /* 8-bit signed integers, equivalent to */
/* BITPIX = 8, BSCALE = 1, BZERO = -128 */
#define USHORT_IMG 20 /* 16-bit unsigned integers, equivalent to */
/* BITPIX = 16, BSCALE = 1, BZERO = 32768 */
#define ULONG_IMG 40 /* 32-bit unsigned integers, equivalent to */
/* BITPIX = 32, BSCALE = 1, BZERO = 2147483648 */
#define ULONGLONG_IMG 80 /* 64-bit unsigned integers, equivalent to */
/* BITPIX = 64, BSCALE = 1, BZERO = 9223372036854775808*/
Codes for the data type of binary table columns and/or for the
data type of variables when reading or writing keywords or data:
DATATYPE TFORM CODE
#define TBIT 1 /* 'X' */
#define TBYTE 11 /* 8-bit unsigned byte, 'B' */
#define TLOGICAL 14 /* logicals (int for keywords */
/* and char for table cols 'L' */
#define TSTRING 16 /* ASCII string, 'A' */
#define TSHORT 21 /* signed short, 'I' */
#define TLONG 41 /* signed long, */
#define TLONGLONG 81 /* 64-bit long signed integer 'K' */
#define TFLOAT 42 /* single precision float, 'E' */
#define TDOUBLE 82 /* double precision float, 'D' */
#define TCOMPLEX 83 /* complex (pair of floats) 'C' */
#define TDBLCOMPLEX 163 /* double complex (2 doubles) 'M' */
The following data type codes are also supported by CFITSIO:
#define TINT 31 /* int */
#define TSBYTE 12 /* 8-bit signed byte, 'S' */
#define TUINT 30 /* unsigned int 'V' */
#define TUSHORT 20 /* unsigned short 'U' */
#define TULONG 40 /* unsigned long */
#define TULONGLONG 80 /* unsigned long long 'W' */
The following data type code is only for use with fits\_get\_coltype
#define TINT32BIT 41 /* signed 32-bit int, 'J' */
HDU type code values (value returned when moving to new HDU):
#define IMAGE_HDU 0 /* Primary Array or IMAGE HDU */
#define ASCII_TBL 1 /* ASCII table HDU */
#define BINARY_TBL 2 /* Binary table HDU */
#define ANY_HDU -1 /* matches any type of HDU */
Column name and string matching case-sensitivity:
#define CASESEN 1 /* do case-sensitive string match */
#define CASEINSEN 0 /* do case-insensitive string match */
Logical states (if TRUE and FALSE are not already defined):
#define TRUE 1
#define FALSE 0
Values to represent undefined floating point numbers:
#define FLOATNULLVALUE -9.11912E-36F
#define DOUBLENULLVALUE -9.1191291391491E-36
Image compression algorithm definitions
#define RICE_1 11
#define GZIP_1 21
#define GZIP_2 22
#define PLIO_1 31
#define HCOMPRESS_1 41
#define NOCOMPRESS -1
#define NO_DITHER -1
#define SUBTRACTIVE_DITHER_1 1
#define SUBTRACTIVE_DITHER_2 2
The concept of the Current Header and Data Unit, or CHDU, is fundamental to the use of the CFITSIO library. A simple FITS image may only contain a single Header and Data unit (HDU), but in general FITS files can contain multiple Header Data Units (also known as ‘extensions’), concatenated one after the other in the file. The user can specify which HDU should be initially opened at run time by giving the HDU name or number after the root file name. For example, ’myfile.fits[4]’ opens the 5th HDU in the file (note that the numbering starts with 0), and ’myfile.fits[EVENTS] opens the HDU with the name ’EVENTS’ (as defined by the EXTNAME or HDUNAME keywords). If no HDU is specified then CFITSIO opens the first HDU (the primary array) by default. The CFITSIO routines which read and write data only operate within the opened HDU, Other CFITSIO routines are provided to move to and open any other existing HDU within the FITS file or to append or insert new HDUs in the FITS file.
Most of the CFITSIO routines have both a short name as well as a longer descriptive name. The short name is only 5 or 6 characters long and is similar to the subroutine name in the Fortran-77 version of FITSIO. The longer name is more descriptive and it is recommended that it be used instead of the short name to more clearly document the source code.
Many of the CFITSIO routines come in families which differ only in the data type of the associated parameter(s). The data type of these routines is indicated by the suffix of the routine name. The short routine names have a 1 or 2 character suffix (e.g., ’j’ in ’ffpkyj’) while the long routine names have a 4 character or longer suffix as shown in the following table:
Long Short Data
Names Names Type
----- ----- ----
_bit x bit
_byt b unsigned byte
_sbyt sb signed byte
_sht i short integer
_lng j long integer
_lnglng jj 8-byte LONGLONG integer (see note below)
_usht ui unsigned short integer
_ulng uj unsigned long integer
_ulnglng ujj unsigned long long integer
_uint uk unsigned int integer
_int k int integer
_flt e real exponential floating point (float)
_fixflt f real fixed-decimal format floating point (float)
_dbl d double precision real floating-point (double)
_fixdbl g double precision fixed-format floating point (double)
_cmp c complex reals (pairs of float values)
_fixcmp fc complex reals, fixed-format floating point
_dblcmp m double precision complex (pairs of double values)
_fixdblcmp fm double precision complex, fixed-format floating point
_log l logical (int)
_str s character string
The logical data type corresponds to ‘int’ for logical keyword values, and ‘byte’ for logical binary table columns. In other words, the value when writing a logical keyword must be stored in an ‘int’ variable, and must be stored in a ‘char’ array when reading or writing to ‘L’ columns in a binary table. Implicit data type conversion is not supported for logical table columns, but is for keywords, so a logical keyword may be read and cast to any numerical data type; a returned value = 0 indicates false, and any other value = true.
The ‘int’ data type may be 2 bytes long on some old PC compilers, but otherwise it is nearly always 4 bytes long. Some 64-bit machines, like the Alpha/OSF, define the ‘short’, ‘int’, and ‘long’ integer data types to be 2, 4, and 8 bytes long, respectively.
Because there is no universal C compiler standard for the name of the 8-byte integer datatype, the fitsio.h include file typedef’s ’LONGLONG’ to be equivalent to an appropriate 8-byte integer data type on each supported platform. For maximum software portability it is recommended that this LONGLONG datatype be used to define 8-byte integer variables rather than using the native data type name on a particular platform. On most 32-bit Unix and Mac OS-X operating systems LONGLONG is equivalent to the intrinsic ’long long’ 8-byte integer datatype. On 64-bit systems (which currently includes Alpha OSF/1, 64-bit Sun Solaris, 64-bit SGI MIPS, and 64-bit Itanium and Opteron PC systems), LONGLONG is simply typedef’ed to be equivalent to ’long’. Microsoft Visual C++ Version 6.0 does not define a ’long long’ data type, so LONGLONG is typedef’ed to be equivalent to the ’__int64’ data type on 32-bit windows systems when using Visual C++.
A related issue that affects the portability of software is how to print out the value of a ’LONGLONG’ variable with printf. Developers may find it convenient to use the following preprocessing statements in their C programs to handle this in a machine-portable manner:
#if defined(_MSC_VER) /* Microsoft Visual C++ */
printf("%I64d", longlongvalue);
#elif (USE_LL_SUFFIX == 1)
printf("%lld", longlongvalue);
#else
printf("%ld", longlongvalue);
#endif
Similarly, the name of the C utility routine that converts a character string of digits into a 8-byte integer value is platform dependent:
#if defined(_MSC_VER) /* Microsoft Visual C++ */
/* VC++ 6.0 does not seem to have an 8-byte conversion routine */
#elif (USE_LL_SUFFIX == 1)
longlongvalue = atoll(*string);
#else
longlongvalue = atol(*string);
#endif
When dealing with the FITS byte data type it is important to remember that the raw values (before any scaling by the BSCALE and BZERO, or TSCALn and TZEROn keyword values) in byte arrays (BITPIX = 8) or byte columns (TFORMn = ’B’) are interpreted as unsigned bytes with values ranging from 0 to 255. Some C compilers define a ’char’ variable as signed, so it is important to explicitly declare a numeric char variable as ’unsigned char’ to avoid any ambiguity
One feature of the CFITSIO routines is that they can operate on a ‘X’ (bit) column in a binary table as though it were a ‘B’ (byte) column. For example a ‘11X’ data type column can be interpreted the same as a ‘2B’ column (i.e., 2 unsigned 8-bit bytes). In some instances, it can be more efficient to read and write whole bytes at a time, rather than reading or writing each individual bit.
The complex and double precision complex data types are not directly supported in ANSI C so these data types should be interpreted as pairs of float or double values, respectively, where the first value in each pair is the real part, and the second is the imaginary part.
Although FITS does not directly support unsigned integers as one of its fundamental data types, FITS can still be used to efficiently store unsigned integer data values in images and binary tables. The convention used in FITS files is to store the unsigned integers as signed integers with an associated offset (specified by the BZERO or TZEROn keyword). For example, to store unsigned 16-bit integer values in a FITS image the image would be defined as a signed 16-bit integer (with BITPIX keyword = SHORT_IMG = 16) with the keywords BSCALE = 1.0 and BZERO = 32768. Thus the unsigned values of 0, 32768, and 65535, for example, are physically stored in the FITS image as -32768, 0, and 32767, respectively; CFITSIO automatically adds the BZERO offset to these values when they are read. Similarly, in the case of unsigned 32-bit integers the BITPIX keyword would be equal to LONG_IMG = 32 and BZERO would be equal to 2147483648 (i.e. 2 raised to the 31st power).
The CFITSIO interface routines will efficiently and transparently apply the appropriate offset in these cases so in general application programs do not need to be concerned with how the unsigned values are actually stored in the FITS file. As a convenience for users, CFITSIO has several predefined constants for the value of BITPIX (USHORT_IMG, ULONG_IMG, ULONGLONG_IMG) and for the TFORMn value in the case of binary tables (‘U’, ‘V’, and ‘W’) which programmers can use when creating FITS files containing unsigned integer values. The following code fragment illustrates how to write a FITS 1-D primary array of unsigned 16-bit integers:
unsigned short uarray[100];
int naxis, status;
long naxes[10], group, firstelem, nelements;
...
status = 0;
naxis = 1;
naxes[0] = 100;
fits_create_img(fptr, USHORT_IMG, naxis, naxes, &status);
firstelem = 1;
nelements = 100;
fits_write_img(fptr, TUSHORT, firstelem, nelements,
uarray, &status);
...
In the above example, the 2nd parameter in fits_create_img tells CFITSIO to write the header keywords appropriate for an array of 16-bit unsigned integers (i.e., BITPIX = 16 and BZERO = 32768). Then the fits_write_img routine writes the array of unsigned short integers (uarray) into the primary array of the FITS file. Similarly, a 32-bit unsigned integer image may be created by setting the second parameter in fits_create_img equal to ‘ULONG_IMG’ and by calling the fits_write_img routine with the second parameter = TULONG to write the array of unsigned long image pixel values.
An analogous set of routines are available for reading or writing unsigned integer values and signed byte values in a FITS binary table extension. When specifying the TFORMn keyword value which defines the format of a column, CFITSIO recognizes 4 additional data type codes besides those already defined in the FITS standard: ‘U’ meaning a 16-bit unsigned integer column, ‘V’ for a 32-bit unsigned integer column, ‘W’ for a 64-bit unsigned integer column, and ’S’ for a signed byte column. These non-standard data type codes are not actually written into the FITS file but instead are just used internally within CFITSIO. The following code fragment illustrates how to use these features:
unsigned short uarray[100];
unsigned int varray[100];
int colnum, tfields, status;
long nrows, firstrow, firstelem, nelements, pcount;
char extname[] = "Test_table"; /* extension name */
/* define the name, data type, and physical units for 4 columns */
char *ttype[] = { "Col_1", "Col_2", "Col_3", "Col_4" };
char *tform[] = { "1U", "1V", "1W", "1S"}; /* special CFITSIO codes */
char *tunit[] = { " ", " ", " ", " " };
...
/* write the header keywords */
status = 0;
nrows = 1;
tfields = 3
pcount = 0;
fits_create_tbl(fptr, BINARY_TBL, nrows, tfields, ttype, tform,
tunit, extname, &status);
/* write the unsigned shorts to the 1st column */
colnum = 1;
firstrow = 1;
firstelem = 1;
nelements = 100;
fits_write_col(fptr, TUSHORT, colnum, firstrow, firstelem,
nelements, uarray, &status);
/* now write the unsigned longs to the 2nd column */
colnum = 2;
fits_write_col(fptr, TUINT, colnum, firstrow, firstelem,
nelements, varray, &status);
...
Note that the non-standard TFORM values for the 3 columns, ‘U’, ‘V’, and ‘W’ tell CFITSIO to write the keywords appropriate for unsigned 16-bit, unsigned 32-bit and unsigned 64-bit integers, respectively (i.e., TFORMn = ’1I’ and TZEROn = 32768 for unsigned 16-bit integers, TFORMn = ’1J’ and TZEROn = 2147483648 for unsigned 32-bit integers, and TFORMn = ’1K’ and TZEROn = 9223372036854775808 for unsigned 64-bit integers). The ’S’ TFORMn value tells CFITSIO to write the keywords appropriate for a signed 8-bit byte column with TFORMn = ’1B’ and TZEROn = -128. The calls to fits_write_col then write the arrays of unsigned integer values to the columns.
The character string values in a FITS header or in an ASCII column in a FITS table extension are generally padded out with non-significant space characters (ASCII 32) to fill up the header record or the column width. When reading a FITS string value, the CFITSIO routines will strip off these non-significant trailing spaces and will return a null-terminated string value containing only the significant characters. Leading spaces in a FITS string are considered significant. If the string contains all blanks, then CFITSIO will return a single blank character, i.e, the first blank is considered to be significant, since it distinguishes the string from a null or undefined string, but the remaining trailing spaces are not significant.
Similarly, when writing string values to a FITS file the CFITSIO routines expect to get a null-terminated string as input; CFITSIO will pad the string with blanks if necessary when writing it to the FITS file.
The FITS standard does not require trailing spaces to be treated in
this way, but it does allow a more seamless transition from the
FORTRAN FITS world where trailing spaces are often treated as
insignificant. Users who wish the greatest fidelity when transferring
strings can use the _byt variants of column readers and writers
(functions fits_{read,write}_col_byt). These routines will
transfer the raw fixed-length vectors of character bytes of the
column, including any trailing blanks of course. The _byt
variants make no attempt to null-terminate any elements. A NULL
string would be indicated by its first character being a NUL byte.
When calling CFITSIO routines that return a character string it is vital that the size of the char array be large enough to hold the entire string of characters, otherwise CFITSIO will overwrite whatever memory locations follow the char array, possibly causing the program to execute incorrectly. This type of error can be difficult to debug, so programmers should always ensure that the char arrays are allocated enough space to hold the longest possible string, including the terminating NULL character. The fitsio.h file contains the following defined constants which programmers are strongly encouraged to use whenever they are allocating space for char arrays:
#define FLEN_FILENAME 1025 /* max length of a filename */ #define FLEN_KEYWORD 72 /* max length of a keyword */ #define FLEN_CARD 81 /* length of a FITS header card */ #define FLEN_VALUE 71 /* max length of a keyword value string */ #define FLEN_COMMENT 73 /* max length of a keyword comment string */ #define FLEN_ERRMSG 81 /* max length of a CFITSIO error message */ #define FLEN_STATUS 31 /* max length of a CFITSIO status text string */
For example, when declaring a char array to hold the value string of FITS keyword, use the following statement:
char value[FLEN_VALUE];
Note that FLEN_KEYWORD is longer than needed for the nominal 8-character keyword name because the HIERARCH convention supports longer keyword names.
The CFITSIO routines that read and write numerical data can perform implicit data type conversion. This means that the data type of the variable or array in the program does not need to be the same as the data type of the value in the FITS file. Data type conversion is supported for numerical and string data types (if the string contains a valid number enclosed in quotes) when reading a FITS header keyword value and for numeric values when reading or writing values in the primary array or a table column. CFITSIO returns status = NUM_OVERFLOW if the converted data value exceeds the range of the output data type. Implicit data type conversion is not supported within binary tables for string, logical, complex, or double complex data types.
In addition, any table column may be read as if it contained string values. In the case of numeric columns the returned string will be formatted using the TDISPn display format if it exists.
When reading numerical data values in the primary array or a table column, the values will be scaled automatically by the BSCALE and BZERO (or TSCALn and TZEROn) header values if they are present in the header. The scaled data that is returned to the reading program will have
output value = (FITS value) * BSCALE + BZERO
(a corresponding formula using TSCALn and TZEROn is used when reading from table columns). In the case of integer output values the floating point scaled value is truncated to an integer (not rounded to the nearest integer). The fits_set_bscale and fits_set_tscale routines (described in the ‘Advanced’ chapter) may be used to override the scaling parameters defined in the header (e.g., to turn off the scaling so that the program can read the raw unscaled values from the FITS file).
When writing numerical data to the primary array or to a table column the data values will generally be automatically inversely scaled by the value of the BSCALE and BZERO (or TSCALn and TZEROn) keyword values if they they exist in the header. These keywords must have been written to the header before any data is written for them to have any immediate effect. One may also use the fits_set_bscale and fits_set_tscale routines to define or override the scaling keywords in the header (e.g., to turn off the scaling so that the program can write the raw unscaled values into the FITS file). If scaling is performed, the inverse scaled output value that is written into the FITS file will have
FITS value = ((input value) - BZERO) / BSCALE
(a corresponding formula using TSCALn and TZEROn is used when writing to table columns). Rounding to the nearest integer, rather than truncation, is performed when writing integer data types to the FITS file.
The ANSI/IEEE-754 floating-point number standard defines certain special values that are used to represent such quantities as Not-a-Number (NaN), denormalized, underflow, overflow, and infinity. (See the Appendix in the FITS standard or the FITS User’s Guide for a list of these values). The CFITSIO routines that read floating point data in FITS files recognize these IEEE special values and by default interpret the overflow and infinity values as being equivalent to a NaN, and convert the underflow and denormalized values into zeros. In some cases programmers may want access to the raw IEEE values, without any modification by CFITSIO. This can be done by calling the fits_read_img or fits_read_col routines while specifying 0.0 as the value of the NULLVAL parameter. This will force CFITSIO to simply pass the IEEE values through to the application program without any modification. This is not fully supported on VAX/VMS machines, however, where there is no easy way to bypass the default interpretation of the IEEE special values. This is also not supported when reading floating-point images that have been compressed with the FITS tiled image compression convention that is discussed in section 5.6; the pixels values in tile compressed images are represented by scaled integers, and a reserved integer value (not a NaN) is used to represent undefined pixels.
Nearly all the CFITSIO routines return an error status value in 2 ways: as the value of the last parameter in the function call, and as the returned value of the function itself. This provides some flexibility in the way programmers can test if an error occurred, as illustrated in the following 2 code fragments:
if ( fits_write_record(fptr, card, &status) )
printf(" Error occurred while writing keyword.");
or,
fits_write_record(fptr, card, &status);
if ( status )
printf(" Error occurred while writing keyword.");
A listing of all the CFITSIO status code values is given at the end of this document. Programmers are encouraged to use the symbolic mnemonics (defined in fitsio.h) rather than the actual integer status values to improve the readability of their code.
The CFITSIO library uses an ‘inherited status’ convention for the status parameter which means that if a routine is called with a positive input value of the status parameter as input, then the routine will exit immediately without changing the value of the status parameter. Thus, if one passes the status value returned from each CFITSIO routine as input to the next CFITSIO routine, then whenever an error is detected all further CFITSIO processing will cease. This convention can simplify the error checking in application programs because it is not necessary to check the value of the status parameter after every single CFITSIO routine call. If a program contains a sequence of several CFITSIO calls, one can just check the status value after the last call. Since the returned status values are generally distinctive, it should be possible to determine which routine originally returned the error status.
CFITSIO also maintains an internal stack of error messages (80-character maximum length) which in many cases provide a more detailed explanation of the cause of the error than is provided by the error status number alone. It is recommended that the error message stack be printed out whenever a program detects a CFITSIO error. The function fits_report_error will print out the entire error message stack, or alternatively one may call fits_read_errmsg to get the error messages one at a time.
CFITSIO provides easy-to-use support for reading and writing data in variable length fields of a binary table. The variable length columns have TFORMn keyword values of the form ‘1Pt(len)’ or ‘1Qt(len)’ where ‘t’ is the data type code (e.g., I, J, E, D, etc.) and ‘len’ is an integer specifying the maximum length of the vector in the table. The ’P’ type variable length columns use 32-bit array length and byte offset values, whereas the ’Q’ type columns use 64-bit values, which may be required when dealing with large arrays. CFITSIO supports a local convention that interprets the ’P’ type descriptors as unsigned 32-bit integers, which provides a factor of 2 greater range for the array length or heap address than is possible with 32-bit ’signed’ integers. Note, however, that other software packages may not support this convention, and may be unable to read thees extended range variable length records.
If the value of ‘len’ is not specified when the table is created (e.g., if the TFORM keyword value is simply specified as ’1PE’ instead of ’1PE(400) ), then CFITSIO will automatically scan the table when it is closed to determine the maximum length of the vector and will append this value to the TFORMn value.
The same routines that read and write data in an ordinary fixed length binary table extension are also used for variable length fields, however, the routine parameters take on a slightly different interpretation as described below.
All the data in a variable length field is written into an area called the ‘heap’ which follows the main fixed-length FITS binary table. The size of the heap, in bytes, is specified by the PCOUNT keyword in the FITS header. When creating a new binary table, the initial value of PCOUNT should usually be set to zero. CFITSIO will recompute the size of the heap as the data is written and will automatically update the PCOUNT keyword value when the table is closed. When writing variable length data to a table, CFITSIO will automatically extend the size of the heap area if necessary, so that any following HDUs do not get overwritten.
By default the heap data area starts immediately after the last row of the fixed-length table. This default starting location may be overridden by the THEAP keyword, but this is not recommended. If additional rows of data are added to the table, CFITSIO will automatically shift the the heap down to make room for the new rows, but it is obviously be more efficient to initially create the table with the necessary number of blank rows, so that the heap does not needed to be constantly moved.
When writing row of data to a variable length field the entire array of values for a given row of the table must be written with a single call to fits_write_col. The total length of the array is given by nelements + firstelem - 1. Additional elements cannot be appended to an existing vector at a later time since any attempt to do so will simply overwrite all the previously written data and the new data will be written to a new area of the heap. The fits_compress_heap routine is provided to compress the heap and recover any unused space. To avoid having to deal with this issue, it is recommended that rows in a variable length field should only be written once. An exception to this general rule occurs when setting elements of an array as undefined. It is allowed to first write a dummy value into the array with fits_write_col, and then call fits_write_col_nul to flag the desired elements as undefined. Note that the rows of a table, whether fixed or variable length, do not have to be written consecutively and may be written in any order.
When writing to a variable length ASCII character field (e.g., TFORM = ’1PA’) only a single character string can be written. The ‘firstelem’ and ‘nelements’ parameter values in the fits_write_col routine are ignored and the number of characters to write is simply determined by the length of the input null-terminated character string.
The fits_write_descript routine is useful in situations where multiple rows of a variable length column have the identical array of values. One can simply write the array once for the first row, and then use fits_write_descript to write the same descriptor values into the other rows; all the rows will then point to the same storage location thus saving disk space.
When reading from a variable length array field one can only read as many elements as actually exist in that row of the table; reading does not automatically continue with the next row of the table as occurs when reading an ordinary fixed length table field. Attempts to read more than this will cause an error status to be returned. One can determine the number of elements in each row of a variable column with the fits_read_descript routine.
CFITSIO supports simultaneous read and write access to different HDUs in the same FITS file in some circumstances, as described below:
When CFITSIO is compiled with the -D_REENTRANT directive (as can be tested with the fits_is_reentrant function) different threads can call any of the CFITSIO routines to simultaneously read or write separate FITS files. Multiple threads can also read data from the same FITS file simultaneously, as long as the file was opened independently by each thread. This relies on the operating system to correctly deal with reading the same file by multiple processes. Different threads should not share the same ’fitsfile’ pointer to read an opened FITS file, unless locks are placed around the calls to the CFITSIO reading routines. Different threads should never try to write to the same FITS file.
A single process may open the same FITS file with READONLY access multiple times, and thus create multiple ’fitsfile*’ pointers to that same file within CFITSIO. This relies on the operating system’s ability to open a single file multiple times and correctly manage the subsequent read requests directed to the different C ’file*’ pointers, which actually all point to the same file. CFITSIO simply executes the read requests to the differnet ’fitsfile*’ pointers the same as if they were physically different files.
CFITSIO supports opening the same FITS file multiple times with WRITE access, but it only physically opens the file (at the operating system level) once, on the first call to fits_open_file. If fits_open_file is subsequently called to open the same file again, CFITSIO will recognize that the file is already open, and will return a new ’fitsfile*’ pointer that logically points to the first ’fitsfile*’ pointer, without actually opening the file a second time. The application program can then treat the 2 ’fitsfile*’ pointers as if they point to different files, and can seemingly move to and write data to 2 different HDUs within the same file. However, each time the application program switches which ’fitsfile*’ pointer it is writing to, CFITSIO will flush any internal buffers that contain data written to the first ’fitsfile*’ pointer, then move to the HDU that the other ’fitsfile*’ pointer is writing to. Obviously, this may add a significant amount of computational overhead if the application program uses this feature to frequently switch back and forth between writing to 2 (or more) HDUs in the same file, so this capability should be used judiciously.
Note that CFITSIO will not allow a FITS file to be opened a second time with READWRITE access if it was opened previously with READONLY access.
It is not required to know the total size of a FITS data array or table before beginning to write the data to the FITS file. In the case of the primary array or an image extension, one should initially create the array with the size of the highest dimension (largest NAXISn keyword) set to a dummy value, such as 1. Then after all the data have been written and the true dimensions are known, then the NAXISn value should be updated using the fits_update_key routine before moving to another extension or closing the FITS file.
When writing to FITS tables, CFITSIO automatically keeps track of the highest row number that is written to, and will increase the size of the table if necessary. CFITSIO will also automatically insert space in the FITS file if necessary, to ensure that the data ’heap’, if it exists, and/or any additional HDUs that follow the table do not get overwritten as new rows are written to the table.
As a general rule it is best to specify the initial number of rows = 0 when the table is created, then let CFITSIO keep track of the number of rows that are actually written. The application program should not manually update the number of rows in the table (as given by the NAXIS2 keyword) since CFITSIO does this automatically. If a table is initially created with more than zero rows, then this will usually be considered as the minimum size of the table, even if fewer rows are actually written to the table. Thus, if a table is initially created with NAXIS2 = 20, and CFITSIO only writes 10 rows of data before closing the table, then NAXIS2 will remain equal to 20. If however, 30 rows of data are written to this table, then NAXIS2 will be increased from 20 to 30. The one exception to this automatic updating of the NAXIS2 keyword is if the application program directly modifies the value of NAXIS2 (up or down) itself just before closing the table. In this case, CFITSIO does not update NAXIS2 again, since it assumes that the application program must have had a good reason for changing the value directly. This is not recommended, however, and is only provided for backward compatibility with software that initially creates a table with a large number of rows, than decreases the NAXIS2 value to the actual smaller value just before closing the table.
CFITSIO places very few restrictions on the size of FITS files that it reads or writes. There are a few limits, however, that may affect some extreme cases:
1. The maximum number of FITS files that may be simultaneously opened by CFITSIO is set by NMAXFILES, as defined in fitsio2.h. The current default value is 1000, but this may be increased if necessary. Note that CFITSIO allocates NIOBUF * 2880 bytes of I/O buffer space for each file that is opened. The default value of NIOBUF is 40 (defined in fitsio.h), so this amounts to more than 115K of memory for each opened file (or 115 MB for 1000 opened files). Note that the underlying operating system, may have a lower limit on the number of files that can be opened simultaneously.
2. It used to be common for computer systems to only support disk files up to 2**31 bytes = 2.1 GB in size, but most systems now support larger files. CFITSIO can optionally read and write these so-called ’large files’ that are greater than 2.1 GB on platforms where they are supported, but this usually requires that special compiler option flags be specified to turn on this option. On linux and solaris systems the compiler flags are ’-D_LARGEFILE_SOURCE’ and ‘-D_FILE_OFFSET_BITS=64’. These flags may also work on other platforms but this has not been tested. Starting with version 3.0 of CFITSIO, the default Makefile that is distributed with CFITSIO will include these 2 compiler flags when building on Solaris and Linux PC systems. Users on other platforms will need to add these compiler flags manually if they want to support large files. In most cases it appears that it is not necessary to include these compiler flags when compiling application code that call the CFITSIO library routines.
When CFITSIO is built with large file support (e.g., on Solaris and Linux PC system by default) then it can read and write FITS data files on disk that have any of these conditions:
The current maximum FITS file size supported by CFITSIO is about 6 terabytes (containing 2**31 FITS blocks, each 2880 bytes in size). Currently, support for large files in CFITSIO has been tested on the Linux, Solaris, and IBM AIX operating systems.
Note that when writing application programs that are intended to support large files it is important to use 64-bit integer variables to store quantities such as the dimensions of images, or the number of rows in a table. These programs must also call the special versions of some of the CFITSIO routines that have been adapted to support 64-bit integers. The names of these routines end in ’ll’ (’el’ ’el’) to distinguish them from the 32-bit integer version (e.g., fits_get_num_rowsll).