ASCII Tables (astropy.io.ascii)

Introduction

astropy.io.ascii provides methods for reading and writing a wide range of ASCII data table formats via built-in Extension Reader Classes. The emphasis is on flexibility and convenience of use, although readers can optionally use a less flexible C-based engine for reading and writing for improved performance. This subpackage was originally developed as asciitable.

The following shows a few of the ASCII formats that are available, while the section on Supported formats contains the full list.

• Basic: basic table with customizable delimiters and header configurations

• Cds: CDS format table (also Vizier)

• Daophot: table from the IRAF DAOphot package

• Ecsv: ECSV Format for lossless round-trip of data tables (recommended)

• FixedWidth: table with fixed-width columns (see also Fixed-Width Gallery)

• Ipac: IPAC format table

• HTML: HTML format table contained in a <table> tag

• Latex: LaTeX table with datavalue in the tabular environment

• Mrt: AAS Machine-Readable Tables (MRT))

• SExtractor: SExtractor format table

The strength of astropy.io.ascii is the support for astronomy-specific formats (often with metadata) and specialized data types such as SkyCoord, Time, and Quantity. For reading or writing large data tables in a generic format such as CSV, using the Table - Pandas interface is an option to consider.

Note

It is strongly encouraged to use the functionality from astropy.io.ascii through a higher level interface in the Data Tables package. See Unified File Read/Write Interface for more details.

Getting Started

Reading Tables

The majority of commonly encountered ASCII tables can be read with the read() function. Assume you have a file named sources.dat with the following contents:

obsid redshift  X      Y     object
3102  0.32      4167  4085   Q1250+568-A
877   0.22      4378  3892   "Source 82"

This table can be read with the following:

>>> from astropy.io import ascii
>>> data = ascii.read("sources.dat")  
>>> print(data)                       
obsid redshift  X    Y      object
----- -------- ---- ---- -----------
 3102     0.32 4167 4085 Q1250+568-A
  877     0.22 4378 3892   Source 82

The first argument to the read() function can be the name of a file, a string representation of a table, or a list of table lines. The return value (data in this case) is a Table object.

By default, read() will try to guess the table format by trying all of the supported formats.

Warning

Guessing the file format is often slow for large files because the reader tries parsing the file with every allowed format until one succeeds. For large files it is recommended to disable guessing with guess=False.

If guessing the format does not work, as in the case for unusually formatted tables, you may need to give astropy.io.ascii additional hints about the format.

To specify specific data types for one or more columns, use the converters argument (see Converters for Specifying Dtype for details). For instance if the obsid is actually a string identifier (instead of an integer) you can read the table with the code below. This also illustrates using the preferred Table interface for reading:

>>> from astropy.table import Table
>>> sources = """
... target observatory obsid
... TW_Hya Chandra     22178
... MP_Mus XMM         0406030101"""
>>> data = Table.read(sources, format='ascii', converters={'obsid': str})
>>> data
<Table length=2>
target observatory   obsid
 str6      str7      str10
------ ----------- ----------
TW_Hya     Chandra      22178
MP_Mus         XMM 0406030101

Writing Tables

The write() function provides a way to write a data table as a formatted ASCII table. Most of the input table Supported Formats for reading are also available for writing. This provides a great deal of flexibility in the format for writing.

The following shows how to write a formatted ASCII table using the write() function:

>>> import numpy as np
>>> from astropy.io import ascii
>>> from astropy.table import Table
>>> data = Table()
>>> data['x'] = np.array([1, 2, 3], dtype=np.int32)
>>> data['y'] = data['x'] ** 2
>>> ascii.write(data, 'values.dat', overwrite=True)

The values.dat file will then contain:

x y
1 1
2 4
3 9

It is also possible and encouraged to use the write functionality from astropy.io.ascii through a higher level interface in the Data Tables package (see Unified File Read/Write Interface for more details). For example:

>>> data.write('values.dat', format='ascii', overwrite=True)

Attention

ECSV is recommended

For a reproducible ASCII version of your table, we recommend using the ECSV Format. This stores all the table meta-data (in particular the column types and units) to a comment section at the beginning while maintaining compatibility with most plain CSV readers. It also allows storing richer data like SkyCoord or multidimensional or variable-length columns. ECSV is also supported in java by STIL and TOPCAT,

To write our simple example table to ECSV we use:

>>> data.write('values.ecsv', overwrite=True)  

The .ecsv extension is recognized and implies using ECSV (equivalent to format='ascii.ecsv'). The values.ecsv file will then contain:

# %ECSV 1.0
# ---
# datatype:
# - {name: x, datatype: int32}
# - {name: y, datatype: int32}
# schema: astropy-2.0
x y
1 1
2 4
3 9

Supported Formats

A full list of the supported format values and corresponding format types for ASCII tables is given below. The Write column indicates which formats support write functionality, and the Fast column indicates which formats are compatible with the fast Cython/C engine for reading and writing.

Format	Write	Fast	Description
aastex	Yes		AASTex: AASTeX deluxetable used for AAS journals
basic	Yes	Yes	Basic: Basic table with custom delimiters
cds	Yes		Cds: CDS format table
commented_header	Yes	Yes	CommentedHeader: Column names in a commented line
csv	Yes	Yes	Csv: Basic table with comma-separated values
daophot			Daophot: IRAF DAOphot format table
ecsv	Yes		Ecsv: Enhanced CSV format (recommended)
fixed_width	Yes		FixedWidth: Fixed width
fixed_width_no_header	Yes		FixedWidthNoHeader: Fixed-width with no header
fixed_width_two_line	Yes		FixedWidthTwoLine: Fixed-width with second header line
html	Yes		HTML: HTML format table
ipac	Yes		Ipac: IPAC format table
latex	Yes		Latex: LaTeX table
mrt	Yes		Mrt: AAS Machine-Readable Table format
no_header	Yes	Yes	NoHeader: Basic table with no headers
qdp	Yes		QDP: Quick and Dandy Plotter files
rdb	Yes	Yes	Rdb: Tab-separated with a type definition header line
rst	Yes		RST: reStructuredText simple format table
sextractor			SExtractor: SExtractor format table
tab	Yes	Yes	Tab: Basic table with tab-separated values

Using astropy.io.ascii

The details of using astropy.io.ascii are provided in the following sections:

Reading tables

• Reading Tables
  • Parameters for read()
  • Specifying Header and Data Location
  • Bad or Missing Values
  • Guess Table Format
  • Comments and Metadata
  • Converters for Specifying Dtype
  • Fortran-Style Exponents
  • Advanced Customization
  • Reading Large Tables in Chunks
  • How to Find and Fix Problems Reading a Table

Writing tables

• Writing Tables
  • Examples
  • Parameters for write()
  • Machine-Readable Table Format

ECSV Format

• ECSV Format
  • Usage
  • Simple Table
  • Masked Data
  • Mixin Columns
  • Multidimensional Columns
  • Variable-length arrays
  • Object arrays

Fixed-Width Gallery

• Fixed-Width Gallery
  • Reading
  • Writing
  • Custom Header Rows

Fast ASCII Engine

• Fast ASCII I/O
  • Examples
  • Parallel and Fast Conversion Options
  • Fast Converter
  • Reading Large Tables

Base Class Elements

• Base Class Elements

Extension Reader Classes

• Extension Reader Classes

Performance Tips

By default, when trying to read a file the reader will guess the format, which involves trying to read it with many different readers. For better performance when dealing with large tables, it is recommended to specify the format and any options explicitly, and turn off guessing as well.

Example

If you are reading a simple CSV file with a one-line header with column names, the following:

read('example.csv', format='basic', delimiter=',', guess=False)  # doctest: +SKIP

can be at least an order of magnitude faster than:

read('example.csv')  # doctest: +SKIP

Reference/API

astropy.io.ascii Package

An extensible ASCII table reader and writer.

Functions

convert_numpy(numpy_type)	Return a tuple containing a function which converts a list into a numpy array and the type produced by the converter function.
get_read_trace()	Return a traceback of the attempted read formats for the last call to read where guessing was enabled.
get_reader([Reader, Inputter, Outputter])	Initialize a table reader allowing for common customizations.
get_writer([Writer, fast_writer])	Initialize a table writer allowing for common customizations.
read(table[, guess])	Read the input table and return the table.
set_guess(guess)	Set the default value of the guess parameter for read()
write(table[, output, format, Writer, ...])	Write the input table to filename.

Classes

AASTex(**kwargs)	AASTeX format table.
AllType()	Subclass of all other data types.
BaseData()	Base table data reader.
BaseHeader()	Base table header reader
BaseInputter()	Get the lines from the table input and return a list of lines.
BaseOutputter()	Output table as a dict of column objects keyed on column name.
BaseReader()	Class providing methods to read and write an ASCII table using the specified header, data, inputter, and outputter instances.
BaseSplitter()	Base splitter that uses python's split method to do the work.
Basic()	Character-delimited table with a single header line at the top.
BasicData()	Basic table Data Reader
BasicHeader()	Basic table Header Reader
Cds([readme])	CDS format table.
Column(name)	Table column.
CommentedHeader()	Character-delimited table with column names in a comment line.
ContinuationLinesInputter()	Inputter where lines ending in continuation_char are joined with the subsequent line. Example::.
Csv()	CSV (comma-separated-values) table.
Daophot()	DAOphot format table.
DefaultSplitter()	Default class to split strings into columns using python csv.
Ecsv()	ECSV (Enhanced Character Separated Values) format table.
FastBasic([default_kwargs])	This class is intended to handle the same format addressed by the ordinary Basic writer, but it acts as a wrapper for underlying C code and is therefore much faster.
FastCommentedHeader(**kwargs)	A faster version of the CommentedHeader reader, which looks for column names in a commented line.
FastCsv(**kwargs)	A faster version of the ordinary Csv writer that uses the optimized C parsing engine.
FastNoHeader(**kwargs)	This class uses the fast C engine to read tables with no header line.
FastRdb(**kwargs)	A faster version of the Rdb reader.
FastTab(**kwargs)	A faster version of the ordinary Tab reader that uses the optimized C parsing engine.
FixedWidth([col_starts, col_ends, ...])	Fixed width table with single header line defining column names and positions.
FixedWidthData()	Base table data reader.
FixedWidthHeader()	Fixed width table header reader.
FixedWidthNoHeader([col_starts, col_ends, ...])	Fixed width table which has no header line.
FixedWidthSplitter()	Split line based on fixed start and end positions for each col in self.cols.
FixedWidthTwoLine([position_line, ...])	Fixed width table which has two header lines.
FloatType()	Describes floating-point data.
HTML([htmldict])	HTML format table.
InconsistentTableError	Indicates that an input table is inconsistent in some way.
IntType()	Describes integer data.
Ipac([definition, DBMS])	IPAC format table.
Latex([ignore_latex_commands, latexdict, ...])	LaTeX format table.
Mrt()	AAS MRT (Machine-Readable Table) format table.
NoHeader()	Character-delimited table with no header line.
NoType()	Superclass for StrType and NumType classes.
NumType()	Indicates that a column consists of numerical data.
ParameterError	Indicates that a reader cannot handle a passed parameter.
QDP([table_id, names, err_specs, sep])	Quick and Dandy Plot table.
RST()	reStructuredText simple format table.
Rdb()	Tab-separated file with an extra line after the column definition line that specifies either numeric (N) or string (S) data.
SExtractor()	SExtractor format table.
StrType()	Indicates that a column consists of text data.
Tab()	Tab-separated table.
TableOutputter()	Output the table as an astropy.table.Table object.
WhitespaceSplitter()

Class Inheritance Diagram