Class Token
source code
Tokens are an intermediate data structure used while
constructing the structuring DOM tree for a formatted docstring. There
are five types of Token:
-
Paragraphs
-
Literal blocks
-
Doctest blocks
-
Headings
-
Bullets
The text contained in each Token is stored in the
contents variable. The string in this variable has been
normalized. For paragraphs, this means that it has been converted into a
single line of text, with newline/indentation replaced by single spaces.
For literal blocks and doctest blocks, this means that the appropriate
amount of leading whitespace has been removed from each line.
Each Token has an indentation level associated with it,
stored in the indent variable. This indentation level is
used by the structuring procedure to assemble hierarchical blocks.
|
|
|
| Call Graph |
|---|
|
string
|
|
|
Element
|
|
| Call Graph |
|---|
|
string
|
PARA = 'para'
The tag value for paragraph Tokens.
|
string
|
LBLOCK = 'literalblock'
The tag value for literal Tokens.
|
string
|
DTBLOCK = 'doctestblock'
The tag value for doctest Tokens.
|
string
|
HEADING = 'heading'
The tag value for heading Tokens.
|
string
|
BULLET = 'bullet'
The tag value for bullet Tokens.
|
string
|
contents
The normalized text contained in this Token.
|
int or None
|
indent
The indentation level of this Token (in number of
leading spaces).
|
bool
|
inline
If True, the element is an inline level element, comparable to an
HTML <span> tag.
|
int or None
|
level
The heading-level of this Token if it is a heading;
None, otherwise.
|
int
|
startline
The line on which this Token begins.
|
string
|
tag
This Token's type.
|
__init__(self,
tag,
startline,
contents,
indent,
level=None,
inline=False)
(Constructor)
| source code
call graph |
|
|
| Call Graph |
|---|
Create a new Token.
- Parameters:
tag (string) - The type of the new Token.startline (int) - The line on which the new Token begins.contents (string) - The normalized contents of the new Token.indent (int or None) - The indentation of the new Token (in number of
leading spaces). A value of None indicates an
unknown indentation.level (int or None) - The heading-level of this Token if it is a heading;
None, otherwise.inline (bool) - Is this Token inline as a
<span>?.
|
- Returns:
string
- the formal representation of this
Token.
Tokens have formal representaitons of the form:
<Token: para at line 12>
|
|
|
| Call Graph |
|---|
- Returns: Element
- a DOM representation of this
Token.
|
BULLET
The tag value for bullet Tokens. This
tag value is also used for field tag Tokens,
since fields function syntactically the same as list items.
- Type:
string
- Value:
-
|
indent
The indentation level of this Token (in number of leading
spaces). A value of None indicates an unknown indentation;
this is used for list items and fields that begin with one-line
paragraphs.
- Type:
int or None
|
inline
If True, the element is an inline level element, comparable to an HTML
<span> tag. Else, it is a block level element,
comparable to an HTML <div>.
- Type:
bool
|
level
The heading-level of this Token if it is a heading;
None, otherwise. Valid heading levels are 0, 1, and 2.
- Type:
int or None
|
startline
The line on which this Token begins. This line number is
only used for issuing errors.
- Type:
int
|
tag
This Token's type. Possible values are
Token.PARA (paragraph), Token.LBLOCK (literal
block), Token.DTBLOCK (doctest block),
Token.HEADINGC, and Token.BULLETC.
- Type:
string
|