Format Text for Display

Functions related to displaying unicode text. Unicode characters don’t all have the same width so we need helper functions for displaying them.

New in version 0.2: kitchen.display API 1.0.0

kitchen.text.display.textual_width(msg, control_chars='guess', encoding='utf-8', errors='replace')

Get the textual width of a string

Parameters

  • msg – str string or byte bytes to get the width of

  • control_chars –

    specify how to deal with control characters. Possible values are:

    guess

    (default) will take a guess for control character widths. Most codes will return zero width. backspace, delete, and clear delete return -1. escape currently returns -1 as well but this is not guaranteed as it’s not always correct

    strict

    will raise kitchen.text.exceptions.ControlCharError if a control character is encountered

  • encoding – If we are given a byte bytes this is used to decode it into str string. Any characters that are not decodable in this encoding will get a value dependent on the errors parameter.

  • errors – How to treat errors encoding the byte bytes to str string. Legal values are the same as for kitchen.text.converters.to_unicode(). The default value of replace will cause undecodable byte sequences to have a width of one. ignore will have a width of zero.

Raises

ControlCharError – if msg contains a control character and control_chars is strict.

Returns

Textual width of the msg. This is the amount of space that the string will consume on a monospace display. It’s measured in the number of cell positions or columns it will take up on a monospace display. This is not the number of glyphs that are in the string.

Note

This function can be wrong sometimes because Unicode does not specify a strict width value for all of the code points. In particular, we’ve found that some Tamil characters take up to four character cells but we return a lesser amount.

kitchen.text.display.textual_width_chop(msg, chop, encoding='utf-8', errors='replace')

Given a string, return it chopped to a given textual width

Parameters

  • msg – str string or byte bytes to chop

  • chop – Chop msg if it exceeds this textual width

  • encoding – If we are given a byte bytes, this is used to decode it into a str string. Any characters that are not decodable in this encoding will be assigned a width of one.

  • errors – How to treat errors encoding the byte bytes to str. Legal values are the same as for kitchen.text.converters.to_unicode()

Return type

str string

Returns

str string of the msg chopped at the given textual width

This is what you want to use instead of %.*s, as it does the “right” thing with regard to UTF-8 sequences, control characters, and characters that take more than one cell position. Eg:

>>> # Wrong: only displays 8 characters because it is operating on bytes
>>> print "%.*s" % (10, 'café ñunru!')
café ñun
>>> # Properly operates on graphemes
>>> '%s' % (textual_width_chop('café ñunru!', 10))
café ñunru
>>> # takes too many columns because the kanji need two cell positions
>>> print '1234567890\n%.*s' % (10, u'一二三四五六七八九十')
1234567890
一二三四五六七八九十
>>> # Properly chops at 10 columns
>>> print '1234567890\n%s' % (textual_width_chop(u'一二三四五六七八九十', 10))
1234567890
一二三四五
kitchen.text.display.textual_width_fill(msg, fill, chop=None, left=True, prefix='', suffix='')

Expand a str string to a specified textual width or chop to same

Parameters

  • msg – str string to format

  • fill – pad string until the textual width of the string is this length

  • chop – before doing anything else, chop the string to this length. Default: Don’t chop the string at all

  • left – If True (default) left justify the string and put the padding on the right. If False, pad on the left side.

  • prefix – Attach this string before the field we’re filling

  • suffix – Append this string to the end of the field we’re filling

Return type

str string

Returns

msg formatted to fill the specified width. If no chop is specified, the string could exceed the fill length when completed. If prefix or suffix are printable characters, the string could be longer than the fill width.

Note

prefix and suffix should be used for “invisible” characters like highlighting, color changing escape codes, etc. The fill characters are appended outside of any prefix or suffix elements. This allows you to only highlight msg inside of the field you’re filling.

Warning

msg, prefix, and suffix should all be representable as unicode characters. In particular, any escape sequences in prefix and suffix need to be convertible to str. If you need to use byte sequences here rather than unicode characters, use byte_string_textual_width_fill() instead.

This function expands a string to fill a field of a particular textual width. Use it instead of %*.*s, as it does the “right” thing with regard to UTF-8 sequences, control characters, and characters that take more than one cell position in a display. Example usage:

>>> msg = u'一二三四五六七八九十'
>>> # Wrong: This uses 10 characters instead of 10 cells:
>>> u":%-*.*s:" % (10, 10, msg[:9])
:一二三四五六七八九 :
>>> # This uses 10 cells like we really want:
>>> u":%s:" % (textual_width_fill(msg[:9], 10, 10))
:一二三四五:

>>> # Wrong: Right aligned in the field, but too many cells
>>> u"%20.10s" % (msg)
          一二三四五六七八九十
>>> # Correct: Right aligned with proper number of cells
>>> u"%s" % (textual_width_fill(msg, 20, 10, left=False))
          一二三四五

>>> # Wrong: Adding some escape characters to highlight the line but too many cells
>>> u"%s%20.10s%s" % (prefix, msg, suffix)
u'          一二三四五六七八九十'
>>> # Correct highlight of the line
>>> u"%s%s%s" % (prefix, display.textual_width_fill(msg, 20, 10, left=False), suffix)
u'          一二三四五'

>>> # Correct way to not highlight the fill
>>> u"%s" % (display.textual_width_fill(msg, 20, 10, left=False, prefix=prefix, suffix=suffix))
u'          一二三四五'
kitchen.text.display.wrap(text, width=70, initial_indent='', subsequent_indent='', encoding='utf-8', errors='replace')

Works like we want textwrap.wrap() to work,

Parameters

  • text – str string or byte bytes to wrap

  • width – textual width at which to wrap. Default: 70

  • initial_indent – string to use to indent the first line. Default: do not indent.

  • subsequent_indent – string to use to wrap subsequent lines. Default: do not indent

  • encoding – Encoding to use if text is a byte bytes

  • errors – error handler to use if text is a byte bytes and contains some undecodable characters.

Return type

list of str strings

Returns

list of lines that have been text wrapped and indented.

textwrap.wrap() from the python standard library has two drawbacks that this attempts to fix:

  1. It does not handle textual width. It only operates on bytes or characters which are both inadequate (due to multi-byte and double width characters).

  2. It malforms lists and blocks.

kitchen.text.display.fill(text, *args, **kwargs)

Works like we want textwrap.fill() to work

Parameters

text – str string or byte bytes to process

Returns

str string with each line separated by a newline

See also

kitchen.text.display.wrap()

for other parameters that you can give this command.

This function is a light wrapper around kitchen.text.display.wrap(). Where that function returns a list of lines, this function returns one string with each line separated by a newline.

kitchen.text.display.byte_string_textual_width_fill(msg, fill, chop=None, left=True, prefix='', suffix='', encoding='utf-8', errors='replace')

Expand a byte bytes to a specified textual width or chop to same

Parameters

  • msg – byte bytes encoded in UTF-8 that we want formatted

  • fill – pad msg until the textual width is this long

  • chop – before doing anything else, chop the string to this length. Default: Don’t chop the string at all

  • left – If True (default) left justify the string and put the padding on the right. If False, pad on the left side.

  • prefix – Attach this byte bytes before the field we’re filling

  • suffix – Append this byte bytes to the end of the field we’re filling

Return type

byte bytes

Returns

msg formatted to fill the specified textual width. If no chop is specified, the string could exceed the fill length when completed. If prefix or suffix are printable characters, the string could be longer than fill width.

Note

prefix and suffix should be used for “invisible” characters like highlighting, color changing escape codes, etc. The fill characters are appended outside of any prefix or suffix elements. This allows you to only highlight msg inside of the field you’re filling.

See also

textual_width_fill()

For example usage. This function has only two differences.

  1. it takes byte bytes for prefix and suffix so you can pass in arbitrary sequences of bytes, not just unicode characters.

  2. it returns a byte bytes instead of a str string.

Internal Data

There are a few internal functions and variables in this module. Code outside of kitchen shouldn’t use them but people coding on kitchen itself may find them useful.

kitchen.text.display._COMBINING = ((768, 879), (1155, 1161), (1425, 1469), (1471, 1471), (1473, 1474), (1476, 1477), (1479, 1479), (1536, 1539), (1552, 1562), (1611, 1631), (1648, 1648), (1750, 1764), (1767, 1768), (1770, 1773), (1807, 1807), (1809, 1809), (1840, 1866), (1958, 1968), (2027, 2035), (2045, 2045), (2070, 2073), (2075, 2083), (2085, 2087), (2089, 2093), (2137, 2139), (2259, 2273), (2275, 2303), (2305, 2306), (2364, 2364), (2369, 2376), (2381, 2381), (2385, 2388), (2402, 2403), (2433, 2433), (2492, 2492), (2497, 2500), (2509, 2509), (2530, 2531), (2558, 2558), (2561, 2562), (2620, 2620), (2625, 2626), (2631, 2632), (2635, 2637), (2672, 2673), (2689, 2690), (2748, 2748), (2753, 2757), (2759, 2760), (2765, 2765), (2786, 2787), (2817, 2817), (2876, 2876), (2879, 2879), (2881, 2883), (2893, 2893), (2902, 2902), (2946, 2946), (3008, 3008), (3021, 3021), (3134, 3136), (3142, 3144), (3146, 3149), (3157, 3158), (3260, 3260), (3263, 3263), (3270, 3270), (3276, 3277), (3298, 3299), (3387, 3388), (3393, 3395), (3405, 3405), (3530, 3530), (3538, 3540), (3542, 3542), (3633, 3633), (3636, 3642), (3655, 3662), (3761, 3761), (3764, 3772), (3784, 3789), (3864, 3865), (3893, 3893), (3895, 3895), (3897, 3897), (3953, 3966), (3968, 3972), (3974, 3975), (3984, 3991), (3993, 4028), (4038, 4038), (4141, 4144), (4146, 4146), (4150, 4151), (4153, 4154), (4184, 4185), (4237, 4237), (4448, 4607), (4957, 4959), (5906, 5908), (5938, 5940), (5970, 5971), (6002, 6003), (6068, 6069), (6071, 6077), (6086, 6086), (6089, 6099), (6109, 6109), (6155, 6157), (6313, 6313), (6432, 6434), (6439, 6440), (6450, 6450), (6457, 6459), (6679, 6680), (6752, 6752), (6773, 6780), (6783, 6783), (6832, 6845), (6912, 6915), (6964, 6964), (6966, 6970), (6972, 6972), (6978, 6978), (6980, 6980), (7019, 7027), (7082, 7083), (7142, 7142), (7154, 7155), (7223, 7223), (7376, 7378), (7380, 7392), (7394, 7400), (7405, 7405), (7412, 7412), (7416, 7417), (7616, 7673), (7675, 7679), (8203, 8207), (8234, 8238), (8288, 8291), (8298, 8303), (8400, 8432), (11503, 11505), (11647, 11647), (11744, 11775), (12330, 12335), (12441, 12442), (42607, 42607), (42612, 42621), (42654, 42655), (42736, 42737), (43014, 43014), (43019, 43019), (43045, 43046), (43204, 43204), (43232, 43249), (43307, 43309), (43347, 43347), (43443, 43443), (43456, 43456), (43696, 43696), (43698, 43700), (43703, 43704), (43710, 43711), (43713, 43713), (43766, 43766), (44013, 44013), (64286, 64286), (65024, 65039), (65056, 65071), (65279, 65279), (65529, 65531), (66045, 66045), (66272, 66272), (66422, 66426), (68097, 68099), (68101, 68102), (68108, 68111), (68152, 68154), (68159, 68159), (68325, 68326), (68900, 68903), (69446, 69456), (69702, 69702), (69759, 69759), (69817, 69818), (69888, 69890), (69939, 69940), (70003, 70003), (70080, 70080), (70090, 70090), (70197, 70198), (70377, 70378), (70459, 70460), (70477, 70477), (70502, 70508), (70512, 70516), (70722, 70722), (70726, 70726), (70750, 70750), (70850, 70851), (71103, 71104), (71231, 71231), (71350, 71351), (71467, 71467), (71737, 71738), (72160, 72160), (72244, 72244), (72263, 72263), (72345, 72345), (72767, 72767), (73026, 73026), (73028, 73029), (73111, 73111), (92912, 92916), (92976, 92982), (113822, 113822), (119141, 119145), (119149, 119170), (119173, 119179), (119210, 119213), (119362, 119364), (122880, 122886), (122888, 122904), (122907, 122913), (122915, 122916), (122918, 122922), (123184, 123190), (123628, 123631), (125136, 125142), (125252, 125258), (917505, 917505), (917536, 917631), (917760, 917999))

Internal table, provided by this module to list code points which combine with other characters and therefore should have no textual width. This is a sorted tuple of non-overlapping intervals. Each interval is a tuple listing a starting code point and ending code point. Every code point between the two end points is a combining character.

See also

_generate_combining_table()

for how this table is generated

This table was last regenerated on python-3.8.0a3 with unicodedata.unidata_version 12.0.0

kitchen.text.display._generate_combining_table()

Combine Markus Kuhn’s data with unicodedata to make combining char list

Return type

tuple of tuples

Returns

tuple of intervals of code points that are combining character. Each interval is a 2-tuple of the starting code point and the ending code point for the combining characters.

In normal use, this function serves to tell how we’re generating the combining char list. For speed reasons, we use this to generate a static list and just use that later.

Markus Kuhn’s list of combining characters is more complete than what’s in the python unicodedata library but the python unicodedata is synced against later versions of the unicode database

This is used to generate the _COMBINING table.

kitchen.text.display._print_combining_table()

Print out a new _COMBINING table

This will print a new _COMBINING table in the format used in kitchen/text/display.py. It’s useful for updating the _COMBINING table with updated data from a new python as the format won’t change from what’s already in the file.

kitchen.text.display._interval_bisearch(value, table)

Binary search in an interval table.

Parameters

  • value – numeric value to search for

  • table – Ordered list of intervals. This is a list of two-tuples. The elements of the two-tuple define an interval’s start and end points.

Returns

If value is found within an interval in the table return True. Otherwise, False

This function checks whether a numeric value is present within a table of intervals. It checks using a binary search algorithm, dividing the list of values in half and checking against the values until it determines whether the value is in the table.

kitchen.text.display._ucp_width(ucs, control_chars='guess')

Get the textual width of a ucs character

Parameters

  • ucs – integer representing a single unicode code point

  • control_chars –

    specify how to deal with control characters. Possible values are:

    guess

    (default) will take a guess for control character widths. Most codes will return zero width. backspace, delete, and clear delete return -1. escape currently returns -1 as well but this is not guaranteed as it’s not always correct

    strict

    will raise ControlCharError if a control character is encountered

Raises

ControlCharError – if the code point is a unicode control character and control_chars is set to ‘strict’

Returns

textual width of the character.

Note

It’s important to remember this is textual width and not the number of characters or bytes.

kitchen.text.display._textual_width_le(width, *args)

Optimize the common case when deciding which textual width is larger

Parameters
Returns

True if the total length of args are less than or equal to width. Otherwise False.

We often want to know “does X fit in Y”. It takes a while to use textual_width() to calculate this. However, we know that the number of canonically composed str characters is always going to have 1 or 2 for the textual width per character. With this we can take the following shortcuts:

  1. If the number of canonically composed characters is more than width, the true textual width cannot be less than width.

  2. If the number of canonically composed characters * 2 is less than the width then the textual width must be ok.

textual width of a canonically composed str string will always be greater than or equal to the the number of str characters. So we can first check if the number of composed str characters is less than the asked for width. If it is we can return True immediately. If not, then we must do a full textual width lookup.