Database administration¶

A Kerberos database contains all of a realm’s Kerberos principals, their passwords, and other administrative information about each principal. For the most part, you will use the kdb5_util program to manipulate the Kerberos database as a whole, and the kadmin program to make changes to the entries in the database. (One notable exception is that users will use the kpasswd program to change their own passwords.) The kadmin program has its own command-line interface, to which you type the database administrating commands.

kdb5_util provides a means to create, delete, load, or dump a Kerberos database. It also contains commands to roll over the database master key, and to stash a copy of the key so that the kadmind and krb5kdc daemons can use the database without manual input.

kadmin provides for the maintenance of Kerberos principals, password policies, and service key tables (keytabs). Normally it operates as a network client using Kerberos authentication to communicate with kadmind, but there is also a variant, named kadmin.local, which directly accesses the Kerberos database on the local filesystem (or through LDAP). kadmin.local is necessary to set up enough of the database to be able to use the remote version.

kadmin can authenticate to the admin server using the service principal kadmin/admin or kadmin/HOST (where HOST is the hostname of the admin server). If the credentials cache contains a ticket for either service principal and the -c ccache option is specified, that ticket is used to authenticate to KADM5. Otherwise, the -p and -k options are used to specify the client Kerberos principal name used to authenticate. Once kadmin has determined the principal name, it requests a kadmin/admin Kerberos service ticket from the KDC, and uses that service ticket to authenticate to KADM5.

See kadmin for the available kadmin and kadmin.local commands and options.

kadmin options¶

You can invoke kadmin or kadmin.local with any of the following options:

kadmin [-O|-N] [-r realm] [-p principal] [-q query] [[-c cache_name]|[-k [-t keytab]]|-n] [-w password] [-s admin_server[:port]] [command args…]

kadmin.local [-r realm] [-p principal] [-q query] [-d dbname] [-e enc:salt …] [-m] [-x db_args] [command args…]

OPTIONS

-r realm: Use realm as the default database realm.
-p principal: Use principal to authenticate. Otherwise, kadmin will append /admin to the primary principal name of the default ccache, the value of the USER environment variable, or the username as obtained with getpwuid, in order of preference.
-k: Use a keytab to decrypt the KDC response instead of prompting for a password. In this case, the default principal will be host/hostname. If there is no keytab specified with the -t option, then the default keytab will be used.
-t keytab: Use keytab to decrypt the KDC response. This can only be used with the -k option.
-n: Requests anonymous processing. Two types of anonymous principals are supported. For fully anonymous Kerberos, configure PKINIT on the KDC and configure pkinit_anchors in the client’s krb5.conf. Then use the -n option with a principal of the form @REALM (an empty principal name followed by the at-sign and a realm name). If permitted by the KDC, an anonymous ticket will be returned. A second form of anonymous tickets is supported; these realm-exposed tickets hide the identity of the client but not the client’s realm. For this mode, use kinit -n with a normal principal name. If supported by the KDC, the principal (but not realm) will be replaced by the anonymous principal. As of release 1.8, the MIT Kerberos KDC only supports fully anonymous operation.
-c credentials_cache: Use credentials_cache as the credentials cache. The cache should contain a service ticket for the kadmin/admin or kadmin/ADMINHOST (where ADMINHOST is the fully-qualified hostname of the admin server) service; it can be acquired with the kinit program. If this option is not specified, kadmin requests a new service ticket from the KDC, and stores it in its own temporary ccache.
-w password: Use password instead of prompting for one. Use this option with care, as it may expose the password to other users on the system via the process list.
-q query: Perform the specified query and then exit.
-d dbname: Specifies the name of the KDC database. This option does not apply to the LDAP database module.
-s admin_server[:port]: Specifies the admin server which kadmin should contact.
-m: If using kadmin.local, prompt for the database master password instead of reading it from a stash file.
-e “enc:salt …”: Sets the keysalt list to be used for any new keys created. See Keysalt lists in kdc.conf for a list of possible values.
-O: Force use of old AUTH_GSSAPI authentication flavor.
-N: Prevent fallback to AUTH_GSSAPI authentication flavor.
-x db_args: Specifies the database specific arguments. See the next section for supported options.

Date Format¶

For the supported date-time formats see getdate time section in Supported date and time formats.

Principals¶

Each entry in the Kerberos database contains a Kerberos principal and the attributes and policies associated with that principal.

Adding, modifying and deleting principals¶

To add a principal to the database, use the kadmin add_principal command.

To modify attributes of a principal, use the kadmin modify_principal command.

To delete a principal, use the kadmin delete_principal command.

add_principal¶

add_principal [options] newprinc

Creates the principal newprinc, prompting twice for a password. If no password policy is specified with the -policy option, and the policy named default is assigned to the principal if it exists. However, creating a policy named default will not automatically assign this policy to previously existing principals. This policy assignment can be suppressed with the -clearpolicy option.

This command requires the add privilege.

Aliases: addprinc, ank

Options:

-expire expdate

(getdate time string) The expiration date of the principal.

-pwexpire pwexpdate

(getdate time string) The password expiration date.

-maxlife maxlife

(Time duration or getdate time string) The maximum ticket life for the principal.

-maxrenewlife maxrenewlife

(Time duration or getdate time string) The maximum renewable life of tickets for the principal.

-kvno kvno

The initial key version number.

-policy policy

The password policy used by this principal. If not specified, the policy default is used if it exists (unless -clearpolicy is specified).

-clearpolicy

Prevents any policy from being assigned when -policy is not specified.

{-|+}allow_postdated

-allow_postdated prohibits this principal from obtaining postdated tickets. +allow_postdated clears this flag.

{-|+}allow_forwardable

-allow_forwardable prohibits this principal from obtaining forwardable tickets. +allow_forwardable clears this flag.

{-|+}allow_renewable

-allow_renewable prohibits this principal from obtaining renewable tickets. +allow_renewable clears this flag.

{-|+}allow_proxiable

-allow_proxiable prohibits this principal from obtaining proxiable tickets. +allow_proxiable clears this flag.

{-|+}allow_dup_skey

-allow_dup_skey disables user-to-user authentication for this principal by prohibiting others from obtaining a service ticket encrypted in this principal’s TGT session key. +allow_dup_skey clears this flag.

{-|+}requires_preauth

+requires_preauth requires this principal to preauthenticate before being allowed to kinit. -requires_preauth clears this flag. When +requires_preauth is set on a service principal, the KDC will only issue service tickets for that service principal if the client’s initial authentication was performed using preauthentication.

{-|+}requires_hwauth

+requires_hwauth requires this principal to preauthenticate using a hardware device before being allowed to kinit. -requires_hwauth clears this flag. When +requires_hwauth is set on a service principal, the KDC will only issue service tickets for that service principal if the client’s initial authentication was performed using a hardware device to preauthenticate.

{-|+}ok_as_delegate

+ok_as_delegate sets the okay as delegate flag on tickets issued with this principal as the service. Clients may use this flag as a hint that credentials should be delegated when authenticating to the service. -ok_as_delegate clears this flag.

{-|+}allow_svr

-allow_svr prohibits the issuance of service tickets for this principal. In release 1.17 and later, user-to-user service tickets are still allowed unless the -allow_dup_skey flag is also set. +allow_svr clears this flag.

{-|+}allow_tgs_req

-allow_tgs_req specifies that a Ticket-Granting Service (TGS) request for a service ticket for this principal is not permitted. +allow_tgs_req clears this flag.

{-|+}allow_tix

-allow_tix forbids the issuance of any tickets for this principal. +allow_tix clears this flag.

{-|+}needchange

+needchange forces a password change on the next initial authentication to this principal. -needchange clears this flag.

{-|+}password_changing_service

+password_changing_service marks this principal as a password change service principal.

{-|+}ok_to_auth_as_delegate

+ok_to_auth_as_delegate allows this principal to acquire forwardable tickets to itself from arbitrary users, for use with constrained delegation.

{-|+}no_auth_data_required

+no_auth_data_required prevents PAC or AD-SIGNEDPATH data from being added to service tickets for the principal.

{-|+}lockdown_keys

+lockdown_keys prevents keys for this principal from leaving the KDC via kadmind. The chpass and extract operations are denied for a principal with this attribute. The chrand operation is allowed, but will not return the new keys. The delete and rename operations are also denied if this attribute is set, in order to prevent a malicious administrator from replacing principals like krbtgt/* or kadmin/* with new principals without the attribute. This attribute can be set via the network protocol, but can only be removed using kadmin.local.

-randkey

Sets the key of the principal to a random value.

-nokey

Causes the principal to be created with no key. New in release 1.12.

-pw password

Sets the password of the principal to the specified string and does not prompt for a password. Note: using this option in a shell script may expose the password to other users on the system via the process list.

-e enc:salt,…

Uses the specified keysalt list for setting the keys of the principal. See Keysalt lists in kdc.conf for a list of possible values.

-x db_princ_args

Indicates database-specific options. The options for the LDAP database module are:

-x dn=dn: Specifies the LDAP object that will contain the Kerberos principal being created.
-x linkdn=dn: Specifies the LDAP object to which the newly created Kerberos principal object will point.
-x containerdn=container_dn: Specifies the container object under which the Kerberos principal is to be created.
-x tktpolicy=policy: Associates a ticket policy to the Kerberos principal.

Note

The containerdn and linkdn options cannot be specified with the dn option.
If the dn or containerdn options are not specified while adding the principal, the principals are created under the principal container configured in the realm or the realm container.
dn and containerdn should be within the subtrees or principal container configured in the realm.

Example:

kadmin: addprinc jennifer
No policy specified for "jennifer@ATHENA.MIT.EDU";
defaulting to no policy.
Enter password for principal jennifer@ATHENA.MIT.EDU:
Re-enter password for principal jennifer@ATHENA.MIT.EDU:
Principal "jennifer@ATHENA.MIT.EDU" created.
kadmin:

modify_principal¶

modify_principal [options] principal

Modifies the specified principal, changing the fields as specified. The options to add_principal also apply to this command, except for the -randkey, -pw, and -e options. In addition, the option -clearpolicy will clear the current policy of a principal.

This command requires the modify privilege.

Alias: modprinc

Options (in addition to the addprinc options):

-unlock: Unlocks a locked principal (one which has received too many failed authentication attempts without enough time between them according to its password policy) so that it can successfully authenticate.

delete_principal¶

delete_principal [-force] principal

Deletes the specified principal from the database. This command prompts for deletion, unless the -force option is given.

This command requires the delete privilege.

Alias: delprinc

Examples¶

If you want to create a principal which is contained by a LDAP object, all you need to do is:

kadmin: addprinc -x dn=cn=jennifer,dc=example,dc=com jennifer
No policy specified for "jennifer@ATHENA.MIT.EDU";
defaulting to no policy.
Enter password for principal jennifer@ATHENA.MIT.EDU:  <= Type the password.
Re-enter password for principal jennifer@ATHENA.MIT.EDU:  <=Type it again.
Principal "jennifer@ATHENA.MIT.EDU" created.
kadmin:

If you want to create a principal under a specific LDAP container and link to an existing LDAP object, all you need to do is:

kadmin: addprinc -x containerdn=dc=example,dc=com -x linkdn=cn=david,dc=example,dc=com david
No policy specified for "david@ATHENA.MIT.EDU";
defaulting to no policy.
Enter password for principal david@ATHENA.MIT.EDU:  <= Type the password.
Re-enter password for principal david@ATHENA.MIT.EDU:  <=Type it again.
Principal "david@ATHENA.MIT.EDU" created.
kadmin:

If you want to associate a ticket policy to a principal, all you need to do is:

kadmin: modprinc -x tktpolicy=userpolicy david
Principal "david@ATHENA.MIT.EDU" modified.
kadmin:

If, on the other hand, you want to set up an account that expires on January 1, 2000, that uses a policy called “stduser”, with a temporary password (which you want the user to change immediately), you would type the following:

kadmin: addprinc david -expire "1/1/2000 12:01am EST" -policy stduser +needchange
Enter password for principal david@ATHENA.MIT.EDU:  <= Type the password.
Re-enter password for principal
david@ATHENA.MIT.EDU:  <= Type it again.
Principal "david@ATHENA.MIT.EDU" created.
kadmin:

If you want to delete a principal:

kadmin: delprinc jennifer
Are you sure you want to delete the principal
"jennifer@ATHENA.MIT.EDU"? (yes/no): yes
Principal "jennifer@ATHENA.MIT.EDU" deleted.
Make sure that you have removed this principal from
all ACLs before reusing.
kadmin:

Retrieving information about a principal¶

To retrieve a listing of the attributes and/or policies associated with a principal, use the kadmin get_principal command.

To generate a listing of principals, use the kadmin list_principals command.

get_principal¶

get_principal [-terse] principal

Gets the attributes of principal. With the -terse option, outputs fields as quoted tab-separated strings.

This command requires the inquire privilege, or that the principal running the the program to be the same as the one being listed.

Alias: getprinc

Examples:

kadmin: getprinc tlyu/admin
Principal: tlyu/admin@BLEEP.COM
Expiration date: [never]
Last password change: Mon Aug 12 14:16:47 EDT 1996
Password expiration date: [never]
Maximum ticket life: 0 days 10:00:00
Maximum renewable life: 7 days 00:00:00
Last modified: Mon Aug 12 14:16:47 EDT 1996 (bjaspan/admin@BLEEP.COM)
Last successful authentication: [never]
Last failed authentication: [never]
Failed password attempts: 0
Number of keys: 1
Key: vno 1, aes256-cts-hmac-sha384-192
MKey: vno 1
Attributes:
Policy: [none]

kadmin: getprinc -terse systest
systest@BLEEP.COM   3    86400     604800    1
785926535 753241234 785900000
tlyu/admin@BLEEP.COM     786100034 0    0
kadmin:

list_principals¶

list_principals [expression]

Retrieves all or some principal names. expression is a shell-style glob expression that can contain the wild-card characters ?, *, and []. All principal names matching the expression are printed. If no expression is provided, all principal names are printed. If the expression does not contain an @ character, an @ character followed by the local realm is appended to the expression.

This command requires the list privilege.

Alias: listprincs, get_principals, getprincs

Example:

kadmin:  listprincs test*
test3@SECURE-TEST.OV.COM
test2@SECURE-TEST.OV.COM
test1@SECURE-TEST.OV.COM
testuser@SECURE-TEST.OV.COM
kadmin:

Changing passwords¶

To change a principal’s password use the kadmin change_password command.

change_password¶

change_password [options] principal

Changes the password of principal. Prompts for a new password if neither -randkey or -pw is specified.

This command requires the changepw privilege, or that the principal running the program is the same as the principal being changed.

Alias: cpw

The following options are available:

-randkey: Sets the key of the principal to a random value.
-pw password: Set the password to the specified string. Using this option in a script may expose the password to other users on the system via the process list.
-e enc:salt,…: Uses the specified keysalt list for setting the keys of the principal. See Keysalt lists in kdc.conf for a list of possible values.
-keepold: Keeps the existing keys in the database. This flag is usually not necessary except perhaps for krbtgt principals.

Example:

kadmin: cpw systest
Enter password for principal systest@BLEEP.COM:
Re-enter password for principal systest@BLEEP.COM:
Password for systest@BLEEP.COM changed.
kadmin:

Note

Password changes through kadmin are subject to the same password policies as would apply to password changes through kpasswd.

Policies¶

A policy is a set of rules governing passwords. Policies can dictate minimum and maximum password lifetimes, minimum number of characters and character classes a password must contain, and the number of old passwords kept in the database.

Adding, modifying and deleting policies¶

To add a new policy, use the kadmin add_policy command.

To modify attributes of a principal, use the kadmin modify_policy command.

To delete a policy, use the kadmin delete_policy command.

add_policy¶

add_policy [options] policy

Adds a password policy named policy to the database.

This command requires the add privilege.

Alias: addpol

The following options are available:

-maxlife time: (Time duration or getdate time string) Sets the maximum lifetime of a password.
-minlife time: (Time duration or getdate time string) Sets the minimum lifetime of a password.
-minlength length: Sets the minimum length of a password.
-minclasses number: Sets the minimum number of character classes required in a password. The five character classes are lower case, upper case, numbers, punctuation, and whitespace/unprintable characters.
-history number: Sets the number of past keys kept for a principal. This option is not supported with the LDAP KDC database module.

-maxfailure maxnumber: Sets the number of authentication failures before the principal is locked. Authentication failures are only tracked for principals which require preauthentication. The counter of failed attempts resets to 0 after a successful attempt to authenticate. A maxnumber value of 0 (the default) disables lockout.

-failurecountinterval failuretime: (Time duration or getdate time string) Sets the allowable time between authentication failures. If an authentication failure happens after failuretime has elapsed since the previous failure, the number of authentication failures is reset to 1. A failuretime value of 0 (the default) means forever.

-lockoutduration lockouttime: (Time duration or getdate time string) Sets the duration for which the principal is locked from authenticating if too many authentication failures occur without the specified failure count interval elapsing. A duration of 0 (the default) means the principal remains locked out until it is administratively unlocked with modprinc -unlock.
-allowedkeysalts: Specifies the key/salt tuples supported for long-term keys when setting or changing a principal’s password/keys. See Keysalt lists in kdc.conf for a list of the accepted values, but note that key/salt tuples must be separated with commas (‘,’) only. To clear the allowed key/salt policy use a value of ‘-‘.

Example:

kadmin: add_policy -maxlife "2 days" -minlength 5 guests
kadmin:

modify_policy¶

modify_policy [options] policy

Modifies the password policy named policy. Options are as described for add_policy.

This command requires the modify privilege.

Alias: modpol

delete_policy¶

delete_policy [-force] policy

Deletes the password policy named policy. Prompts for confirmation before deletion. The command will fail if the policy is in use by any principals.

This command requires the delete privilege.

Alias: delpol

Example:

kadmin: del_policy guests
Are you sure you want to delete the policy "guests"?
(yes/no): yes
kadmin:

Note

You must cancel the policy from all principals before deleting it. The delete_policy command will fail if the policy is in use by any principals.

Retrieving policies¶

To retrieve a policy, use the kadmin get_policy command.

You can retrieve the list of policies with the kadmin list_policies command.

get_policy¶

get_policy [ -terse ] policy

Displays the values of the password policy named policy. With the -terse flag, outputs the fields as quoted strings separated by tabs.

This command requires the inquire privilege.

Alias: getpol

Examples:

kadmin: get_policy admin
Policy: admin
Maximum password life: 180 days 00:00:00
Minimum password life: 00:00:00
Minimum password length: 6
Minimum number of password character classes: 2
Number of old keys kept: 5
Reference count: 17

kadmin: get_policy -terse admin
admin     15552000  0    6    2    5    17
kadmin:

The “Reference count” is the number of principals using that policy. With the LDAP KDC database module, the reference count field is not meaningful.

list_policies¶

list_policies [expression]

Retrieves all or some policy names. expression is a shell-style glob expression that can contain the wild-card characters ?, *, and []. All policy names matching the expression are printed. If no expression is provided, all existing policy names are printed.

This command requires the list privilege.

Aliases: listpols, get_policies, getpols.

Examples:

kadmin:  listpols
test-pol
dict-only
once-a-min
test-pol-nopw

kadmin:  listpols t*
test-pol
test-pol-nopw
kadmin:

Policies and principals¶

Policies can be applied to principals as they are created by using the -policy flag to add_principal. Existing principals can be modified by using the -policy or -clearpolicy flag to modify_principal.

Updating the history key¶

If a policy specifies a number of old keys kept of two or more, the stored old keys are encrypted in a history key, which is found in the key data of the kadmin/history principal.

Currently there is no support for proper rollover of the history key, but you can change the history key (for example, to use a better encryption type) at the cost of invalidating currently stored old keys. To change the history key, run:

kadmin: change_password -randkey kadmin/history

This command will fail if you specify the -keepold flag. Only one new history key will be created, even if you specify multiple key/salt combinations.

In the future, we plan to migrate towards encrypting old keys in the master key instead of the history key, and implementing proper rollover support for stored old keys.

Privileges¶

Administrative privileges for the Kerberos database are stored in the file kadm5.acl.

Note

A common use of an admin instance is so you can grant separate permissions (such as administrator access to the Kerberos database) to a separate Kerberos principal. For example, the user joeadmin might have a principal for his administrative use, called joeadmin/admin. This way, joeadmin would obtain joeadmin/admin tickets only when he actually needs to use those permissions.

Operations on the Kerberos database¶

The kdb5_util command is the primary tool for administrating the Kerberos database.

kdb5_util [-r realm] [-d dbname] [-k mkeytype] [-kv mkeyVNO] [-M mkeyname] [-m] [-sf stashfilename] [-P password] [-x db_args] command [command_options]

OPTIONS

-r realm: specifies the Kerberos realm of the database.
-d dbname: specifies the name under which the principal database is stored; by default the database is that listed in kdc.conf. The password policy database and lock files are also derived from this value.
-k mkeytype: specifies the key type of the master key in the database. The default is given by the master_key_type variable in kdc.conf.
-kv mkeyVNO: Specifies the version number of the master key in the database; the default is 1. Note that 0 is not allowed.
-M mkeyname: principal name for the master key in the database. If not specified, the name is determined by the master_key_name variable in kdc.conf.
-m: specifies that the master database password should be read from the keyboard rather than fetched from a file on disk.
-sf stash_file: specifies the stash filename of the master database password. If not specified, the filename is determined by the key_stash_file variable in kdc.conf.
-P password: specifies the master database password. Using this option may expose the password to other users on the system via the process list.
-x db_args: specifies database-specific options. See kadmin for supported options.

Dumping a Kerberos database to a file¶

To dump a Kerberos database into a file, use the kdb5_util dump command on one of the KDCs.

dump [-b7|-r13|-r18] [-verbose] [-mkey_convert] [-new_mkey_file mkey_file] [-rev] [-recurse] [filename [principals…]]

Dumps the current Kerberos and KADM5 database into an ASCII file. By default, the database is dumped in current format, “kdb5_util load_dump version 7”. If filename is not specified, or is the string “-”, the dump is sent to standard output. Options:

-b7: causes the dump to be in the Kerberos 5 Beta 7 format (“kdb5_util load_dump version 4”). This was the dump format produced on releases prior to 1.2.2.
-r13: causes the dump to be in the Kerberos 5 1.3 format (“kdb5_util load_dump version 5”). This was the dump format produced on releases prior to 1.8.
-r18: causes the dump to be in the Kerberos 5 1.8 format (“kdb5_util load_dump version 6”). This was the dump format produced on releases prior to 1.11.
-verbose: causes the name of each principal and policy to be printed as it is dumped.
-mkey_convert: prompts for a new master key. This new master key will be used to re-encrypt principal key data in the dumpfile. The principal keys themselves will not be changed.
-new_mkey_file mkey_file: the filename of a stash file. The master key in this stash file will be used to re-encrypt the key data in the dumpfile. The key data in the database will not be changed.
-rev: dumps in reverse order. This may recover principals that do not dump normally, in cases where database corruption has occurred.
-recurse: causes the dump to walk the database recursively (btree only). This may recover principals that do not dump normally, in cases where database corruption has occurred. In cases of such corruption, this option will probably retrieve more principals than the -rev option will.

Changed in version 1.15: Release 1.15 restored the functionality of the -recurse option.

Changed in version 1.5: The -recurse option ceased working until release 1.15, doing a normal dump instead of a recursive traversal.

Examples¶

shell% kdb5_util dump dumpfile
shell%

shell% kbd5_util dump -verbose dumpfile
kadmin/admin@ATHENA.MIT.EDU
krbtgt/ATHENA.MIT.EDU@ATHENA.MIT.EDU
kadmin/history@ATHENA.MIT.EDU
K/M@ATHENA.MIT.EDU
kadmin/changepw@ATHENA.MIT.EDU
shell%

If you specify which principals to dump, you must use the full principal, as in the following example:

shell% kdb5_util dump -verbose dumpfile K/M@ATHENA.MIT.EDU kadmin/admin@ATHENA.MIT.EDU
kadmin/admin@ATHENA.MIT.EDU
K/M@ATHENA.MIT.EDU
shell%

Otherwise, the principals will not match those in the database and will not be dumped:

shell% kdb5_util dump -verbose dumpfile K/M kadmin/admin
shell%

If you do not specify a dump file, kdb5_util will dump the database to the standard output.

Restoring a Kerberos database from a dump file¶

To restore a Kerberos database dump from a file, use the kdb5_util load command on one of the KDCs.

load [-b7|-r13|-r18] [-hash] [-verbose] [-update] filename

Loads a database dump from the named file into the named database. If no option is given to determine the format of the dump file, the format is detected automatically and handled as appropriate. Unless the -update option is given, load creates a new database containing only the data in the dump file, overwriting the contents of any previously existing database. Note that when using the LDAP KDC database module, the -update flag is required.

Options:

-b7: requires the database to be in the Kerberos 5 Beta 7 format (“kdb5_util load_dump version 4”). This was the dump format produced on releases prior to 1.2.2.
-r13: requires the database to be in Kerberos 5 1.3 format (“kdb5_util load_dump version 5”). This was the dump format produced on releases prior to 1.8.
-r18: requires the database to be in Kerberos 5 1.8 format (“kdb5_util load_dump version 6”). This was the dump format produced on releases prior to 1.11.
-hash: stores the database in hash format, if using the DB2 database type. If this option is not specified, the database will be stored in btree format. This option is not recommended, as databases stored in hash format are known to corrupt data and lose principals.
-verbose: causes the name of each principal and policy to be printed as it is dumped.
-update: records from the dump file are added to or updated in the existing database. Otherwise, a new database is created containing only what is in the dump file and the old one destroyed upon successful completion.

Examples¶

To dump a single principal and later load it, updating the database:

shell% kdb5_util dump dumpfile principal@REALM
shell%

shell% kdb5_util load -update dumpfile
shell%

Note

If the database file exists, and the -update flag was not given, kdb5_util will overwrite the existing database.

Note

Using kdb5_util to dump and reload the principal database is only necessary when upgrading from versions of krb5 prior to 1.2.0—newer versions will use the existing database as-is.

Creating a stash file¶

A stash file allows a KDC to authenticate itself to the database utilities, such as kadmind, krb5kdc, and kdb5_util.

To create a stash file, use the kdb5_util stash command.

stash [-f keyfile]

Stores the master principal’s keys in a stash file. The -f argument can be used to override the keyfile specified in kdc.conf.

Example¶

shell% kdb5_util stash kdb5_util: Cannot find/read stored master key while reading master key kdb5_util: Warning: proceeding without master key Enter KDC database master key: <= Type the KDC database master password. shell%

If you do not specify a stash file, kdb5_util will stash the key in the file specified in your kdc.conf file.

Creating and destroying a Kerberos database¶

If you need to create a new Kerberos database, use the kdb5_util create command.

create [-s]

Creates a new database. If the -s option is specified, the stash file is also created. This command fails if the database already exists. If the command is successful, the database is opened just as if it had already existed when the program was first run.

If you need to destroy the current Kerberos database, use the kdb5_util destroy command.

destroy [-f]

Destroys the database, first overwriting the disk sectors and then unlinking the files, after prompting the user for confirmation. With the -f argument, does not prompt the user.

Examples¶

shell% kdb5_util -r ATHENA.MIT.EDU create -s
Loading random data
Initializing database '/usr/local/var/krb5kdc/principal' for realm 'ATHENA.MIT.EDU',
master key name 'K/M@ATHENA.MIT.EDU'
You will be prompted for the database Master Password.
It is important that you NOT FORGET this password.
Enter KDC database master key:  <= Type the master password.
Re-enter KDC database master key to verify:  <= Type it again.
shell%

shell% kdb5_util -r ATHENA.MIT.EDU destroy
Deleting KDC database stored in '/usr/local/var/krb5kdc/principal', are you sure?
(type 'yes' to confirm)?  <= yes
OK, deleting database '/usr/local/var/krb5kdc/principal'...
** Database '/usr/local/var/krb5kdc/principal' destroyed.
shell%

Updating the master key¶

Starting with release 1.7, kdb5_util allows the master key to be changed using a rollover process, with minimal loss of availability. To roll over the master key, follow these steps:

On the primary KDC, run kdb5_util list_mkeys to view the current master key version number (KVNO). If you have never rolled over the master key before, this will likely be version 1:
```
$ kdb5_util list_mkeys
Master keys for Principal: K/M@KRBTEST.COM
KVNO: 1, Enctype: aes256-cts-hmac-sha384-192, Active on: Thu Jan 01 00:00:00 UTC 1970 *
```
On the primary KDC, run kdb5_util use_mkey 1 to ensure that a master key activation list is present in the database. This step is unnecessary in release 1.11.4 or later, or if the database was initially created with release 1.7 or later.
On the primary KDC, run kdb5_util add_mkey -s to create a new master key and write it to the stash file. Enter a secure password when prompted. If this is the first time you are changing the master key, the new key will have version 2. The new master key will not be used until you make it active.
Propagate the database to all replica KDCs, either manually or by waiting until the next scheduled propagation. If you do not have any replica KDCs, you can skip this and the next step.
On each replica KDC, run kdb5_util list_mkeys to verify that the new master key is present, and then kdb5_util stash to write the new master key to the replica KDC’s stash file.
On the primary KDC, run kdb5_util use_mkey 2 to begin using the new master key. Replace 2 with the version of the new master key, as appropriate. You can optionally specify a date for the new master key to become active; by default, it will become active immediately. Prior to release 1.12, kadmind must be restarted for this change to take full effect.
On the primary KDC, run kdb5_util update_princ_encryption. This command will iterate over the database and re-encrypt all keys in the new master key. If the database is large and uses DB2, the primary KDC will become unavailable while this command runs, but clients should fail over to replica KDCs (if any are present) during this time period. In release 1.13 and later, you can instead run kdb5_util -x unlockiter update_princ_encryption to use unlocked iteration; this variant will take longer, but will keep the database available to the KDC and kadmind while it runs.
Wait until the above changes have propagated to all replica KDCs and until all running KDC and kadmind processes have serviced requests using updated principal entries.
On the primary KDC, run kdb5_util purge_mkeys to clean up the old master key.

Operations on the LDAP database¶

The kdb5_ldap_util is the primary tool for administrating the Kerberos LDAP database. It allows an administrator to manage realms, Kerberos services (KDC and Admin Server) and ticket policies.

kdb5_ldap_util [-D user_dn [-w passwd]] [-H ldapuri] command [command_options]

OPTIONS

-r realm: Specifies the realm to be operated on.
-D user_dn: Specifies the Distinguished Name (DN) of the user who has sufficient rights to perform the operation on the LDAP server.
-w passwd: Specifies the password of user_dn. This option is not recommended.
-H ldapuri: Specifies the URI of the LDAP server.

By default, kdb5_ldap_util operates on the default realm (as specified in krb5.conf) and connects and authenticates to the LDAP server in the same manner as :ref:kadmind(8)` would given the parameters in [dbdefaults] in kdc.conf.

Creating a Kerberos realm¶

If you need to create a new realm, use the kdb5_ldap_util create command as follows.

create [-subtrees subtree_dn_list] [-sscope search_scope] [-containerref container_reference_dn] [-k mkeytype] [-kv mkeyVNO] [-M mkeyname] [-m|-P password|-sf stashfilename] [-s] [-maxtktlife max_ticket_life] [-maxrenewlife max_renewable_ticket_life] [ticket_flags]

Creates realm in directory. Options:

-subtrees subtree_dn_list: Specifies the list of subtrees containing the principals of a realm. The list contains the DNs of the subtree objects separated by colon (:).
-sscope search_scope: Specifies the scope for searching the principals under the subtree. The possible values are 1 or one (one level), 2 or sub (subtrees).
-containerref container_reference_dn: Specifies the DN of the container object in which the principals of a realm will be created. If the container reference is not configured for a realm, the principals will be created in the realm container.
-k mkeytype: Specifies the key type of the master key in the database. The default is given by the master_key_type variable in kdc.conf.
-kv mkeyVNO: Specifies the version number of the master key in the database; the default is 1. Note that 0 is not allowed.
-M mkeyname: Specifies the principal name for the master key in the database. If not specified, the name is determined by the master_key_name variable in kdc.conf.
-m: Specifies that the master database password should be read from the TTY rather than fetched from a file on the disk.
-P password: Specifies the master database password. This option is not recommended.
-sf stashfilename: Specifies the stash file of the master database password.
-s: Specifies that the stash file is to be created.
-maxtktlife max_ticket_life: (getdate time string) Specifies maximum ticket life for principals in this realm.
-maxrenewlife max_renewable_ticket_life: (getdate time string) Specifies maximum renewable life of tickets for principals in this realm.
ticket_flags: Specifies global ticket flags for the realm. Allowable flags are documented in the description of the add_principal command in kadmin.

Example:

kdb5_ldap_util -D cn=admin,o=org -H ldaps://ldap-server1.mit.edu
    -r ATHENA.MIT.EDU create -subtrees o=org -sscope SUB
Password for "cn=admin,o=org":
Initializing database for realm 'ATHENA.MIT.EDU'
You will be prompted for the database Master Password.
It is important that you NOT FORGET this password.
Enter KDC database master key:
Re-enter KDC database master key to verify:

Modifying a Kerberos realm¶

If you need to modify a realm, use the kdb5_ldap_util modify command as follows.

modify [-subtrees subtree_dn_list] [-sscope search_scope] [-containerref container_reference_dn] [-maxtktlife max_ticket_life] [-maxrenewlife max_renewable_ticket_life] [ticket_flags]

Modifies the attributes of a realm. Options:

-subtrees subtree_dn_list: Specifies the list of subtrees containing the principals of a realm. The list contains the DNs of the subtree objects separated by colon (:). This list replaces the existing list.
-sscope search_scope: Specifies the scope for searching the principals under the subtrees. The possible values are 1 or one (one level), 2 or sub (subtrees).
-containerref container_reference_dn Specifies the DN of the: container object in which the principals of a realm will be created.
-maxtktlife max_ticket_life: (getdate time string) Specifies maximum ticket life for principals in this realm.
-maxrenewlife max_renewable_ticket_life: (getdate time string) Specifies maximum renewable life of tickets for principals in this realm.
ticket_flags: Specifies global ticket flags for the realm. Allowable flags are documented in the description of the add_principal command in kadmin.

Example:

shell% kdb5_ldap_util -r ATHENA.MIT.EDU -D cn=admin,o=org -H
    ldaps://ldap-server1.mit.edu modify +requires_preauth
Password for "cn=admin,o=org":
shell%

Destroying a Kerberos realm¶

If you need to destroy a Kerberos realm, use the kdb5_ldap_util destroy command as follows.

destroy [-f]

Destroys an existing realm. Options:

-f: If specified, will not prompt the user for confirmation.

Example:

shell% kdb5_ldap_util -r ATHENA.MIT.EDU -D cn=admin,o=org -H
    ldaps://ldap-server1.mit.edu destroy
Password for "cn=admin,o=org":
Deleting KDC database of 'ATHENA.MIT.EDU', are you sure?
(type 'yes' to confirm)? yes
OK, deleting database of 'ATHENA.MIT.EDU'...
shell%

Retrieving information about a Kerberos realm¶

If you need to display the attributes of a realm, use the kdb5_ldap_util view command as follows.

view

Displays the attributes of a realm.

Example:

kdb5_ldap_util -D cn=admin,o=org -H ldaps://ldap-server1.mit.edu
    -r ATHENA.MIT.EDU view
Password for "cn=admin,o=org":
Realm Name: ATHENA.MIT.EDU
Subtree: ou=users,o=org
Subtree: ou=servers,o=org
SearchScope: ONE
Maximum ticket life: 0 days 01:00:00
Maximum renewable life: 0 days 10:00:00
Ticket flags: DISALLOW_FORWARDABLE REQUIRES_PWCHANGE

Listing available Kerberos realms¶

If you need to display the list of the realms, use the kdb5_ldap_util list command as follows.

list

Lists the names of realms under the container.

Example:

shell% kdb5_ldap_util -D cn=admin,o=org -H
    ldaps://ldap-server1.mit.edu list
Password for "cn=admin,o=org":
ATHENA.MIT.EDU
OPENLDAP.MIT.EDU
MEDIA-LAB.MIT.EDU
shell%

Stashing service object’s password¶

The kdb5_ldap_util stashsrvpw command allows an administrator to store the password of service object in a file. The KDC and Administration server uses this password to authenticate to the LDAP server.

stashsrvpw [-f filename] name

Allows an administrator to store the password for service object in a file so that KDC and Administration server can use it to authenticate to the LDAP server. Options:

-f filename: Specifies the complete path of the service password file. By default, /usr/local/var/service_passwd is used.
name: Specifies the name of the object whose password is to be stored. If krb5kdc or kadmind are configured for simple binding, this should be the distinguished name it will use as given by the ldap_kdc_dn or ldap_kadmind_dn variable in kdc.conf. If the KDC or kadmind is configured for SASL binding, this should be the authentication name it will use as given by the ldap_kdc_sasl_authcid or ldap_kadmind_sasl_authcid variable.

Example:

kdb5_ldap_util stashsrvpw -f /home/andrew/conf_keyfile
    cn=service-kdc,o=org
Password for "cn=service-kdc,o=org":
Re-enter password for "cn=service-kdc,o=org":

Ticket Policy operations¶

Creating a Ticket Policy¶

To create a new ticket policy in directory , use the kdb5_ldap_util create_policy command. Ticket policy objects are created under the realm container.

create_policy [-maxtktlife max_ticket_life] [-maxrenewlife max_renewable_ticket_life] [ticket_flags] policy_name

Creates a ticket policy in the directory. Options:

-maxtktlife max_ticket_life: (getdate time string) Specifies maximum ticket life for principals.
-maxrenewlife max_renewable_ticket_life: (getdate time string) Specifies maximum renewable life of tickets for principals.
ticket_flags: Specifies the ticket flags. If this option is not specified, by default, no restriction will be set by the policy. Allowable flags are documented in the description of the add_principal command in kadmin.
policy_name: Specifies the name of the ticket policy.

Example:

kdb5_ldap_util -D cn=admin,o=org -H ldaps://ldap-server1.mit.edu
    -r ATHENA.MIT.EDU create_policy -maxtktlife "1 day"
    -maxrenewlife "1 week" -allow_postdated +needchange
    -allow_forwardable tktpolicy
Password for "cn=admin,o=org":

Modifying a Ticket Policy¶

To modify a ticket policy in directory, use the kdb5_ldap_util modify_policy command.

modify_policy [-maxtktlife max_ticket_life] [-maxrenewlife max_renewable_ticket_life] [ticket_flags] policy_name

Modifies the attributes of a ticket policy. Options are same as for create_policy.

Example:

kdb5_ldap_util -D cn=admin,o=org -H
    ldaps://ldap-server1.mit.edu -r ATHENA.MIT.EDU modify_policy
    -maxtktlife "60 minutes" -maxrenewlife "10 hours"
    +allow_postdated -requires_preauth tktpolicy
Password for "cn=admin,o=org":

Retrieving Information About a Ticket Policy¶

To display the attributes of a ticket policy, use the kdb5_ldap_util view_policy command.

view_policy policy_name

Displays the attributes of the named ticket policy.

Example:

kdb5_ldap_util -D cn=admin,o=org -H ldaps://ldap-server1.mit.edu
    -r ATHENA.MIT.EDU view_policy tktpolicy
Password for "cn=admin,o=org":
Ticket policy: tktpolicy
Maximum ticket life: 0 days 01:00:00
Maximum renewable life: 0 days 10:00:00
Ticket flags: DISALLOW_FORWARDABLE REQUIRES_PWCHANGE

Destroying a Ticket Policy¶

To destroy an existing ticket policy, use the kdb5_ldap_util destroy_policy command.

destroy_policy [-force] policy_name

Destroys an existing ticket policy. Options:

-force: Forces the deletion of the policy object. If not specified, the user will be prompted for confirmation before deleting the policy.
policy_name: Specifies the name of the ticket policy.

Example:

kdb5_ldap_util -D cn=admin,o=org -H ldaps://ldap-server1.mit.edu
    -r ATHENA.MIT.EDU destroy_policy tktpolicy
Password for "cn=admin,o=org":
This will delete the policy object 'tktpolicy', are you sure?
(type 'yes' to confirm)? yes
** policy object 'tktpolicy' deleted.

Listing available Ticket Policies¶

To list the name of ticket policies in a realm, use the kdb5_ldap_util list_policy command.

list_policy

Lists ticket policies.

Example:

kdb5_ldap_util -D cn=admin,o=org -H ldaps://ldap-server1.mit.edu
    -r ATHENA.MIT.EDU list_policy
Password for "cn=admin,o=org":
tktpolicy
tmppolicy
userpolicy

Cross-realm authentication¶

In order for a KDC in one realm to authenticate Kerberos users in a different realm, it must share a key with the KDC in the other realm. In both databases, there must be krbtgt service principals for both realms. For example, if you need to do cross-realm authentication between the realms ATHENA.MIT.EDU and EXAMPLE.COM, you would need to add the principals krbtgt/EXAMPLE.COM@ATHENA.MIT.EDU and krbtgt/ATHENA.MIT.EDU@EXAMPLE.COM to both databases. These principals must all have the same passwords, key version numbers, and encryption types; this may require explicitly setting the key version number with the -kvno option.

In the ATHENA.MIT.EDU and EXAMPLE.COM cross-realm case, the administrators would run the following commands on the KDCs in both realms:

shell%: kadmin.local -e "aes256-cts:normal"
kadmin: addprinc -requires_preauth krbtgt/ATHENA.MIT.EDU@EXAMPLE.COM
Enter password for principal krbtgt/ATHENA.MIT.EDU@EXAMPLE.COM:
Re-enter password for principal krbtgt/ATHENA.MIT.EDU@EXAMPLE.COM:
kadmin: addprinc -requires_preauth krbtgt/EXAMPLE.COM@ATHENA.MIT.EDU
Enter password for principal krbtgt/EXAMPLE.COM@ATHENA.MIT.EDU:
Enter password for principal krbtgt/EXAMPLE.COM@ATHENA.MIT.EDU:
kadmin:

Note

Even if most principals in a realm are generally created with the requires_preauth flag enabled, this flag is not desirable on cross-realm authentication keys because doing so makes it impossible to disable preauthentication on a service-by-service basis. Disabling it as in the example above is recommended.

Note

It is very important that these principals have good passwords. MIT recommends that TGT principal passwords be at least 26 characters of random ASCII text.

Changing the krbtgt key¶

A Kerberos Ticket Granting Ticket (TGT) is a service ticket for the principal krbtgt/REALM. The key for this principal is created when the Kerberos database is initialized and need not be changed. However, it will only have the encryption types supported by the KDC at the time of the initial database creation. To allow use of newer encryption types for the TGT, this key has to be changed.

Changing this key using the normal kadmin change_password command would invalidate any previously issued TGTs. Therefore, when changing this key, normally one should use the -keepold flag to change_password to retain the previous key in the database as well as the new key. For example:

kadmin: change_password -randkey -keepold krbtgt/ATHENA.MIT.EDU@ATHENA.MIT.EDU

Warning

After issuing this command, the old key is still valid and is still vulnerable to (for instance) brute force attacks. To completely retire an old key or encryption type, run the kadmin purgekeys command to delete keys with older kvnos, ideally first making sure that all tickets issued with the old keys have expired.

Only the first krbtgt key of the newest key version is used to encrypt ticket-granting tickets. However, the set of encryption types present in the krbtgt keys is used by default to determine the session key types supported by the krbtgt service (see Session key selection). Because non-MIT Kerberos clients sometimes send a limited set of encryption types when making AS requests, it can be important for the krbtgt service to support multiple encryption types. This can be accomplished by giving the krbtgt principal multiple keys, which is usually as simple as not specifying any -e option when changing the krbtgt key, or by setting the session_enctypes string attribute on the krbtgt principal (see set_string).

Due to a bug in releases 1.8 through 1.13, renewed and forwarded tickets may not work if the original ticket was obtained prior to a krbtgt key change and the modified ticket is obtained afterwards. Upgrading the KDC to release 1.14 or later will correct this bug.

Incremental database propagation¶

Overview¶

At some very large sites, dumping and transmitting the database can take more time than is desirable for changes to propagate from the primary KDC to the replica KDCs. The incremental propagation support added in the 1.7 release is intended to address this.

With incremental propagation enabled, all programs on the primary KDC that change the database also write information about the changes to an “update log” file, maintained as a circular buffer of a certain size. A process on each replica KDC connects to a service on the primary KDC (currently implemented in the kadmind server) and periodically requests the changes that have been made since the last check. By default, this check is done every two minutes.

Incremental propagation uses the following entries in the per-realm data in the KDC config file (See kdc.conf):

iprop_enable	boolean	If true, then incremental propagation is enabled, and (as noted below) normal kprop propagation is disabled. The default is false.
iprop_master_ulogsize	integer	Indicates the number of entries that should be retained in the update log. The default is 1000; the maximum number is 2500.
iprop_replica_poll	time interval	Indicates how often the replica should poll the primary KDC for changes to the database. The default is two minutes.
iprop_port	integer	Specifies the port number to be used for incremental propagation. This is required in both primary and replica configuration files.
iprop_resync_timeout	integer	Specifies the number of seconds to wait for a full propagation to complete. This is optional on replica configurations. Defaults to 300 seconds (5 minutes).
iprop_logfile	file name	Specifies where the update log file for the realm database is to be stored. The default is to use the database_name entry from the realms section of the config file kdc.conf, with .ulog appended. (NOTE: If database_name isn’t specified in the realms section, perhaps because the LDAP database back end is being used, or the file name is specified in the dbmodules section, then the hard-coded default for database_name is used. Determination of the iprop_logfile default value will not use values from the dbmodules section.)

Both primary and replica sides must have a principal named kiprop/hostname (where hostname is the lowercase, fully-qualified, canonical name for the host) registered in the Kerberos database, and have keys for that principal stored in the default keytab file (FILE:/etc/krb5.keytab). The kiprop/hostname principal may have been created automatically for the primary KDC, but it must always be created for replica KDCs.

On the primary KDC side, the kiprop/hostname principal must be listed in the kadmind ACL file kadm5.acl, and given the p privilege (see Privileges).

On the replica KDC side, kpropd should be run. When incremental propagation is enabled, it will connect to the kadmind on the primary KDC and start requesting updates.

The normal kprop mechanism is disabled by the incremental propagation support. However, if the replica has been unable to fetch changes from the primary KDC for too long (network problems, perhaps), the log on the primary may wrap around and overwrite some of the updates that the replica has not yet retrieved. In this case, the replica will instruct the primary KDC to dump the current database out to a file and invoke a one-time kprop propagation, with special options to also convey the point in the update log at which the replica should resume fetching incremental updates. Thus, all the keytab and ACL setup previously described for kprop propagation is still needed.

If an environment has a large number of replicas, it may be desirable to arrange them in a hierarchy instead of having the primary serve updates to every replica. To do this, run kadmind -proponly on each intermediate replica, and kpropd -A upstreamhostname on downstream replicas to direct each one to the appropriate upstream replica.

There are several known restrictions in the current implementation:

The incremental update protocol does not transport changes to policy objects. Any policy changes on the primary will result in full resyncs to all replicas.
The replica’s KDB module must support locking; it cannot be using the LDAP KDB module.
The primary and replica must be able to initiate TCP connections in both directions, without an intervening NAT.

Sun/MIT incremental propagation differences¶

Sun donated the original code for supporting incremental database propagation to MIT. Some changes have been made in the MIT source tree that will be visible to administrators. (These notes are based on Sun’s patches. Changes to Sun’s implementation since then may not be reflected here.)

The Sun config file support looks for sunw_dbprop_enable, sunw_dbprop_master_ulogsize, and sunw_dbprop_slave_poll.

The incremental propagation service is implemented as an ONC RPC service. In the Sun implementation, the service is registered with rpcbind (also known as portmapper) and the client looks up the port number to contact. In the MIT implementation, where interaction with some modern versions of rpcbind doesn’t always work well, the port number must be specified in the config file on both the primary and replica sides.

The Sun implementation hard-codes pathnames in /var/krb5 for the update log and the per-replica kprop dump files. In the MIT implementation, the pathname for the update log is specified in the config file, and the per-replica dump files are stored in /etc/krb5kdc/replica_datatrans_hostname.

MIT Kerberos Documentation

Database administration¶

kadmin options¶

Date Format¶

Principals¶

Adding, modifying and deleting principals¶

add_principal¶

modify_principal¶

delete_principal¶

Examples¶

Retrieving information about a principal¶

get_principal¶

list_principals¶

Changing passwords¶

change_password¶

Policies¶

Adding, modifying and deleting policies¶

add_policy¶

modify_policy¶

delete_policy¶

Retrieving policies¶

get_policy¶

list_policies¶

Policies and principals¶

Updating the history key¶

Privileges¶

Operations on the Kerberos database¶

Dumping a Kerberos database to a file¶

Examples¶

Restoring a Kerberos database from a dump file¶

Examples¶

Creating a stash file¶

Example¶

Creating and destroying a Kerberos database¶

Examples¶

Updating the master key¶

Operations on the LDAP database¶

Creating a Kerberos realm¶

Modifying a Kerberos realm¶

Destroying a Kerberos realm¶

Retrieving information about a Kerberos realm¶

Listing available Kerberos realms¶

Stashing service object’s password¶

Ticket Policy operations¶

Creating a Ticket Policy¶

Modifying a Ticket Policy¶

Retrieving Information About a Ticket Policy¶

Destroying a Ticket Policy¶

Listing available Ticket Policies¶

Cross-realm authentication¶

Changing the krbtgt key¶

Incremental database propagation¶

Overview¶

Sun/MIT incremental propagation differences¶

On this page

Table of contents

Full Table of Contents

Search