Hacking

Note

All participants must follow the pimutils Code of Conduct.

Please discuss your ideas with us, before investing a lot of time into khal (to make sure, no efforts are wasted). Also, if you have any questions on khal’s codebase, please don’t hesitate to contact us, we will gladly provide you with any information you need or set up a joined hacking session.

The preferred way of submitting patches is via github pull requests (PRs). If you are not comfortable with that, please contact us and we can work out something else. If you have something working, don’t hesitate to open a PR very early and ask for opinions.

Before we will accept your PR, we will ask you to:

  • add yourself to AUTHORS.txt if you haven’t done it before

  • add a note to CHANGELOG.rst explaining your changes (if you changed anything user facing)

  • edit the documentation (again, only if your changes impact the way users interact with khal)

  • make sure all tests pass (see below)

  • write some tests covering your patch (this really is mandatory, unless it’s in the urwid part, testing which is often difficult)

  • make sure your patch conforms with PEP 008 (should be covered by passing tests)

General notes for developing khal (and lots of other python packages)

The below notes are meant to be helpful if you are new to developing python packages in general and/or khal specifically. While some of these notes are therefore specific to khal, most should apply to lots of other python packages developed in comparable setup. Please note that all commands (if not otherwise noted) should be executed at the root of khal’s source directory, i.e., the directory you got by cloning khal via git.

Please note that fixes and enhancements to these notes are very welcome, too.

Isolation

When working on khal (or for any other python package) it has proved very beneficial to create a new virtual environments (with the help of virtualenv), to be able to run khal in isolation from your globally installed python packages and to ensure to not run into any conflicts very different python packages depend on different version of the same library. virtualenvwrapper (for bash and zsh users) and virtualfish (for fish users) are handy wrappers that make working with virtual environments very comfortable.

After you have created and activated a virtual environment, it is recommended to install khal via pip install -e . (from the base of khal’s source directory), this install khal in an editable development mode, where you do not have to reinstall khal after every change you made, but where khal will always have picked up all the latest changes (except for adding new files, hereafter reinstalling khal is necessary).

Testing

khal has an extensive self test suite, that lives in tests/. To run the test suite, install pytest and run py.test tests, pytest will then collect and run all tests and report on any failures (which you should then proceed to fix). If you only want to run tests contained in one file, run, e.g., py.test tests/backend_test.py. If you only want to run one or more specific tests, you can filter for them with py.test -k calendar, which would only run tests including calendar in their name.

To ensure that khal runs on all currently supported version of python, the self test suite should also be run with all supported versions of python. This can locally be done with tox. After installing tox, running tox will create new virtual environments (which it will reuse on later runs), one for each python version specified in tox.ini, run the test suite and report on it.

If you open a pull request (PR) on github, the continuous integration service GitHub Actions will automatically perform exactly those tasks and then comment on the success or failure.

If you make any non-trivial changes to khal, please ensure that those changes are covered by (new) tests. As testing ikhal (the part of khal making use of urwid) has proven rather complicated (as can be seen in the lack tests covering that part of khal), automated testing of changes of that part is therefore not mandatory, but very welcome nonetheless.

To make sure all major code paths are run through at least once, please check the coverage the tests provide. This can be done with pytest-cov. After installing pytest-cov, running py.test --cov khal --cov-report=html tests will generate an html-based report on test coverage (which can be found in htmlcov), including a color-coded version of khal’s source code, indicating which lines have been run and which haven’t.

Debugging

For an improved debugging experience on the command line, pdb++ is recommended (install with pip install pdbpp). pdb++ is a drop in replacement for python’s default debugger, and can therefore be used like the default debugger, e.g., invoked by placing import pdb; pdb.set_trace() at the respective place. One of the main reasons for choosing pdb++ over alternatives like IPython’s debugger ipdb, is that it works nicely with pytest, e.g., running py.test –pdb tests will drop you at a pdb++ prompt at the place of the first failing test.

Documentation

Khal’s documentation, which is living in doc, is using sphinx to generate the html documentation as well as the man page from the same sources. After install sphinx and sphinxcontrib-newsfeed you should be able to build the documentation with make html and make man respectively from the root of the doc directory (note that this requires GNU make, so on some system running gmake make be required).

If you make any changes to how a user would interact with khal, please change or add the relevant section(s) in the documentation, which uses the reStructuredText format, which shouldn’t be too hard to use after looking at some of the existing documentation (even for users who never used it before).

Also, summarize your changes in CHANGELOG.rst, pointing readers to the (updated) documentation is fine.

Code Style

khal’s source code should adhere to the rules laid out in PEP 008, except for allowing line lengths of up to 100 characters if it improves overall legibility (use your judgement). This can be checked by installing and running flake8 (run with flake8 from khal’s source directory), which will also be run with tox and GitHub Actions, see section above.

We try to document the parameters functions and methods accept, including their types, and their return values in the sphinx style, though this is currently not used thoroughly.

Note that we try to use double quotes for human readable strings, e.g., strings that one would internationalize and single quotes for strings used as identifiers, e.g., in dictionary keys:

my_event['greeting'] = "Hello World!"

iCalendar peculiarities

These notes are meant for people who want to deep dive into khal.khalendar.backend.py and are not recommended reading material for anyone else.

A single .ics can contain several VEVENTS, which might or might not be the part of the same event. This can lead to issues with straight forward implementations. Some of these, and the way khal is dealing with them, are described below.

While one would expect every VEVENT to have its own unique UID (for what it’s worth they are named unique identifier), there is a case where several VEVENTS have the same UID, but do describe the same (recurring) event. In this case, one VEVENT, containing an RRULE or RDATE element would be the proto event, from which all recurrence instances are derived. All other VEVENTS with the same UID would then have a RECURRENCE-ID element (I’ll call them child event from now on) and describe deviations of at least one recurrence instance (RECURRENCE-ID elements can also have the added property RANGE=THISANDFUTURE, meaning the deviations described by this child event also apply to all further recurrence instances.

Because it is possible that an event already in the database consists of a master event and at least one child event gets updated and then consists only of a master event, we currently delete all events with the same UID from the database when inserting or updating a new event. But this means that we need to update an event always at once (master and all child events) at the same time (using Calendar.update() or Calendar.new() in this case)

As this wouldn’t be bad enough, the standard looses no words on the ordering on those VEVENTS in any given .ics file (at least I didn’t find any). Not only can the proto event be behind any or all RECURRENCE-ID events, but also events with different UIDs can be in between.

We therefore currently first collect all events with the same UID and then sort those by their type (proto or child), and the children by the value of the RECURRENCE-ID property.