Contributing to Python

So you have some free time to donate to Python, and don’t know where to start. Good. If you follow these cookbook instructions, you can also be a Contributor to Python™ 60 minutes from now. Most of what I’ll cover here is covered elsewhere (I’ll link), but this concentrates everything together so you really don’t have any excuses for not contributing anymore. I assume you have at least passing knowledge in Python or C, any decent editor, any version control system (like svn, git, etc), any bug-tracking system and a typical build process (./configure, make, etc; or your platform’s equivalent). You don’t have to be an expert, but if this is your first day in any of these, maybe this isn’t the right document for you.

Let’s get our gear in order. In my experience, it’s easiest to get ready to start using Ubuntu, and Ubuntu examples are what I’m going to give. If you use something else you should do your platform’s equivalent (or, uh, ‘upgrade’…). I assume you have a toolchain (compiler, linker, etc) installed, so start by making sure you have svn (sudo apt-get install subversion), then checkout one of Python’s latest development trees. Checking out is covered here, but I found that explanation a bit tiring. So, briefly: for the foreseeable future, you’re going to want to work either on trunk (Python 2.x development branch, currently 2.7) or on py3k (Python 3.x development branch, currently 3.2). The commands you need are these two:
[sourcecode]
svn co http://svn.python.org/projects/python/branches/py3k
svn co http://svn.python.org/projects/python/trunk
[/sourcecode]

Read below for stuff to do during the checkout, but as soon as Subversion will be done, you best configure and compile the source, to see that you can build Python. I assume you won’t be installing this copy of Python at all, so you can simply run ./configure && make without fiddling much with --prefix et al. By default, Python will not build some of the things you may be used to from whichever build you’ve been using so far. Fixing that isn’t mandatory for many bugs, but if you run into ‘missing features’, note two things. First, to include some modules you’ll have to edit ./Modules/Setup.dist and enable whichever modules you want. Second, Python may finish the build process successfully, but issue a notice like this towards the end of the build:
[sourcecode]
Python build finished, but the necessary bits to build these modules were not found:
_curses _curses_panel _dbm
_gdbm _sqlite3 _ssl
_tkinter bz2 readline
To find the necessary bits, look in setup.py in detect_modules() for the module’s name.
[/sourcecode]
The most likely reason is that you don’t have the development version of the listed packages installed. I made almost everything listed above disappear on my system with sudo apt-get install libreadline-dev libbz2-dev libssl-dev libsqlite3-dev libncurses-dev libgdbm-dev. Re-read the error message and the apt-get I issued, you’ll see it’s rather easy to guess the development package name from Python’s error message. Much like you’ve done during the checkout, read below for stuff to do during the configuration / build. If your internet connection is blazing and your CPU screams or you otherwise took your time in choosing a bug and the build process finished before you chose a bug, you can also run the full test suite: ./python -m test.regrtest -uall and see that nothing beyond the really esoteric breaks.

Finding a bug to work on is not hard, but it’s not trivial. Register with the Python bug-tracking site and go to the search page. For a start, I recommend to use this search: Stage: needs patch; Keyword: easy. It’s easiest to start afresh (no patch submitted with the bug), and the bugs marked ‘easy’ are easy. You will notice that not many bugs return from the search – that’s because so many easy bugs already have a patch issued to them; we’ll get back to that later. Of the returned bugs, I chose issues #6610 and #7834. #6610 reports a potential flaw in subprocess; turns out that when forking out a coprocess from a process with closed standard streams, there’s a subtlety in dup2()-ing the child’s standard streams from the pipes the parent inherited to it. #7834 reports an issue with socketmodule.c‘s creation of AF_BLUETOOTH sockets; the implementation of getsockaddrarg() is fragile as it doesn’t zero out a returned sockaddr_l2 struct which might be left with garbage.

First, make sure you understand the bug. For #6610 I read an article that was linked to the bug about the subtlety of coprocess forking for daemons, as well as the relevant code in Python (here and here, the relevant bits are near the dup2 calls). Once you understand what happens, and sometimes even if you're not sure, the best thing is to write test code that will trigger the bug. I didn’t want to touch Python’s test suite quite yet, so I jotted a small script in /tmp that will create a parent with closed stdio streamed that then fork a /bin/cat coprocess. Writing the test forced me to think harder about the bug, and to realize that… it isn’t. Bug #6610 reports a potential bug, but as it turns out, the author(s) of subprocess already protected themselves against the bug’s manifestation. My test code, assuming it was correct, proved my hypothesis – the test worked fine.

Case closed, no? No. While its possible to comment “sorry, not a bug, move along”, I wanted to nail the bug. From what I’ve seen on other bugs, nothing will convince a trusted developer to close the bug more than working coverage of the alleged bug in Python’s suite of unit test. Before touching a test, make sure it runs on your system, otherwise you might waste lots of time hunting issues. After running the subprocess test, I took my jotted script from earlier, cleaned it up and fit it in Lib/test/test_subprocess.py. Important! Python, rightfully, is rather anal about the style of code they accept into the repository (tests are code just like any other code). Save yourself lots of grief, and make certain you abide to PEP8 and PEP7. You don’t have to read them to the letter to do small fixes, but make sure you caught the spirit of what’s in there and especially that your new code fits with its old surrounding as far as style is concerned. Once I was happy with how the new test looked, I re-ran this particular test (./python -m test.regrtest test_subprocess), created an svn patch (running svn diff | tee ~/test_subprocess_with_standard_fds_closed.diff after I modified subprocess’ test was all it took) and posted the patch along with the description of what I did and found.

#7834 is simpler in most regards but much harder in one important aspect. It’s rather obvious the person who submitted the bug really ran into it in real life and probably knows more about Bluetooth sockets than I do. Fixing the bug is trivial, but writing a test for it that will run even without using actual Bluetooth hardware is not. I looked into bluez’ sources and #bluez on Freenode for help, didn’t see a managable way to do it, prepared just a patch of the fix and submitted it. I did however mention my attempt at finding a way to write a test, so whomever reviews my bug will see the complexities involved and decide for themselves whether or not this testless fix is worthy of a commit.

Ha! Two bugs clos… no. Not closed, just updated in the bug tracker. Truth is, most easy/medium bugs have some patch waiting for them, and are pending review. It is my meager understanding that if you really want to help out, reviewing some of these patches, cleaning them up, adding tests and updating documentations are probably the best things you can do for Python. Python could use an alleviation of the GIL, but if you’re an early and casual contributor, I doubt you’ll be the one to do it. What you can do, especially if you’re a student or otherwise have time and are keen to learn, is to improve as many bugs as you can into pristine condition. The less small flaws there are in a bug, the more mature it looks and it becomes easier it is for a trusted/core developer to just commit them and be done with it. You’d like someone to do it with your bugs, why not do it for someone else. Besides, soon enough you’ll be tangled enough being nosy with so many bugs that you’ll get deeper into the scene, find more interesting aspects of seemingly simple bugs, and so on and so forth, until one day you can overthrowkneel before the BDFL and be knighted to say Ni!

Happy bug hunting!