Correction for ‘Python’s Innards: pystate’

2010/06/05 § Leave a comment

Graham Dumpleton (of mod_wsgi fame) pointed out a glaring omission and subtle inaccuracy in my post about Python’s state structures.

When discussing what I called “Pythonic threads”, which are threads created and managed by Python, which have a PyThreadState structure allocated to them and that are able to call into the Python API and run Python code, I said: “theoretically threads can be created which will not be under the interpreter’s control; these threads won’t have a PyThreadState structure and must never call a Python API; this is not very common“. This is sort of correct, but misleading: such threads are quite obvious to have in a non-Python applications that has a Python interpreter embedded in it (something Graham probably knows a thing or two about :). These ‘foreign’ (I don’t know of formal term) threads indeed can’t call the Python API and run Python code unless they’re ‘migrated’ (don’t know of a formal term) to come under the control of a Python interpreter, having the correct bookkeeping structures allocated and initialized for them.

Graham also noted correctly that such ‘migration’ is easier when there’s just one Python interpreter in the process, and harder if there’s more than one. If you have just one interpreter, you can use the (somewhat confusingly named) PyGILState_Ensure and PyGILState_Release calls to do the job. These calls allow a thread to ensure it has the GIL (blocking if necessary until it can be acquired), but importantly can be called by a thread that isn’t under Python’s control at all; the necessary work to allow the thread to call Python’s API will be done for you. These calls assume there is only one Python interpreter (interp_head points to a PyInterpreterState which points to NULL), hence they mustn’t be used in a multi-interpreter process, if that is your case you’ll have to ‘migrate’ the threat manually with lower level primitives.

As per my errata policy, I’ve chosen to update the original post as if the error never happened and write this post note the mistake for future reference and for those subscribed to the RSS feed of the Errata category.

I’d like to thank Graham for the important correction.


Python’s Innards: Series Renamed

2010/05/22 § Leave a comment

Just yesterday I promised to tell you if I make significant mistakes, hoping it would be a seldom occasion. Yet here I am writing to you about another mistake I made, though this time not one of technical nature. I chose the old name for this series of articles about Python’s inner working as I was musing about how creators may see their creations far more thoroughly than other people possibly can (the old name of the series was “Guido’s Python”). It was recently brought to my attention that the name I chose might have been incorrect, misleading and/or simply tasteless. I certainly did not mean to give a false impression of affiliation or endorsement; of course Python’s creator and community leader never had anything to do with the series. Due to these reasons and to set things straight, the series will be promptly and retroactively renamed to “Python’s Innards”.

Since the series’ old name was used in URLs and category names, things will break as I do the rename and readers will have to update their bookmarks. I’m sorry for the discomfort I’ll cause my readers with the rename, but far worse, I apologize if I misled anyone or tainted someone’s good name. I’ll use this unfortunate opportunity to say explicitly that unless stated otherwise, everything you can find anywhere under the domain (so far there’s only this blog) is solely my creation, not endorsed nor affiliated with anyone else (be assured that I’ll give credit where such is due if I’ll receive some kind of help in the future).

Genuinely no malice was intended and I hope this will be forgotten soon enough; I’m working on the next article and look forward to see you as we continue this journey into Python.

Correction for ‘Python’s Innards: Objects 102’

2010/05/20 § 1 Comment

Alas, it has happened, the first mistake in the ‘Python’s Innards’ series has been found. I was trying to answer a question raised by one of my Reddit readers regarding properties, and realized that I have overlooked a fine point about descriptors in my post. Oops.

As was originally (and correctly) written in the post, descriptors are objects whose type has their tp_descr_get and/or tp_descr_set slots set to non-NULL. However, I also wrote, incorrectly, that descriptors take precedence over regular instance attributes (i.e., attributes in the object’s __dict__). This is partly correct but misleading, as it doesn’t distinguish non-data descriptors from data-descriptors. An object is said to be a data descriptor if its type has its tp_descr_set slot implemented (there’s no particularly special term for a non-data descriptor). Only data descriptors override regular object attributes, non-data descriptors do not.

I apologize for the mistake, but alas can’t promise it will never happen again. I’ve created an Errata Policy page which explains how I intend to treat mistakes that will be found from now on. Briefly, it says that the policy is to correct the original page as if the mistake never existed, and if the mistake is significant enough, also write a separate post (like this one) which explains what happened. I plan not to tag errata posts with all the tags of the original post (I’ve done it with this post as an exception), so unless you explicitly register to the RSS feed of the Errata category in my blog, you won’t get further errata via Planet Python or other aggregation services that rely on the ‘python’ or ‘pythons-innards’ tags.

I hope you will keep enjoying and relying on this blog.

Where Am I?

You are currently browsing the Errata category at NIL: .to write(1) ~ help:about.