Recently, one particular thread at python-dev has piqued my interest. The thread discusses the possible pronouncement of PEP 3148 (re. the addition of futures to stdlib), and has been going on for over a hundred messages with no end in site. I’d like to use some examples from that thread in this post, but it could have been another thread from another list. I do not want to discuss the contents of the PEP itself or even the contents of the discussion – if I thought I had interesting opinions about these matters, I’d say them in public on the mailing list. I am interested in talking about the form of the discussion, which I think models many similar discussions in other threads and other mailing lists. To make a long story short, I feel the broad OSS community is largely under equipped for massive online debate.
We need a better way to debate in large, decentralized and open groups, because this is a lot of what we do, probably more than actually coding. Mailing list, newsgroups and bug-tracker comments are a good enough place to debate things which are one or more of (a) under a relative consensus, or (b) not too interesting to many of the listening audience, or (c) too arcane to actually be debated by many of the listening audience. The problem lies with the “hot” topics, about which it seems everyone has something to say and which are arguably the most important topics to debate effectively (to be able to do things, to conserve time, etc). The nature of these large, decentralized and open debates is that the participants vary a lot from one another. Some are well known members of the community and some passersby, some of them are better informed and some don’t support your opinion, some of them easier to take offence and some easier to deal it, some of them care a lot about the subject under discussion and some have nothing good on TV, some read and reply to new posts in near-real-time and some are married. In these kinds of debates you see the tools we currently use in all their crippled anti-glory; they’re just not meant to handle these things.
Because our tools don’t rise to the challenge, once a discussion goes beyond a few posts it’s hard to keep track of who is arguing for and who is arguing against (indeed in many cases people may do both). It’s hard to gauge whether the silent majority generally agrees with any particular argument (‘+1’ and ‘-1’ are a hack to alleviate this, not a terribly good one). Gauging the correct weight to attribute to what people are saying (and people vary a lot, as I said) is difficult and relies mostly on one being familiar or Googling the speakers. It’s even hard to simply keep the discussion on the same track: more often than not, the discussion creeps onto completely unrelated issues (maybe important ones), and ‘steals’ the focus from the original, probably also-important, subject. In short, we’re arguing in a mess and I’m not sure we apply our hackerish instincts of science, openness and rule-of-the-competent to how we make decisions.
I’m by no means an expert on these matters, so I Googled a bit (wasn’t sure which keywords to use) and looked at Wikipedia, the source of all knowledge which covers everything and is never wrong or inaccurate(™). It seems to me like the challenge of harnessing collective intelligence is well known, but it doesn’t seem like any of the vast array of ‘collaboration software‘ solutions out there tackles this particular issue in manner that befits a group of hackers working on an OSS project (what’s the name for a group of hackers? a pod of hackers? a pack of hackers? a pride of hackers? but I digress). IBM has a natural-language-processing 100,000 ton behemoth which can crunch the collective opinions of hundreds of thousands of people, but in typical IBM fashion it seems to also require a few millions dollars and the sacrifice of a virgin PhD (abundant resource) to operate. If indeed I didn’t miss an existing solutions, it seems obvious that a solution is needed. Maybe I missed a good solution, but the fact of the matter is that hackers around the globe still debate in mailing lists, groups, wiki-discussion-pages, tracker issues, forums, IRC channels, etc, all of which seem to me like the wrong tool for the job.
I think we need a rather simple solution, and luckily I think we’re pretty much the right crowd to implement it. It appears to me that even a naive analysis of the problem is sufficient to implement a ‘good enough’ solution, or at least a ‘better than now’ one. Consider this: a single subject (introduce futures into stdlib) can be debated from many different angles: Is the terminology chosen adequate, or should futures be called something else in Python? Should this be a part of stdlib or a part of PyPI? Should futures be implemented as one object or two separate instances, one on the emitting side and one on the consuming side? Each one of these angles can be argued for or against: futures is a good name because it’s a well-established name in computer science, futures is a bad name because Python already has __future__, and so on, and so forth.
I’m thinking of a tool a bit like Rietveld, only not intended to discuss the very specific implementation aspects of patches, but rather to discuss opinions. I think my naive analysis of ‘a debate’ reveals three abstractions: “what’s under discussion” (PEP 3148), “why don’t we just accept whatever it is that’s under discussion as-is” (because futures is the wrong terminology, etc) and “who has an opinion about any of the ‘why’s” (I do, I think ‘deferreds’ is a better idea, etc). I think these three abstractions cover most of what we need to implement a ‘debating tool’, if coupled with some development of the idea that people’s opinion carry different weights (the ‘who’s differ; Guido carries more weight than I do, and thought should be given to how the implementation represents that). My assessment is that implementing something like this shouldn’t be too much work for web-technology-savvy people, even a small team could probably finish it in one sprint.
I agree that some questions remain about the interaction of this with current tools (this isn’t a mailing list or bug tracker or anything else’s replacement, much like Rietveld doesn’t replace any of them), as well as maintaining ‘law and order’ in the discussion (among the target audience I’m talking about, I think this isn’t such a big deal). Also, there are lots of possible additions to this concept, and maybe some should be added, but I think simplicity is key here. Part of the whole idea is that it won’t turn into incredibly complex ‘jamming’ software. And finally, no, I don’t know how to monetize it other than adwords and no, I don’t mind if you implement it so long as you keep it free as in speech. That’s mostly it, I could post the post right now and be reasonably satisfied with it. Ten minutes later someone would come along, point me at a website that already does it, and then I’d just have to advocate using that website on python-dev (the debate over whether to use the debating-website for future debates is a self-hosting one). But no. I didn’t just write this post and let ‘good enough’ be; I had to expose myself to public humiliation by creating a mockup UI for this imaginary website.
I hope that in this case a picture will be worth more than the thousand words I wrote here (1250+ words after editing, I must be doing something wrong), so here goes (*gulp*, I know it’s not the best mockup UI you’ve seen):
Thanks for listening, I’ll await your comments.
EDIT: I found this on the interwebs. Not exactly what I was thinking of, and maybe not streamlined enough to really use as a debating tool in online communities, but probably better than a mailing list.
Comments
24 responses to “Mailing list debates considered harmful”
“pride of hackers” sounds about right 🙂
Agreed mailing lists are a fairly lousy tool, but one of their current benefits is their ubiquity and interoperability.
In theory, PEPs are meant to serve as the summaries of related python-dev discussions, with the PEP authors and champions doing the collation. I suspect a tool like the one you describe would initially fit in as something to help someone summarise a discussion, rather than as a venue for the discussion itself, although over time it may mature into something capable of handling the latter role.
So, not a crazy idea, but perhaps not as easy to get right as you suggest either (email’s greatest weakness, it’s comparative lack of structure, it’s also one of it’s major strengths)
What you wrote in the second paragraph crossed my mind, this is what I alluded to in ‘some questions remain about the interaction of this with current tools’; I guess we’ll have to see it to know how it will work.
Welcome to the Bike Shed: http://en.wikipedia.org/wiki/Parkinson's_Law_of_Triviality
The problem with mailing lists is the same problem of human nature. No one is willing to debate the functionality of the internal GIL implementation, because few people feel equipped to say anything intelligent or rational about it.
However; if you bring up thread pools – e.g. futures, plenty of people think they’re well equipped to debate the API, or the names, or the functionality because it is a pretty simple concept at it’s core, and a common pattern in computer science in general.
Mailing lists rapidly break down when the subject is easy – when the subject is hard, it keeps itself relatively rational.
As for PEPs, I’ve never found the discussions that hard to follow – the futures one is relatively typical of a “simple” PEP, but also displays something interesting, despite there being mailing list archives, and some of the self-same people asking the same questions months ago, those same people ask the same questions and rehash the same stuff that was dealt with months ago.
Why? Not because they can’t look up the threads from the past – we have that – but rather people have short memories and little time to really engage, therefore, they will rehash the same thing over and over.
In any case; the PEP is supposed to encapsulate and abstract the discussion post-facto, but not during the debate process. I don’t know that writing tools is the right thing to do – undoing years of learned email-memory would be nearly impossible and it creates yet another information silo where people have to go look for things.
I’m never one to accept the status quo – but in this case, it feels like you’re asking for a breathable atmosphere on mars. 😉
I think I should have better clarified that Parkinson’s Law isn’t what this post is lamenting, though it is a bit related. I agree to let people argue about marginal or easy subjects as much they like, or at least admit I can’t make them not do it. I just want to give them good tools for these discussions, marginal or not.
To be honest and not intending to offend anyone, I’m not 100% interested in PEP 3148, I guess I’m +0 for it. Given this level of interest, I found it hard (or time consuming) to get what people are talking about in the hefty thread that ensued. Searching the archives of mailing lists is usually inconvenient, and its hard to get the gist of what’s going on without reading much of the thread’s text anyway. In a debate, it’s important not to let new and crufty subjects overshadow the old and important subjects that were already discussed, and it seems that this is precisely what email threads or long comment pages do.
I argue that a debate among hackers can and should be created in a structured way, we are all (/most) instinctively wired that way anyway. As Nick noted above, it should at least be rendered into such a form by the person championing the subject under discussion in near-real time (i.e., when they see someone saying something new on the list, quickly jot it in the subject’s debate page), so people who /do/ use the debate page can click a link and see at a glance who said what so far, and who/how many supports which opinions. Again, this is a bit like the relationship between an issue’s comment page on the tracker and Rietveld.
Finally, let it be put on record that I think a breathable atmosphere on Mars or some other monumental feat in space exploration is difficult, but probably the best thing humanity can do just about now. Well, after implementing this idea of course. 🙂
If such a topic is a Gordian Knot, its the Benevolent Dictator for Life’s job to cut right through the center of it.
Not every issue is best resolved through consensus or democratic process.
Having the BDFL to zap endless arguments about something useless or that he feels isn’t for discussion is absolutely terrific, but this doesn’t mean we shouldn’t have effective tools to argue at length about something important that the BDFL would like to see a good discussion of.
An argument map, perhaps?
http://en.wikipedia.org/wiki/Argument_map
Yep, this is quite what I’m talking about. But quickly browsing through the relevant “external links” section, it seemed to me none match the implementation I think is best. The thing plaguing all these implementations is that they allow for too much freedom than what I intuitively think is good for a focused debate.
Note in the text: Did you mean to say “piqued my interest” instead of “picked my interest”?
Note in the topic: I think this problem is much harder than it looks. It’s like trying to come up with a tool that makes parliamentary democracy more civil.
Hrmf, yes, of course I meant piqued. Thanks! (next time, we do this conversation in my native language, buster 🙂
re. topic: quite possibly. One of the main reasons I hope it may work despite the obvious simplification is the fact hackers are what they are. Many a thread it felt to me like people were instinctively “implementing” this with threaded email, with techniques like a successsion of short replies to the various people/discrete arguments they are currently supporting/attacking instead of a more emailish-natural single long reply, or +1/-1 poor-man’s voting.
Then again, I agree I might be wrong. Maybe some day I’ll code it, just to prove that I’m indeed wrong 🙂
I rather like “A Hubris of Hackers” myself. Nevertheless, you make a great point. The python-dev list is incredibly hard to follow, and most certainly requires some amount of prior knowledge about the speaker in each message. I hope something good comes out of this. I, for one, would find it tremendously helpful.
A Hubris of Hackers is indeed a good name.
I’m glad you see my point, but I’d like to stress I don’t think what I described applies only to python-dev. I’ve been off the Linux kernel’s mailing list for a while, but I recall similar things going on there.
When there are fires in the real world we put them out with water, and other flame retardant chemicals. When there are flamewars on the Internet how do you put them out?
Simple. You just do something.
Take the example of Dudebro
http://www.neogaf.com/forum/showthread.php?t=381946
There was a flamewar because some kid liked some babyz game. Then someone took the argument and twisted it to creative ends. Rather than continuing to debate, they started to make. They made art, they made games, they made animations, they made lots of stuff. When you make, you win.
In the open source world, making is even easier. Just write the code. If there are some powers that be that don’t accept your code, just ignore them. Every second you waste trying to convince someone to accept your patch, you could be getting something done.
I use tons of open source software. Tons. I participate in 0 communities and follow 0 mailing lists. It’s just not worth it. I’ll file bugs now and then, and submit patches/fixes if I come up with them. But I won’t fight. I have better things to do. And in the end, the code I write and the things I create are stronger than any argument that can possibly be typed.
I understand where you’re coming from, but if you want to do significant contribution to a significant OSS project, I don’t think you have a choice. It’s always fun to work on a started-from-stretch hobby project, but these hobby projects won’t be used by millions of people, and if they would – debates are bound to ensue.
I got chills while reading your article, because as a single founder of a bootstrapped startup, the problem of dated mailing lists and discussion forums is the exact problem I’ve been trying to solve and this is one of the first times someone articulated the problem so well.
Check out my startup: http://BraintrustHQ.com.
If you’re interested, I’d love to work with you and whomever else is interested to see how we can make this free and make it work for the Open Source Community.
I’d be happy to share my ideas freely, feel free to email me at yaniv at aknin dot name, if you’d like. That said, I don’t see myself getting significantly committed to such a project any time soon.
Braintrust looks like a really nice option but for something a little different you’re welcome to use http://www.debatewise.org – it’s a wiki approach to debating, people can collaborate on arguments and each point has a counterpoint so making it easier for the audience to compare and decide.
We’ve got voting tools too, so the silent majority can indicate how they feel. We don’t have user profiles though, so you can’t gauge the authoritaveness of a post by who posted it.
We’re also open source, or rather will be soon. If anyone wants to develop what we’ve done or get involved with what we’re doing please get in touch.
When I read your comment about being “[unsure] sure which keywords to use”, I remembered coming across something like this a while ago. Thankfully, I tagged it in Delicious, so I was able to locate it again. I found it in 2005, and hadn’t revisited it since, but it looks like it is still plugging along, and may be relevant to your desire for a tool to aid group decision making. It’s implementation history will take bit more reading on the website to accurately piece together, but it appears to have gone through a number of implementations: from PHP, to Ruby on Rails, and recently to Django, and it seems to be open source.
Here are two relevant sites for the that will connect you to others:
http://www.groupspace.org/
http://deme.stanford.edu/
And just to note, I have no first-hand experience or relationship with this project… I just stumbled across it many years back and found it interesting enough to bookmark.
While the mailing list implementation of Python discussion could probably be improved (eg, something making it even easier to find threads and particular posts in them), the real cause of such discussions is the requirement for consensus. Some topics are inherently controversial, and getting to consensus on those takes talk and time (and sometimes fails; that’s why having a BDFL is important).
I rather suspect that no matter how good you make the forum for such debates, there are still going to be threads that seem important but put you off because of their length and the lack of coherence in the arguments you find in the first sample you take.
Another suggestion for a collaboration / debate model: http://www.eleatica.com — which is built on an opinion -> reason -> counteropinion model. I’m not affiliated with it, although I know the developers, but it sounds like a good match for what you’re thinking of.
This tool looks like a good way to “manage” a debate.
But this “managing” is also its downside. Once a dilaogue is sturctured it is also limited, to the possibilities poised by its own framework.
Another issue stemming from this fact, is the identity of the debate facilitator and her control over the results of the debate.
Why, hello mate! I thought about this tool for technological debates among a group of reasonably trusted hackers rather than as a general purpose tool. Limiting the structure of a debate might indeed limit the possibility of expression (and adoption), but I also believe it’s instrumental in allowing the debate to take place in a decent manner. As for the second point, as I don’t think this tool is a replacement for regular mailing list, I’m certain that anyone who’d feel they were ‘silenced’ would have no problem reaching their audience.
Haven’t checked it out, but http://dotmocracy.org looks like providing some of the functionality you’re talking about here.
I think stackexchange.com/sites (search for its libre clones) is a good enough platform for what I’d consider good debating software. The format: one person asks a question, and a number of people take a stab at the answer, while also offering commenting on individual answers (EG, asking for clarification). This can be adapted to debate form, where a question can be “to GIL or not to GIL” and people give reasons, which would get up/down votes.