RESTfully atomically incrementing a counter using HTTP PATCH

2012/02/08 § Leave a comment

So today I ran into the question of incrementing a counter in a RESTful manner, and wasn’t sure how to go about doing it. Googling around a bit didn’t find me a satisfactory answer, though I did find @idangazit asked the same question on Stack Overflow, but alas the question was answered by what I humbly felt was an inadequate answer.

Idan had “PUT vs. POST” in his question, but quoting the answer I just added to that question (#selfplagiarism!), I believe PATCH is the answer as RFC 2068 says very well:

The PATCH method is similar to PUT except that the entity contains a list of differences between the original version of the resource identified by the Request-URI and the desired content of the resource after the PATCH action has been applied. The list of differences is in a format defined by the media type of the entity (e.g., “application/diff”) and MUST include sufficient information to allow the server to recreate the changes necessary to convert the original version of the resource to the desired version.

So, for example, to update profile 123’s view count, I would do (using requests, what else?):

import requests

requests.patch(
    'http://localhost:8000/profiles/123',
    'views + 1\n',
    headers={"Content-Type": "application/x-counters"}
)

Which would emit something like:

PATCH /profiles/123 HTTP/1.1
Host: localhost:8000
Content-Length: 10
Content-Type: application/x-counters
Accept-Encoding: identity, deflate, compress, gzip
Accept: */*
User-Agent: python-requests/0.10.0

views + 1

Where the x-counters media type (which I just made up) is made of multiple lines of field operator scalar tuples. views + 1, views = 500, views - 1 or views + 3 are all valid syntactically (but some may be forbidden semantically). I can understand some frowning-upon making up yet another media type, but I think this approach matches the intention of the RFC quite well, it’s extremely simple and if the backend is implemented correctly, it’s atomically correct.

Suggestions for another approach?

EDIT: I’ve had a long discussion with a friend who disliked the use of a non-standard media type. Perhaps something like this is better, though I’m still not entirely convinced:

import requests

requests.patch(
    'http://localhost:8000/profiles/123',
    '[{"field": "views", "operator": "+", "operand": 1}]',
    headers={"Content-Type": "application/json"}
)

I’m not sure what’s the bigger crime – using a non-standard media type, which, in the words of the RFC, is “discouraged”, or using a standard generic serialization format as the media type, which doesn’t say much about the scheme you’d like to use within it. Both are better than anything else I can think of.

p.s.: escaping spaces in field names are left as an exercise to the reader, I suggest application/x-www-form-urlencoded or simply using sane field names, ffs.

Advertisements

Walking Python objects recursively

2011/12/11 § 6 Comments

Here’s a small function that walks over any* Python object and yields the objects contained within (if any) along with the path to reach them. I wrote it and am using it to validate a deserialized datastructure, but you can probably use it for many things. In fact, I’m rather surprised I didn’t find something like this on the web already, and perhaps it should go in itertools.

Edit: Since the original post I added infinite recursion protection following Eli and Greg’s good advice, added Python 3 compatibility and did some refactoring (which means I had to add proper unit test). You will always be able to get the latest version here, on ActiveState’s Python Cookbook (at least until it makes its way into stdlib, fingers crossed…).

from collections import Mapping, Set, Sequence

# dual python 2/3 compatability, inspired by the "six" library
string_types = (str, unicode) if str is bytes else (str, bytes)
iteritems = lambda mapping: getattr(mapping, 'iteritems', mapping.items)()

def objwalk(obj, path=(), memo=None):
    if memo is None:
        memo = set()
    iterator = None
    if isinstance(obj, Mapping):
        iterator = iteritems
    elif isinstance(obj, (Sequence, Set)) and not isinstance(obj, string_types):
        iterator = enumerate
    if iterator: 
        if id(obj) not in memo:
            memo.add(id(obj))
            for path_component, value in iterator(obj):
                for result in objwalk(value, path + (path_component,), memo):
                    yield result
            memo.remove(id(obj))
    else:       
        yield path, obj

And here’s a little bit of sample usage:

>>> tuple(objwalk(True))
(((), True),)
>>> tuple(objwalk({}))
()
>>> tuple(objwalk([1,2,3]))
(((0,), 1), ((1,), 2), ((2,), 3))
>>> tuple(objwalk({"http": {"port": 80, "interface": "0.0.0.0"}}))
((('http', 'interface'), '0.0.0.0'), (('http', 'port'), 80))
>>> 

"any" is a strong word and Python is flexible language; I wrote this function to work with container objects that respect the ABCs in the collections module, which mostly cover the usual builtin types and their subclasses. If there’s something significant I missed, I’d be happy to hear about it.

pv: the pipe swiss army knife

2011/11/05 § Leave a comment

When using UNIX, every now and then you run into a relatively unknown command line application which, once you master it, becomes part of your “first class” commands along with cut or tr. You wince every time you work on a computer that doesn’t have it (and promptly wget-configure-make-install it) and you’re amazed your colleagues never heard of it. I often feel pv is such a command for me. Really, this command, much like netcat, should have been written in Berkley sometime circa 1985 and be in every /usr/bin today. Alas, somehow Hobbit only wrote netcat in 1996, and it took a long while for for it to reach /usr/bin ubiquity. Similarly, Andrew Wood only wrote pv in 2002, and I hope this post will convince you to place it in all your /usr/local/bins today and convince distribution makers to promote it to the status of a standard package as soon as possible.

The basic premise of pv is simple – it’s a program that copies stdin to stdout, while richly displaying progress using terminal graphics on stderr. If you use UNIX a lot and you never heard of pv before, I’m pretty sure the lightbulb is already lit above your head (if not, maybe pv isn’t for you after all or maybe it would help if you’d take a look this review of pv to help you see why it’s so great). pv has evolved rather nicely over the years, it’s available from Ubuntu universe for a while now (why only universe? why??), and it has a slew of useful features, like rate limiting, ETA prediction for an expected amount of data, on-the-fly parameter change (increase/decrease rate limit without breaking the pipe!), multiple invocation support (measure speed in two different points of the pipe!) and so on.

If you’re using pv, I hope you may want to see some of the recipes I use it in; if you don’t, maybe they’ll whet your appetite (I’m using long options for pv and short options for everything else):

  1. The basics: copy /tmp/src/ to /tmp/dst/, with progress
  2. $ src=/tmp/src ; tar -cC "$src" . |
      pv --size $(du -hsk "$src" | cut -f1)k |
      tar -xC /tmp/dst
     142MB 0:00:02 [43.4MB/s] [======>    ] 58% ETA 0:00:01
    $
    

    By the way, this works great if you add nc and compression, pv can even help you decide what level of compression to use to achieve the best throughput before the CPU becomes the bottleneck.

  3. Scale a bunch of images to a specific size, using multiple cores and with progress
  4. $ cd /tmp/src ; ls *.jpg |
      xargs -P 4 -I % -t convert -resize 1024 % /tmp/dst/% 2>&1 |
      pv --line-mode --size $(ls *.jpg | wc -l) > /dev/null
      96 0:00:16 [7.85/s] [===>       ] 36% ETA 0:00:28
    $
    
  5. Get a quick assessment of the traffic rate going through some interface
  6. $ sudo tcpdump -c 10000 -i eth1 -w - 2>/dev/null | pv > /dev/null
    35.4MB 0:00:07 [4.56MB/s] [ <=>          ]
    $
    

Nifty, eh? I find myself inserting pv in any pipe I expect to exist for more than a few moments. How do you use pv?

zsh and virtualenv

2010/10/14 § 8 Comments

A week ago or so I finally got off my arse and did the pragmatic programmer thing, setting aside those measly ten minutes to check out virtualenv (well, I also checked out buildout, but I won’t discuss it in this post). I knew pretty much what to expect, but I wanted to get my hands dirty with them so I could see what I assumed I’ve been missing out on for so long (and indeed I promptly kicked myself for not doing it sooner, yada yada, you probably know the drill about well-known-must-know-techniques-and-tools-that-somehow-you-don’t-know). Much as I liked virtualenv, there were two things I didn’t like about environment activation in virtualenv. First, I found typing ‘source bin/activate’ (or similar) cumbersome, I wanted something short and snazzy that worked regardless of where inside the virtualenv I am so long as I’m somewhere in it (it makes sense to me to say that I’m ‘in’ a virtualenv when my current working directory is somewhere under the virtualenv’s directory). Note that being “in” a virtualenv isn’t the same as activating it; you can change directory from virtualenv foo to virtualenv bar, and virtualenv foo will remain active. Indeed, this was the second problem I had: I kept forgetting to activate my virtualenv as I started using it or to deactivate the old one as I switched from one to another.

zsh to the rescue. You may recall that I already mentioned the tinkering I’ve done to make it easier to remember my current DVCS branch. Briefly, I have a function called _rprompt_dvcs which is evaluated whenever zsh displays my prompt and if I’m in a git/Mercurial repository it sets my right prompt to the name of the current branch in blue/green. You may also recall that while I use git itself to tell me if I’m in a git repository at all and which branch I’m at (using git branch --no-color 2> /dev/null | sed -e '/^[^*]/d' -e 's/* \(.*\)/(\1)/'), I had to resort to a small C program (fast_hg_root) in order to decide whether I’m in a Mercurial repository or not and then I manually parse the branch with cat. As I said in the previous post about hg and prompt, I’m not into giving hg grief about speed vs. git, but when it comes to the prompt things are different.

With this background in mind, I was perfectly armed to solve my woes with virtualenv. First, I changed fast_hg_root to be slightly more generic and search for a user-specified “magic” filename upwards from the current working directory (I called the outcome walkup, it’s really simple and nothing to write home about…). For example, to mimic fast_hg_root with walkup, you’d run it like so: $ walkup .hg. Using $ walkup bin/activate to find my current virtualenv (if any at all), I could easily add the following function to my zsh environment:

act () {
        if [ -n "$1" ]
        then
                if [ ! -d "$1" ]
                then
                        echo "act: $1 no such directory"
                        return 1
                fi
                if [ ! -e "$1/bin/activate" ]
                then
                        echo "act: $1 is not a virtualenv"
                        return 1
                fi
                if which deactivate > /dev/null
                then
                        deactivate
                fi
                cd "$1"
                source bin/activate
        else
                virtualenv="$(walkup bin/activate)" 
                if [ $? -eq 1 ]
                then
                        echo "act: not in a virtualenv"
                        return 1
                fi
                source "$virtualenv"/bin/activate
        fi
}

Now I can type $ act anywhere I want in a virtualenv, and that virtualenv will become active; this saves figuring out the path to bin/activate and ending up typing something ugly like $ source ../../bin/activate. If you want something that can work for you without a special binary on your host, there’s also a pure-shell version of the same function in the collapsed snippet below.

function act() {
    if [ -n "$1" ]; then
        if [ ! -d "$1" ]; then
            echo "act: $1 no such directory"
            return 1
        fi
        if [ ! -e "$1/bin/activate" ]; then
            echo "act: $1 is not a virtualenv"
            return 1
        fi

        if which deactivate > /dev/null; then
            deactivate
        fi
        cd "$1"
        source bin/activate
    else
        stored_dir="$(pwd)"
        while [ ! -f bin/activate ]; do
            if [ $(pwd) = / ]; then
                echo "act: not in a virtualenv"
                cd "$stored_dir"
                return 1
            fi
            cd ..
        done
        source bin/activate
        cd "$stored_dir"
    fi
}

This was nice, but only solved half the problem: I still kept forgetting to activate the virtualenv, or moving out of a virtualenv and forgetting that I left it activated (this can cause lots of confusion, for example, if you’re simultaneously trying out this, this, this or that django-facebook integration modules, more than one of them thinks that facebook is a good idea for a namespace to take!). To remind me, I wanted my left prompt to reflect my virtualenv in the following manner (much like my right prompt reflects my current git/hg branch if any):

  1. If I’m not in a virtualenv and no virtualenv is active, do nothing.
  2. If I’m in a virtualenv and it is not active, display its name as part of the prompt in white.
  3. If I’m in a virtualenv and it is active, display its name as part of the prompt in green.
  4. If I’m not in a virtualenv but some virtualenv is active, display its name in yellow.
  5. Finally, if I’m in one virtualenv but another virtualenv is active, display both their names in red.

So, using walkup, I wrote the virtualenv parsing functions:

function active_virtualenv() {
    if [ -z "$VIRTUAL_ENV" ]; then
        # not in a virtualenv
        return
    fi

    basename "$VIRTUAL_ENV"
}

function enclosing_virtualenv() {
    if ! which walkup > /dev/null; then
        return
    fi
    virtualenv="$(walkup bin/activate)"
    if [ -z "$virtualenv" ]; then
        # not in a virtualenv
        return
    fi

    basename $(grep VIRTUAL_ENV= "$virtualenv"/bin/activate | sed -E 's/VIRTUAL_ENV="(.*)"$/\1/')
}

All that remained was to change my lprompt function to look like so (remember I have setopt prompt_subst on):

function _lprompt_env {
    local active="$(active_virtualenv)"
    local enclosing="$(enclosing_virtualenv)"
    if [ -z "$active" -a -z "$enclosing" ]; then
        # no active virtual env, no enclosing virtualenv, just leave
        return
    fi
    if [ -z "$active" ]; then
        local color=white
        local text="$enclosing"
    else
        if [ -z "$enclosing" ]; then
            local color=yellow
            local text="$active"
        elif [ "$enclosing" = "$active" ]; then
            local color=green
            local text="$active"
        else
            local color=red
            local text="$active":"$enclosing"
        fi
    fi
    local result="%{$fg[$color]%}${text}$rst "
    echo -n $result
}

function lprompt {
    local col1 col2 ch1 ch2
    col1="%{%b$fg[$2]%}"
    col2="%{$4$fg[$3]%}"
    ch1=$col1${1[1]}
    ch2=$col1${1[2]}

    local _env='$(_lprompt_env)'

    local col_b col_s
    col_b="%{$fg[green]%}"
    col_s="%{$fg[red]%}"

    PROMPT="\
$bgc$ch1\
$_env\
%{$fg_bold[white]%}%m:\
$bgc$col2%B%1~%b\
$ch2$rst \
$col2%#$rst "
}

A bit lengthy, but not very difficult. I suffered a bit until I figured out that I should escape the result of _lprompt_virtualenv using a percent sign (like so: "%{$fg[$color]%}${text}$rst "), or else the ANSII color escapes are counted for cursor positioning purposes and screw up the prompt’s alignment. Meh. Also, remember to set VIRTUAL_ENV_DISABLE_PROMPT=True somewhere, so virtualenv’s simple/default prompt manipulation functionality won’t kick in and screw things up for you, and we’re good to go.

The result looks like so (I still don’t know how to do a terminal-“screenshot”-to-html, here’s a crummy png):

Voila! Feel free to use these snippets, and happy zshelling!

Eulogy to a server

2010/10/01 § 2 Comments

You don’t know it, but I’ve started writing this blog several times before it actually went live, and every time I scraped whatever post I started with (the initial run was on blogger.com). I just didn’t think these posts were all too interesting, they were about my monstrous home server, donny. Maybe this is still not interesting, but I’m metaphorically on the verge of tears and I just have to tell someone of what happened, to repent me of my horrible sin. You may not read if you don’t want to. I bought donny about 2.5-3 years ago, to replace my aging home storage server (I had about 3x250GB at the time, no RAID). There’s not much to say about donny’s hardware (Core 2 Duo, 2GB of RAM, Asus P5K-WS motherboard), other than the gargantuan CoolerMaster Stacker 810 chassis with room for 14 (!) SATA disks. Initially I bought 8×0.5TB SATA Hitachi disks for it, and added more as I had the chance. I guess I bought it because at the time I’d hang around disks all day long, I must’ve felt the need to compensate for something (my job at the time was mostly around software, but still, you can’t ignore the shipping crates of SATA disks in lab).

Anyway, most of its life donny ran OpenSolaris. One of our customers had a big ZFS deployment, I’ve always liked Solaris most of all the big Unices (I never thought it really better than Linux, it just Sucked Less™ than the other big iron Unices), I totally drank the cool-aid about “ZFS: the last word in File System” (notice how the first Google hit for this search term is “Bad Request” :) and dtrace really blew me away. So I chose OpenSolaris. Some of those started-but-never-finished posts were about whether I was happy with OpenSolaris and ZFS or not, I never found them interesting enough to even finish them. So even if I don’t wanna discuss that particularly, it should be noted that if we look at how I voted with my feet, I ended up migrating donny to Ubuntu 10.04.1/mdadm RAID5/ext4 when my wife and I got back from our long trip abroad.

Migration was a breeze, the actual migration process convinced me I’ve made the right choice in this case. Over the time with ZFS (both at work and at home) I realized it’s probably good but certainly not magical and not the end of human suffering with regard to storage. In exchange for giving up zfs and dtrace I received the joys of Ubuntu, most notably a working package management system and sensible defaults to make life so much easier, along with the most vibrant eco-system there is. I bought donny 4×2.0TB SATA WD Cavier Green disks, made a rolling upgrade for the data while relying on zfs-fuse (it went well, despite a small and old bug) and overall the downtime was less than an hour for the installation of the disks. At the time of the disaster, donny held one RAID5 array made of 4x2TB, one RAID5 array made of 4x.5TB, one soon-to-be-made RAID5 array made of 3x1TB+1×1.5TB (I bought a larger drive after one of the 1TB failed a while ago), and its two boot disks. I was happy. donny, my wife and me, one happy family. Until last night.

I was always eyeing donny’s small boot disks (what a waste of room… and with all these useless SATA disks I’ve accumulated over the years and have lying about…), so last night I wanted to break the 2x80GB mirror and roll-upgrade to a 2x1TB boot configuration, planning on using the extra space for… well, I’ll be honest, I don’t know for what. I’ll admit it – I got a bit addicted to seeing the TB suffix near the free space column of df -h at home (at work you can see better suffixes :). I just have hardware lying around, and I love never deleting ANYTHING, and I love keeping all the .isos of everything ever (Hmm… RHEL3 for Itanium… that must come in handy some day…) and keeping an image of every friend and relative’s Windows computer I ever fixed (it’s amazing how much time this saves), and never keeping any media I buy in plastic… and, well, the fetish of just having a big server. Heck, it sure beats farmville.

So, indeed, last night I broke that mirror, and installed that 1TB drive, and this morning I started re-mirroring the boot, and while I was at it I started seeing some of the directory structure was wrong so I redistributed stuff inside the RAIDs, and all the disks where whirring merrily at the corner of the room, and I was getting cold so I turned off the AC, and suddenly donny starts beeping (I didn’t even remember I installed that pcspkr script for mdadm) and I get a flurry of emails regarding disks failures in the md devices. WTF? Quickly I realized that donny was practically boiling hot (SMART read one of the disks at 70 degrees celsius), at which point I did an emergency shutdown and realized… that last night I disconnected the power cable running from the PSU to several fans, forgot to reconnect it, and now I’ve effectively cooked my server. Damn.

I’m not sure what to do now. I still have some friends who know stuff about harddisks (like, know the stuff you have to sign NDAs with the disk manufacturers in order to know), and I’m trying to pick my network about what to do next. Basically, from what I hear so far, I should keep donny off, let the disks cool down, be ready with lots of room on a separate host to copy stuff out of it, boot it up in a cool room, take the most critical stuff out and then do whatever, it doesn’t matter, cuz the disks are dead even if they seem alive. I’m told never to trust any of the disks that were inside during the malfunction (that’s >$1,000USD worth of disks…), once a disk reached 70 degrees, even far less, don’t get near it, even if it’s new. Admittedly, these guys are used to handling enterprise disk faults, where $1,000USD in hardware costs (and even many many times that amount) is nothing compared to data loss, but this is the gist of what I’m hearing so far. If you have other observations, let me know. It’s frustratingly difficult to get reliable data about disk failures on the Internet; I know just what to do in case of logical corruption of any sort; but I don’t know precisely what to do in a case like this, and in case of a controller failure, and a head crash, and so on, and so forth. I know it’s a lot about luck, but what’s the best way to give donny the highest chance of survival?

On a parting note, I’ll add that I’m a very sceptic kind of guy, but when it comes to these things I’m rather mystical. It all stems from my roots as a System Administrator; what else can comfort a lonely 19-year-old sysadmin trying to salvage something from nothing in a cold server room at 03:27AM on a Saturday? So now I blame all of this for the name I gave donny. I named it so because I name all my hosts at home after characters from Big Lebowski (I’m typing this on Dude, my laptop), and I called the server donny. The email address I gave it (so it could send me those FzA#$%! S.M.A.R.T reports it never did!) was named Theodore Donald Kerabatsos. The backup server, which is tiny compared to donny and doesn’t hold nearly as much stuff as I’d like to have back now, is called Francis Donnelly. The storage pools (and then RAID volumes) were called folgers, receptacle, modest and cookies (if you don’t understand, you should have paid more attention to Donny’s funeral in The Big Lebowski). And, indeed, as I predicted without knowing it, it ended up being friggin’ cremated. May Stallman bless its soul.

I guess I’m a moron for not thinking about this exact scenario; I was kinda assuming smartmontools and would be SMART (ha!) enough to shutdown when the disks reach 50 degrees, and maybe there is such a setting and I just didn’t enable it… I guess by now it doesn’t matter. I’m one sad hacker. I can’t believe I did this to myself.

The Curious Case of HID Malfunction

2010/08/21 § Leave a comment

A quick tidbit for any interested hardware wizards out there (I know no one is likely to care, this is really more of an excuse for why the next Python’s Innards post is progressing slowly). As some of you know, I’m currently on a long trip with my wife (a trip which is already nearing its end…). This means I’m rather poor in hardware, and that sometimes the environment is harsh – hot, cold, humid, occasionally vibrating (flights, boats), rich with small particles (sand, dust), etc. As I expected for a long time, the elements finally had their toll on my small but until now trusty Asus EeePC 1005HA. The thing is, the toll was taken in a rather odd way.

For about three days now the builtin trackpad stopped working – most of the time. Usually I get no cursor movement nor clicks, and there are no particular messages in dmesg//var/log/messages. On rare occasions the trackpad resumes working for a bit, I wasn’t able to find a pattern in what makes it work for these short periods of time (heating, cooling, sleeping, booting… nothing seems to make the short ‘work-periods’ predictable). On one occasion the trackpad worked but behaved erratically (jumping around, random clicks, etc), on others it works fine, but for a few seconds and up to a few minutes. I’m running Ubuntu 10.04, kernel 2.6.32-24, keeping it reasonably apt-get updated. I didn’t change anything significant in the software configuration of the computer before this happened, and booting a vanilla 10.04 from a USB stick I have around doesn’t help, so I’m pretty sure it’s not a vanilla software issue (despite the oddity listed below).

This is patently unpleasent but not entirely odd, and I would chalk it down to some undefined hardware damage and let it be. I could buy an external mouse for the remaining few weeks of the trip and otherwise ignore the issue, lest the builtin keyboard started showing similar behaviour. It works far more often than the mouse, but has spells of brokeness. An external USB keyboard works fine when plugged in. I don’t even know if my internal keyboard interfaces via some kind of internal USB controller or not; seems not, as even when it’s working it’s not listed in lsusb -v. /proc/bus/input/devices lists an “AT Translated Set 2 keyboard”, but I have no idea if this is really my keyboard or not. Anyway, the really weird thing is that the keyboard’s broken behaviour has a few extra odd quirks:

  • It works perfectly prior to loading the kernel: in the BIOS configuration screen, or GRUB’s menu, or the USB stick’s boot menu. It seems that as soon as the kernel is loaded, no more keyboard (X11 or console).
  • The “special” EeePC keys, like toggling wifi or changing screen brightness, work perfectly. They aren’t special keys, but rather a key combination, and the keys used in the combination don’t reach the OS discreetly.
  • When I open the laptop’s lid in sleep, I need to hit a key to bring it out of sleep. Any key works well enough for the computer to wakeup, and the very same key (or any other key) will promptly stop working when the OS is awake enough to ask for a password.

So what gives? My keyboard isn’t broken, but some kind of interface between the keyboard and the system which is circumvented by the BIOS but is used by the kernel is broken? Huh, WTF?

The bit of Googling I did yielded nothing, Internet here isn’t really scarce but it isn’t abundant and sure is not fast or pleasent (I’m on a beach in Thailand at the moment). I’m left with a big WTF and apt-get install keynav. Any tips will be greatly appreciated (and speed up the next post in the Python’s Innards series, too!).

Update: I’ve decided to disassemble and reassemble the keyboard, following these instructions, using a swiss-army knife, my wife’s fingernail file and a camping torch. Following the work both keyboard and touchpad are working for about 10 minutes now, one of the longer durations in the past few days. I can only hope I fixed the problem. Either way, I’m curious why the keyboard consistently didn’t work with a loaded kernel yet seemed to work fine using the BIOS (in the BIOS’ setup, GRUB’s boot menu, etc). Any explanations?

Hacker irked by reincarnation

2010/08/16 § 20 Comments

Today I chose a rather peculiar topic for a technology blog: the history of reincarnation research and its implications on science. This might seem a bit awkward or even off-topic, I think it’s neither (and I make up the rules here). Before we begin, I gather I should say that I’m a sceptic, I’ve always been a sceptic and I never saw myself much as a very spiritual or new-age kind of person. When my wife and I entered a 10-day course about Buddhism about a month ago, you can imagine I arrived with hefty sacks full of various grains of salt to take everything with.

I didn’t know much about Buddhism before the course, and I’m not any kind of an expert about it now, either. This post isn’t about Buddhism at all, actually – just about a small thing I ran into during the course. As it happened the course material mentioned a certain research by a Dr. Ian Stevenson from the University of Virginia, who dedicated much of his career to research of reincarnation. I don’t recall precisely how the wording went, but as best as I can recall the course material took his research to show that reincarnation is scientifically proven. Naturally, that was my queue for the Grand Entrance of the Grains of Salt.

So as soon as we left the course I spent a while Googling the late Dr. Stevenson, his work, and the work of others in the field. What I found was very disturbing: Dr. Stevenson seems, to the best of my ability to assess, as a reasonable researcher with reasonable methods who was never properly refuted. And he found some ‘disturbing’ (i.e., unexplainable, ‘supernatural’) results in his research. However, hardly anyone seems to have noticed or cared and hardly anyone continues his research today (I know Jim Tucker continues his work directly and that other researchers in the past and present also looked at the topic, but overall it seems to me like awfully too little).

OK, fine, so we have established evidence of reincarnation and we’re ignoring them. Uhm, what?! Hello? Am I missing something? Did a respectable member of the scientific community say (for several decades) that he found thousands of cases that are impossible to explain by modern science, cases that should shake our understanding of physics and/or biology and/or psychology and/or whatnot, and the collective response is to ignore this guy? How can the relevant scientific community look itself in the eye? Isn’t this an interesting and important subject? You think proof that actually P=NP would be big? How about friggin’ reincarnation!

I’ve read Bill Bryson’s A Short History of Nearly Everything lately; it has been rather sad to realize so many amazing discoveries and theories in history have been ignored for several decades (sometimes centuries) before anyone caught on to the fact that they’re actually true and significant, often yielding further discoveries and further results. I’m not at all saying the fact that many true discoveries were ignored means that reincarnation is true only on the basis of it being ignored. But whatever evidence Dr. Stevenson (and others) have found should be refuted seriously, not merely shrugged off.

To some extent, things like this undermine my (otherwise very strong) belief in science as a whole. What other grand truths are already known to a few, and we just didn’t hear of them because we are all collectively waiting for a bunch of old Professors to die or for someone to pick up where a deceased researcher left off? What kind of price might humanity be paying because this avenue of research is ‘weird’ or ‘unsexy’? What if rebirthing is possible, and the obvious logical reservations1 are somehow solvable, and we’re just ignoring it?

Call me naive, but I don’t get why people like you and me don’t demand from our governments and universities that this issue be cleared sooner rather than later (and the issues of other, similarly odd scientific observations, while we’re at it; the hairs on the back of my neck still stand when I recall a lecture from Prof. Shulamith Kreitler of TAU’s department of Psychology about long-range sub-awareness and super-awareness phenomena, which to a layman like me it sounded a lot like bloody Telepathy and I think were never thoroughly researched by a Neurologist/Physicist combo, as I think it should have). I can list a ton of reasons why understanding of reincarnation may be important, but above all else – we’re humans, aren’t we? Isn’t curiosity reason enough?


1 Where do new souls come from as the population is expanding? What would happen if we nuke every living being on Earth, where would all these souls go? Do we share souls with aliens? How do the physics of rebirthing work, that is, what kind of particles pass from the dead being to the newborn one? Why is rebirthing evidence so much more common in the East, where it is far more commonly believed to be true?

Where Am I?

You are currently browsing the Uncategorized category at NIL: .to write(1) ~ help:about.