Introductions

For my first blog post to the Decision Frameworks blog, I thought I should probably introduce myself. My name is Jan and my main responsibility is the software development side of Decision Frameworks.

I love to develop software, and so I am in the admirable position to spend a great deal of time doing what I love every day. Most of our software development is undertaken in C# and is geared specifically towards the Windows platform. I have always been very interested in graphical systems and user interaction design issues – both of which are topics that are core to our current development tasks. But I will delve into more detail on those in future blog posts.
In my spare time I read a lot, ranging from rather technical computer science and mathematics books over business books (preferably business biographies) to fiction. On the fiction side I quite enjoy authors like Dale Brown, Tom Clancy, James Patterson, Chris Ryan, etc.

I have to admit to a bike fetish. I own three race bikes, a mountain bike, and various other contraptions on two wheels. The area that I live in (in Germany) is quite nice for biking, since I can choose quite freely whether I am up for an uphill battle or prefer a relaxed ride with almost no climbing at all.

On the subject of sports, I also run from time to time. I have never really enjoyed running – and can’t imagine that I ever will. But I find it a time-effective way to burn some calories in an attempt to stay in shape. Running also gives me the time to listen to the latest pod casts by other geeks out there – or sometimes I just use the time to churn through some of the tougher software development problems we face.

In addition to biking and reading, I am also in a perennial search for the best affordable home entertainment system. The next addition to that end of things is most likely going to be a Mac Mini. But I reserve the right to change my mind on that one.

I am also starting to develop an interest in digital photography. We own a Nikon D80 and enjoy taking pictures with it. We catalogue and edit the images using Adobe Photoshop Lightroom – which has turned out to be a great piece of software for the task. We’re still pretty early on the learning curve both in terms of actually taking photos as well as the software – but it looks like there’s fun ride ahead of us.

That’s probably enough information on me to get started. You’ll get to know me better in future blog posts. You can expect blog posts about software development, usually quite specifically about what we’re working on. Or, in fact, pretty much anything that comes to mind. I am working on a piece around project scheduling, as well as something around the next wave of .coms.

If you’re more interested in a CV or Bio, you can also look me up in the section about me on our website.

Stay tuned!

Gumball Game Epilogue

Friday, May 18th

Epilogue

As a follow-up to the CPDEP forum gumball quiz, I took matter into my own hands to see if I could create another opportunity to test Surowiecki’s theories….in this case, the results were much different.

I am the Cub Leader at Pack 641 in Houston and we had a year-end gathering at my house for the eldest boys (called Webelos) and leaders tonight. Families were invited and we had about 45 people show up. The ages of the group ranged from about 5 years of age to senior statesman status. People were approached to estimate the number of gumballs in a glass container. This container was smaller and more cylindrical than the one at CPDEP. Of all the attendees, 37 took a shot at estimating the total….some parents were scared away from guessing when they learned that the winner took home the gumballs…(I mean, what else are we going to do with 1337 gumballs?...my wife is concerned I may be on a first named basis with our dentist in about a year’s time if they remain within this household for another week).

When I charted out the results, I saw something that was very interesting.

There were 282 gumballs in the container, the average guess was 257….there were only two people closer, one guessed 275 and another guessed 295. Clearly, this showed that there was some wisdom here. There was only a 2/37 chance that any one person’s estimate would be more correct than the average of all the estimates. Surowiecki should be proud – in this instance, his theory holds up pretty well.

What this says about a conference full of decision analysts versus a pack of cub scouts…I will not comment on.

- Jeremy

Gumball Game Winner Commentary

The Jar and the Gumballs - CPDEP Forum

This game was spawned from James Surowiecki’s best selling book, “The Wisdom of Crowds”, which proposes a vastly different view to the popular belief that collective decision making may be flawed at times. Unlike Surowiecki, there are those who believe that group interactions tend to make people dumb or crazy, or both. Friedrich Nietzsche, summed up his views on group decision making when he stated, “Madness is the exception in individuals but the rule in groups.”

Surowiecki asserts that this statement is not true, contending that groups make excellent decisions when they are able “to maintain order and coherence”. He also asserts that “… groups benefit from members talking to and learning from each other, but too much communication, paradoxically, can actually make the group also be unmanageable and inefficient”. “The idea of the wisdom of crowds is not that a group will always give you the right answer but that on average it will consistently come up with a better answer than any individual could provide.”

With this in mind, we thought that this year’s CPDEP Forum would be the perfect opportunity to run a little test of Surowiecki’s theory. So, we bought some gumballs (red, blue, green and yellow) and a big glass container…and ran the game.

The Results

How many gumballs were in the container? 1337 Who was closest to estimating the correct number? Two people guessed 1350 …which was within 13 gumballs of the correct answer. To break the tie, we consulted the second question on the survey which read, “What do you think the mean of all of the estimates will be?” The winner is Fred Abbott who works in MCP Legal in Houston. In speaking with him about his estimate, he explained that he used his keen legal mind and some precise calculations to figure out his estimate – then use used his dad’s badge number from when he worked at Aramco ???? – ah yes, it is as good to be lucky as to be skilled. Please abuse him and his good fortune.
What was the mean of all of the estimates? 2160 What does this tell us? Well, for one thing it shows that in this instance “The Wisdom of Crowds” failed to prove out. There were 49 of the 78 people who played the game who were closer to the right answer than the average guess. The average of the estimates was 2160. The correct answer was 1337. The difference between the two was 823; 49 people provided estimates that were closer to the correct answer than was the average of all the estimates.

There are a few reasons why the test may have failed. Author, James Surowiecki, claims “…that a group will not always give you the right answer but that on average [italics and bolding added] it will consistently come up with a better answer than any individual could provide.” Perhaps this test was an anomaly and we should run more tests to find out? We plan to do this, using our website as the vehicle for the guessing to allow for broad participation….so, watch this space.

Maybe we didn’t have a large enough number of participants. 78 people submitted estimates. This may not be a large enough sample size. Perhaps if more people were to participate in the game, the extremely large estimates submitted would balance out and the average of the guesses would be closer to the real number of gumballs. (Ten people guessed 4000 or higher; two of those well-calibrated souls guessed 10,000, certainly skewing the mean of the estimates.)

Perhaps we had some very insightful spirits who figured out that if they guessed very inaccurately, then they could skew the average guess to being in their favor. It was never clearly explained that we would use the “mean” of all the guesses as the tie-breaker. Perhaps people reasoned that they could tilt the game in their favor with a large number estimate that would skew the mean to the high side.

After all, seven of the ten high guessers chose estimates of the mean that were below their own estimate AND Decision Frameworks’ presentation was sandwiched between two booths of Game Theory consultants. So, go figure, was their some game theory at play? Or, is there always some game theory at play?

The shape of the container might have had something to do with the over-estimation. It was shaped like a giant beer glass with a wide throat, a narrow neck and a broad base… a bit hourglass in shape. If people simply counted the gumballs they could see in the open top, and then multiplied the sum by the number of rows deep, they would have over-estimated the total.

Lastly, we tend to be better at estimating things we are more familiar with. For example, if you’ve ever rolled coins before, after a couple times, you become pretty good at grabbing a group of around 50 pennies to make a roll (adding or subtracting a few to make the roll complete)….but what do 100 gumballs look like? How many are really in that gumball machine at the car wash? In this case, this may be the first time someone actually saw 1337 gumballs.

In summary, there may be a variety of reasons that the gumball game fell short of proving the utility of the wisdom of crowds for estimation of uncertainty… because there was definitely wisdom in this crowd. (It was, after all, a decision quality oriented forum.) Interestingly, 50 of the 78 people had “means” lower than their estimates. Perhaps they already sensed that they were over-estimating the answer?

We don’t want to make any definitive conclusions because it was only one test; we’d like to run another couple of games and see if the results change.

Thanks to everyone who participated and congratulations to the winner (who has been sent an email announcing his sound estimating capabilities), and we will send you a note when another public estimating game ensues.

- Jeremy