THE INTELLIGENT SINGAPOREAN

Powered by the Plogosphere

Singapore’s Preoccupation with Statistical Nonsense!

Posted by intellisg on May 27, 2007

statistics.jpgTHE FRENCH have a joke that they direct at statisticians whenever they are caught in the Parisian traffic jam. It goes something like this: “It may work fine in practice, but I don’t think it works very well in theory. So why don’t we make our lives simple and swap the theory with the practice? Besides, no one would ever know.” I am sure it’s supposed to be funny, if something wasn’t lost in the translation, only I am not laughing. Statistics after all litters the world and its serious business.

Lately (and I could be obscure again), I have noticed an increasing use of statistics creeping into the main stream media. This doesn’t just include newspapers but also TV and the radio especially. It’s used to explain, justify, compare, substantiate and drive home everything from selling cat food to suggesting why you should forgo plastic bags and instead used a carrier when you next go shopping. Recently salaries for Ministers were revised and what did the government use to justify the pay rise? Correct, a comparative statistical survey which highlighted the pay disparity between ministers and corporate honcho’s in Singapore. What about the survey that list Singapore as the most competitive country in the world just because we shuffle faster than anyone else. Again statistics was used, and. Most recently Mr Warren Fernandez’s a reporter from Straits Times wrote an article on 19 May 2007 entitled “All societies have elites, but some become elitist.” Where he explored the issue of what students considered to be the most important factor for succeeding in Singapore and to establish his plank again he conducted a poll using a type of statistical analysis. He wrote:

“Only 2 to 3 per cent of those polled from all backgrounds cited family wealth as a major factor in someone making it to the top..”

Now I just want to tell you all that I have nothing per se against reporters, ministers, corporate leaders, businessmen, crow shooters, circus acrobats or even prostitutes using math & stats to make sense of the world. I do it everyday in my job and I wouldn’t be considered prudent or wise if I decided to skip the number crunching step – but you know what? I deal with nuts and bolts, that’s to say the statistical tools that I typically deploy to get the job done. Whether it’s using a simple Likert analysis to determine cumulative productivity – or a state of the art 3D regression simulation to profile how much resources needs to be directed to a task – isn’t related to living creatures. There’s a very good reason why I don’t see the wisdom of transplanting the science of statistical analysis to people. Statistical anything is just plain “hit and miss,” misleading and sometimes deliberately distorting. To quote Mark Twain who summed it all, “untruths” generally fall into 3 definable categories,

“Lies, damn lies and statistics”.

In fact to save me the trouble of writing this lengthy yarn, I am prepared to wager if both I and Dr Chandra had 45 just minutes with Mr Fernandez right here on the IS and we posed a few pointed questions like: How was the data collected? What questions were asked? Who asked them? Who answered them? What time was it? Where was it conducted? How many respondents were there? Who selected them? Who was present in the interview was in a group or was it done in private? How many minutes were the students given to reply? What was the median age? Were they good, average or poor performers?

We could even re-write the whole article the other way round. I am not joking, I am dead serious! That should roughly give you all an indication of my disdain for using statistics to make sense of people. Let alone deploy it, to draw any conclusions beyond the per functionary, such as writing an article about whether bunch of students think running around the padang three times stark naked actually improves the chances of succeeding in life!

Having said that, I don’t for one moment dispute, the world is swarming with statistics, and the average Joe is bombarded with at least 10 statistics a day, but just because a practice continues doesn’t necessarily make it right or logical. I am reminded witch craft like cult churches in shopping malls run by crooked pastors who drive sports cars still continue to attract millions of adherents, but how close are they to the truth?

Central to the whole debate of using statistics either to write articles or to justify a position, be in politics, business or even in a family setting. One really needs to ask: can statistical results i.e data – be equated with knowledge? Especially when it’s directed to people, the wider community and the broader populace. It would appear so judging from the way it’s regularly used to shore up arguments, to explain and account for events and developments in politics, economics, sociology and technology. The experts of course would have you all believe – they’ve pinned it right down to a fine science. They even have a name for it; “management by statistics,” the science of rendering “clear and unambiguous” whole groups of people. It’s often used for social planning, housing to something a mundane as calculating the optimal number of urinals and shit pots one should ideally incorporate in toilets in bus interchanges – its serious stuff! Without it there would be no relief.

Indeed judging from the inroads statistics has made into management in both the private and public sector in Singapore. It may be a palpable truth it certainly represents a valuable form of data which can be effectively used to enhance the lives of millions of people. But despite the seemingly infinite range of new possibilities statistic analysis offers. A serpent lurks in paradise. While statistics when used prudently is effective working for our greater good, if we are not mindful it can also threaten to confused and destroy us. That’s especially the case when it is taken to extremes where data acquisition is promoted as an end by itself – rather than delineating its usage to being only a means to an end.

Usually the abuse of statistical tools occurs when people unwittingly or knowingly choose to draw what statis typically call, “a straight line zero margin for error correlation,” between data and events – that sort of simplification turns their results into unadulterated dribble.

Just in case you think. I am running around in circles trying to whip up another shit storm. Consider the population control measures instituted by the Singapore government during the early 80’s – when technocrats in an attempt to manage the population growth resorted to using statistics to prescribe a solution in the form of the, “two is company, three is a crowd” program. Result: something resembling “social Chernobyl” when the misguided use of statistics especially when it’s directed in an unimaginative straight line fashion is directed to something as complex as people.

Fast forward to present day, what problems has this policy generated? Can we say today much of underlying the reasons accounting for the lamentable baby rates, stems directly from this failed social policy to take stock of the longer term ramifications? Is it economically and socially viable to reverse the mindset? Its something to consider, if you think statistical analysis is just harmless bean counting – it isn’t! It’s hazardous to a countries health if you get it wrong! And it affects millions of people like you and me.

That’s what happens; when statistical tools become the sole means of promoting, justifying and implementing an idea. That’s only valid if you define humanity as a simple matter of data processing. Or if you believe all humans can be reduced to roughly understandable predictive aggregates – they cannot, trust me, even if the say “yes” today, they stand an even chance of saying, “No” tomorrow (and to add to the confusion a “no” is a “yes” in the case of girls) and “maybe,” the day after – people are tough to figure out.

As the population control or eugenics program which once failed and came back to bite us showed only too clearly; using a single track unimaginative tool is usually the problem rather than the solution. That in a nutshell is why I don’t like the whole idea of journalist, politicians or businessmen using statistics to come across! It’s just dumb!

As I mentioned earlier data by itself is worthless and drawing a straight line to derive at a conclusion is even worse unless it attempts to coherently link all the ancillary factors to enable it’s possessors to make sense as to what’s actually happening. And this brings us to next question; if statistics are really so useless when it comes to managing people why is it used so often then by banks, governments, hospitals and the entire bureaucracy? Surely darkness you don’t expect us to believe that statistical tooling should be abandoned. You can’t be serious can you? After all how would we plan, control and command?

Plan, control and command? Yes that’s possible if you are trying to figure out how many workers you need to produce X number of battery operated dildo’s in Y number days with Z resources. But when it comes to people its all together another thing. For one people can work around almost any numerical target to make it look good. The last regime who tried out this great experiment and failed gloriously in the name of the motherland was the Soviet Union who realized to their dismay. When Soviet statisticians measured the output of their glass factories in tons, they ended up with mountains of warehouses of glass that was too thick. When they measured them according to square meters, they made them so thin it was impractical for long distance rail transportation. That’s what happens when you try to use statistics to measure or manage people, they get smart, they work to the target not the task as a whole! It’s known as Goodhart’s law – a former banker – who discovered it by chance and it’s a sobering reminder to anyone who decides to play tinker ;

“Any statistic will eventually become distorted if used by authorities to control the way people behave.”

The consequences of pinning down the wrong factor statistically can be catastrophic, as the Pentagon discovered in the Vietnam war, when they audited the success of military units by the body count they achieved. Result: terrible loss of life among the Vietnamese, but no American victory. Thereby proving the military adage truer than true: Win all the battles and lose the war! Sucker!

At this juncture you are probably wondering, we have reached one of those strange paradoxes – a forked road even that throws out the question: is this the end of the road for statistical usage? Because from what you telling me darkness, it’s useless!

Correction: I didn’t say statistical analysis is useless, I am just restating the reality that like checklist doctoring, checklist accountancy, checklist welfare, checklist dating, checklist Kamasutra or checklist personality test remains frightfully “hit and miss” and this simply brings into sharp focus the need to continually look deeper at the real linkages which bind people to events and circumstances. The allure of statistical analysis is insidious, because it offers an easy road map of making sense of complexity. But again, that only holds true, if you believe everything can be reduced into simple Simon sound bites – the truth suggest otherwise and let me just give you another real life example to illustrate what I mean.

Recently, I came across a CV for the prospective hire. The computer program which was designed to facilitate the job application detected an anomaly which indicated that he “wasn’t suitable” because there was a four year gap in his working history. During the pre-hiring meeting, the HR Manager and even the hiring manager continued to insist he wasn’t suitable because he hadn’t worked for 4 years. So of course being the spoiler that I always am, I asked the question, “Did anyone ask him? Was he abducted by aliens?” There were a few lame retorts like, “the computer rates him as ‘unsuitable’ or more accurately a ‘reject’” To which I answered, “that’s why we humans are having this meeting stupid!” To cut a long story short, I invited this man over to my place for some cold beer and a dip with his family and it didn’t take me long after saying hello to his 5 year old toddler, to realize he was autistic. I needn’t have asked more and the next question, I found myself asking was whether: this man who a machine had rated as a “reject” had made the right management decision to give up his career to perhaps take care of his son that needed him more than a company producing toilet seats! I took me about the 5 seconds to make my mind up.

The reason why I am sharing this excerpt is to highlight what typically happens when we reduce and relegate everything to just numbers and cloudy equations; something gets lost, a line gets rubbed out, someone falls through the cracks and we just end making Byzantine sized mistakes that we hope don’t come back and take a swipe at us!

There is nothing wrong with checklist, using computers as an aid to hire people or using a poll to write a story, anymore than there is with measuring or counting stuff. Quite the reverse – they remind us all, even most diligent amongst us can miss out on things. You can even say statistical simplifications, lends an analysis as certain degree of objectively by filtering out our own prejudices. The danger is when in the name of either securing the premise for a pay rise or wordsmithing an article, we forget that checklisting, polling or statistical tooling typically misses out those events or circumstances that go beyond the ordinary. Instead it requires a higher level of discernment which leverages on; intuition, judgment, experience, common sense, love, relief, truth and above all what it means to be a human being.

Once people and their broader history are reduced into numbers – it simply cuts off the opportunity to retrace the 1,001 small linkages that forms the bigger picture. The fact remains people are terribly complex, so at best all these statistical tools are only vignettes offering very narrow peeks and at worst cues rather than whole seminal disquisitions. To suggest statistical analysis should for one moment be the ratio to substitute substantive debate and analysis is to fail to consider the Faustian price of knowledge – that the power to create also harbor the same benign forces to blind us completely should we wield it unwisely. There in the palm of your hands lie the perils of using statistical methods for anything!

May I take this opportunity to wish you all a very happy week ahead!

Darkness 2007

(By Darkness – Socio/Political – EP 9939378-2007 – The Brotherhood Press 2007)

Advertisement

16 Responses to “Singapore’s Preoccupation with Statistical Nonsense!”

  1. scb said

    Statistics at their best is figuratives and figuratively speaking!

  2. also singaporean said

    ‘There are lies, damn lies – and statistics.’

  3. caleb said

    statistically ppl like Bill Gates and Steve Jobs dont even exist. I think one valuable lesson that I have learnt from this write up is statistical methods are usually preferred because they offer an “easy way of explaining the complex.” I somewhat agree that it is a very unimaginative approach the sidelines many important considerations.

    That is why I believe real journalist covering the news in Iraq embed themselves with the US forces. The whole idea is to go through what the soldiers themselves experiences to inject a higher level of reality into their write up’s. Perhaps Warren should have done the same thing, spent time with those kids, interact with them and their family, try to get deeper beyond the hi and bye’s – I agree, but thats a tall order in Singapore.

    For one creativity and innovation isnt held in high regard in SPH, the only thing they really care abt is quantity and not quality.

    Thanks.

  4. scb said

    Warren Fernandez and Chua Mui Hoong are bolt and nut, parts to the Machinery and 100% elitist!

  5. lazysusan said

    Hi,

    I think Chua Mui Hoong is quite a brave and courageous girl.

  6. lazysusan said

    Bambi Bad Boy Darkness,

    I dont think they will take up the challenge with you and dr chandra.

    You see bambi, they need to maintain the illusion they are credible and competent, like our govt, it simply means they can never admit their mistakes. That I believe is why edb, mti, shin corp have remained silent. Say nothing and hopefully everyone will forget and it will go away. That of course also means they will never ever learn from their mistakes. So nothing really ever changes Bambi. They will keep on making the mistakes and passing them off as doing the right things. As time goes by, we wouldnt even know what is right or wrong any longer.

    This just makes it all right. This just resets many of the things that has been bothering me for years, but I just couldn’t put my finger on it. Thanks.

    Susan Teo

  7. As It Is said

    I spent two years reading, studying, researching and experimenting with Statistics and Statistical Analysis as a major part of my MBA program. I had realised then that statistical data can be and have often been manipulated to influence a particular, singular, outcome in the decision-making process to serve the user of that particular set of statistical data. After so many years have past, I still maintain the same conclusion.

    So, it is wise to be very cautious to accept any opinion in order to make judgement or form any conclusion on the basis of statistical comparisions presented by one single person or one single group of person, or even a single organisation.

    In today’s world of the get-rich-quick and get-famous-quick rat race, there are many shrewd, crafty and cunning deceptions and frauds committed by highly educated, highly influencial, gentlemenly-looking and seemingly honest and trustworthy personalities or authorities who have hidden agendas for their personal gains, self-interests or group-interest.

    Statistics don’t lie but people do. When statistics and people are mixed together, what is the outcome?

    Beware of the Great Liars of Statistics!

    And remain:
    AS IT IS.

  8. scb said

    Statistics are like rubberband too, they can be stretched and often get twisted by the manipulators(the statistics provider/not necessarily a statistician)

  9. anongal said

    Actually, I am usually quite a non-confrontational sort of person. Often preferring quiet diplomacy to open conflict. But I will give anything to see a show down involving a tag team of Bambi & Co VS The World!:0

    Regrettably, I have to agree with Susan. They or rather we have to maintain the illusion.

    Great piece and I think much required especially these days when so many ppl seem to just believe without questioning further. I think this piece goes a long way to set the record straight.

  10. techme said

    Our newspapers are full of damn lies and fake statistics that are manned for many damn stupid Singaporean. No joke, after speaking with so many Singaporean, many believe only what newspaper says, afterall, they say gov don’t lie. And as long as gov do anything at all, they remain credible, so these ppl believe.

  11. shoestring said

    I have been waiting for this for a long time. Hats off to all involved in this piece.

    So, is globalization and all that it seems to necessitate 100% real?

  12. techme said

    All the gahmen take the easy way out since they don’t to be responsible for anything at all even though they are paid highest salary in universe.

    Just blame everything on terrorist, globalization, competition, stupid and dumb Singaporean, lazy and crude mentality Singaporean.

    So simply blame everything except themselves, and still nothing will happen to the gahmen.

    Gahmen have taken Singaporean for granted since they know very well chicken and mammy Singaporean can’t do any fuc32#@$@$#king at all.

  13. Stats said

    There’s an excellent little booklet by Darrell Huff called “How to lie with statistics” that shows up the kinds of statistical twists and fudges (deliberate or otherwise) that politicians use to spin data.

  14. perm sec said

    Brilliant!

    This is definitely an inside look that most people are not privy too, the conspiracy of numbers. I luv the soviet analogy of the glass factory, tell me do you happen to have the reference or attribution? It would help immensely, if I need to give a speech just to illustrate the point.

    Some of you boys, especially Darkness sir, should do well to consider having a spot of tea and biscuits with us one day. There are many things, we can discuss and I am sure, if we care to square off the accounts – we may even see eye to eye on most issues.

    Till keep up the good reads and it’s always a joy and a pleasure to come over to the IS.

    Do keep up the good work. Jolly good!

  15. spursfan said

    Nice article Darkness.

    Fact: 97% of the MRT trains arrive on time.
    Observation: The working population is delayed during rush hour, either (1) on the way to work or (2) on the way back. Do not hold your breath for the study that determines whether it is (1) or (2) that occurs more often.

    Fact: Less that 15% of the domestic economy is taken up by Government and Statutory Boards
    Observation: Many private business owners engage in non-airconditioned food & beverage industry/air-conditioned transportation industry.

    Fact: 100% of statistics involve some form of numeric language/algorithm
    Observation: Hardly any of these statistics add up.

  16. Lassie said

    Aiyo, when I read this I said Err OK maybe……..Then I saw this,
    Blogger exposes rent error
    Posted: 25 July 2007 1007 hrs……….and they wonder why advertizing money is going somewhere else.

    How many times have they mislead us with statistics simply bc they can even add? Or worst still cannot be bothered to check! I am glad there are really eagle eye ppl like Alex Au around, it time the ST realized the gala days are over.

Sorry, the comment form is closed at this time.

 
%d bloggers like this: