code and the oracular

Over-abstracting -> bullshit

leave a comment »

Amidst the chaos of so much important activity in the world right now, particularly in the middle east, I sometimes find it hard to reliably concentrate on the immediate task. I am in the middle of working on a complex website, and earlier today had problems. But reflecting on these I was struck by a political analogy. Trying to start a web development project and having a problem setting up Apache is after all like trying to get to a business meeting but having to fix your car on the way – you get this unpleasant feeling of being sidetracked ! But sidetracks are always there in unstructured maelstrom of real life away from the keyboard, where bosses or pupils need attention, and communication is so critical but ever impossible to perfect.

The general impression I have received about this new idea of “The Big Society” is that no one is quite sure what it is supposed to mean. This maybe because of various errors made by our prime minister, in which he has failed to be a very good logician, I’ll try to communicate this idea.

In computer programming we have a very important concept called abstraction. Many people will be familiar with the contrasts between the particular and the general in practical daily life, so I’ll start with an analogy. In business a greengrocer might broaden her market share by starting to sell sweets as well as veg. She is making her operations more multi-purpose. If she then starts to do a stand of newspapers and other basic provisions then its easy to see that she is generalising away from the specific of veg into a range of different product categories, but ones that can still be run using her initial set of “selling retail goods” skills that she learned as a grocer.

This process is abstraction, in computers we often start with code that processes a unique set of one-off data. But then as the task matures we decide slowly to allow our program to work on different sets of similar data and then perhaps later different formats of more different data… and then later again design for different platforms or different user styles. That way systems are portable and more marketable too. By making our systems more general in what they can handle we save work by enabling the transfer of technologies from one domain to another. This is also the power of mathematics which is comfortable with facts like “3 apples plus 2 apples makes 5 apples” which is plainly amenable to the generalisation that “3 of anything plus 2 of the same thing = 5 of that thing”. indeed numbers are the very most general of all concepts.

In programming we also have another useful concept which i’d like to bring in to this discussion which is that of levels of description. This concept is related to abstraction. It is really about building blocks. So much of our work is about algorithms, and every algorithm has a description. these days the description however becomes amenable to a kind of shorthand. If I give you instructions (like an algorithm) to make me a cup of tea, I would usually leave out all the minor details like “first make sure the kettle is plugged in at the wall” since I would assume you didn’t need that kind of level of task verbosity. indeed you might find it condescending to receive instructions like this because we all assume an ability to solve such simple problems. (On another angle try getting a robot to do this though…)

The computational “jacob’s ladder” of machine code to low level code as the bottom rungs and higher levels of abstraction on the way up is a part of the very fundamentals of how to build complexity in our systems which most computer programmers know a good bit about. I sometimes wonder about trying to start a kind of open conspiracy where us geeks buried up to our elbows in code and data are finally recognised as the ones who should truly be in charge. To me a government is a complex system, and an inefficient bureaucracy perhaps something like a ‘legacy system’ written in some dinosaurish and inefficient langauge like COBOL. Mastering and organising the huge data that comes to you must be one of the parts of government, and correct planning based on this data sounds like a classic AI problem.

Much good human communication is very high level in terms of its abstractions but there are different styles for this and arguments we disagree with are often built upon philosophical “platforms” we dislike and thus may deem “buggy” like bad software, while aficionadoes will defend their own chosen philosophical technology. An individual’s political views are often like this, left and right wing each being like a different operating system, say windows, mac or linux. But the lesson of the sofware analogy is that these turf wars are more futile than trying to build common ground. Standards are everywhere and are usually seen as progress because they overcome the Tower of Babel effect and let us or our systems talk.

Maybe this is why some of us the find the pat language of gurus and charismatics so transparent, unless of course we are under their spell, it is because they utter such powerful sounding (but ultimately insubstantial) generalisations that masquerade as wisdom but being devoid of ay specific examples will soon fall down in the messiness of the real world. This could apply also to the nigh-universally applicable dicta of horoscopes but frankly I find it applies best right now to the words of David Cameron in his talk about the “big society.”

The big society sounds like a kind of ultimate high-level concept, a vision that might guide us. The lesson though from programming is that abstraction is always leaky and thus the grass roots of ordinary processes, think of a kind of societal stream of ones and zeros rolling dwn a matrix like greenscreen, is the only place to start. But this is something that Cameron doesn’t seem to be very good at. Indeed to insulate yourself from the rabble is a part of such a leader’s classy image, and is counterproductive in the same way as a programmer who doesn’t study their data enough.

With establishing new skills, especially in complex areas, finding your own comfortable learning path that leads in to the subject at hand is crucial. what is your personal way or style with which you solve this problem ? Some programmers take existing pre writtencode from libraries on trust, others favour a lower level and build up from the basics, while the others say they are wasting time reinventing the wheel. It’s an open debate but an interesting one because I have always felt that reinventing the wheel may mean you come up with a new original approach. If you want something done properly do it yourself is of course another dictum that casts different political styles into relief. Again we programmers have to deal with these issues daily. I’ve never set up Apache on Ubuntu before for example… so its a trip around the web techie forums for me later, having of course to try and sift the incorrect how-to data from that authored by the real experts, some of whom are modest and don’t blare their knowledge from the rooftops or make public party broadcasts !

Learning to encode so much data in textual or tagged forms has been great fun, with general skills and aha insights being acquired all the time to keep my sense of motivation and accomplishment sweet. I want to follow my open-source government ideas more because we all should have the freedom of information (or the wiki-leakability) at heart when designing the complex code of nation-management, and progress comes through sharing our code also.

I wonder what are the evolutionary algorithms that we and our brains use in all this? Or whether what the brain does is even algorithmic? In writing code we learn to manage big complexity consciously and intentionally but the magical subconscious processes in the brain are doing that all the time without us knowing. Indeed the fashionable current insight is that object recognition (where’s that kettle?) is a much greater and harder act of cognition than equation solving or chess… Our leaders are often past masters at subconsciously guiding themselves up the slippery pole too. The compuational wonder and richness of ordinary life acts against this though, maybe degrees in high falutin economics are less satisfactory as skills than hard nosed experience in running a greengrocer’s. Such intellectual elitism is partly marketing and branding these days, I’d trust the data.

Recognising glyphs or letters is at first a game of geometric pattern recognition to the learning child, but later we view it as linguistic. are programming languages like natural languages, yes there is still logic, semantics and syntax, many similarities. All of what we do can be termed “General Intelligence” which so many boffins around the world are dreaming of implementing in silico as “Artificial General Intelligence” right now.

A database saves memory by reading rows or columns one at a time, must be like this or none of our favoruite webistes would work. SQL – Structured Query Language is again a generalisation of how to phrase questions of any simple type, this lies beyond merely computer databases and is an abstraction of what all data retrieval processes share in common. Code clarifies thinking skills, UML – the Unified Modelling Language is also a generalisation of modelling any process / structure visually in pretty diagrams or textually in a language. The formalisation of good coding style or notation is a generalisation of any social knowledge sharing process where we need common protocols. What should we take on trust – lots usually! But we must also have a “smart trust module “that involves wisdom and skepticism too ! It’s only good news when that kicks in in trying to understand what our government is saying to us.

Was complexity built in the brain in the same way ? kind of “lower animals think in machine code”. and moreover in this kind of complexity you need to take many code objects on trust. the very use of natural language is a social act, our definitions and concepts are part of mental software libraries whose origins lie in past history.

When we or often our children ask some questions like “why do birds fly ?” one hallmark of a credible answer is often to use abstraction. (I had to use this in some Prolog code I recently wrote). In programming we might say that “animal” is a superclass of “bird”, an explanation comes: “because all animals have means of locomotion, and a bird does this by locomoting through the air, that’s why they have wings” instance -> class -> inherited property.

This explanation sounds good because it is bandying around an abstraction and referring to that instead of the actual case under question. How detailed after all do you want your answer to be ? Completeness wants evry relevant fact but so often the constraints of soundbiting do limit us. So many politicians do this generalisation trick, but its more psychology than effective explanation, because we think that the (often academic) categories of knowledge are set in stone and a matter of weighty authority.

Us coders are natural radicals though because we have to respect what works rather than what the theory says will work, and this often means drilling down to the detail of individual cases to get a real feel for the data. This is like grass roots politics too ! I just wish our leaders would do this drilling down more often and sample the raw data of ordinary peoples’ lives a little better before they think they have captured that great abstraction in the sky !


Written by Luke Dunn

February 25, 2011 at 6:58 pm

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: