So okay… it’s Friday night, and while all and sundry are out making merry, I’m going to sit in and try to articulate some thoughts on the tension between theory and practice in Software Engineering. Never let it be said that I don’t know how to rock out…
I got my start in programming in a very practical and unglamorous way. I was working in a data analysis role a few years ago and the other team members had set up regular processes using VBA in Access and Excel, so I learnt how to hack my own modules together in an inelegant but more-or-less effective way. The code I produced was atrocious by any objective standard, but what it showed me was that a basic grasp of syntax could already make me far, far more productive. Oh and also that running Excel macros with screen updates turned on looks awesome.
I fairly quickly became a lot more serious about learning both languages/ tools and the underlying theory. Something just clicked with me about applying fundamental rules and approaches to solving technical problems, building up on a physical level from bits and bytes and on a logical level from predicates, transformations and carefully defined relationships that attempted to model the real world in code. I find it very rare nowadays that I come across a piece of technology for which I don’t want to know the roots and the implementation details, and I spend a lot of time trying to prioritize the many, many excellent books, articles and academic papers that demand my limited time and attention.
So yes, the direction of travel for me has been almost exclusively towards greater theoretical understanding and in-depth knowledge. I hope I’m never arrogant enough to imply that I know it all, but I definitely find some frustration in situations where other people are not so much unable as unwilling or uninterested in following the same path. There’s a flip-side to that approach though, and developers like myself who take learning and personal development seriously need to be careful not to lose sight of the bigger picture.
I absolutely don’t want to name names, but I was just reading an article from a year or so ago where one SQL expert of some years’ standing was commenting on the work of another venerable guru of the database movement. Both men are clearly immensely smart and have contributed a lot to the field, but the tone and chosen topics of dispute left the unfortunate impression that one if not both of them were more interested in a) their own particular, very specific mode of data representation and manipulation and b) scoring points, than in making their ideas understood more widely and helping other people to work with relational databases.
I take this seriously because I recognise it in myself. Even as a working programmer, it’s incredibly easy to get sucked into this kind of intellectual one-upmanship. If I’m completely honest, some part of this attitude has its place. I’m sure the vast majority of professionals in our industry have experienced the constant pressure to deliver what the business wants as fast and as cheaply as possible, piling up and ignoring technical debt until software becomes hard to maintain and risky to change. We do have a responsibility to apply some polite-but-firm pressure in the opposite direction, insisting on standards, software craftmanship and investment in creating a quality product that will stand the test of time. An obsessive focus, pride and attention to detail are necessary parts of this.
The point though, is as follows. We need to be working with, not against, the businesses we are a part of. We need to be responsive to their needs and understand the way in which every function of the organisation – technical and otherwise – is dependent on every other part in order to operate. It isn’t selling out if we accept that building something great, something that has a real impact, takes compromise and the maturity to understand other people’s points of view.
Whenever you came into software development, even if you’re decades in to the game and were firing your way through recursion and pointer arithmetic in your early teens, you must be able to think back and remember when you had no idea how a computer worked. To all of us, born in the shadow of Alan Turing and the early greats of Computer Science, they were once just magic boxes that did amazing things. It’s important to remember that that perspective still makes sense – the things that computers can do have the greatest value because they can impact everyone, not just the smartest, most technically clued-up people in the room. By all means, get stuck in to an intense debate about the relative merits of your favourite language, framework or feature (in fact, CC me or send me a link – I’m sure I can learn something from it) – but do remember why it is that anyone else takes this stuff halfway seriously in the first place.