Some of the conversation that happened last night at the Hudson Valley Programmers gathering made me think about reinvention. There was a fundamental search engine problem that a few people were working with, which seemed like a solved problem, with good open source tools around it. When I asked the question, there was a lot of dismissal on those existing solutions as too slow. That could be true. It could also be that the existing solutions solve a lot of problems that they don’t realize they’ve got to solve yet. Honestly, I don’t know.
It raise and interesting general question, when is it appropriate to reinvent a solution to a solved problem? The answer isn’t never, that leads to staleness in progress, as the solutions of old sometimes aren’t appropriate any more. But the answer isn’t always either, because that’s just navel gazing with no forward progress. My experiences with Drupal have made me lean a little more towards less reinvention, if only because my time is precious, and if I’m going to do anything interesting in my hobby time I’ve got to build on top of the shoulders of giants. After building a dozen password reset systems over the years, I don’t really find that that interesting.
It's always more complicated 
Quite a few basic computer technologies were developed in the mainframe era. Examples: virtual memory, multiprogramming, tightly-coupled multiprocessing. Each of these technologies took about a decade to get right, but all were mature by the time the PC was invented. Yet, each of these technologies was reinvented over time by PC people, apparently without reference to the already-established mainframe technology. Each took about a decade to get right — again. It was never clear to me why the PC folks couldn’t just transfer the designs, greatly speeding up their development. There seemed to be some sort of cultural divide between PC and mainframe folks, with the PC folks assuming they couldn’t learn anything useful from the mainframe world.
LikeLike
Having been there at the x86 reinvention of virtualization, I understand why it became new again. The hardware, until recently, didn’t support actual virtualization, so there was a whole lot of research and invention about how to do virtualization on hardware that won’t help you out. When I was working on that project I knew very well many reasons why the mainframe tech and approaches wasn’t viable.
That being said, I was still surprised that document indexing was being reinvented, again. 🙂
LikeLike
I’ve often wondered about this question myself, particularly at my current workplace where there is a lot of reinvention. Since I personally too often veer towards reinventing, I think of it in terms of “why are you reinventing this?” The answers usually come down to something like:
– I didn’t know about the existing solution(s).
– The existing solution isn’t completely what I need, to make it what I need requires an adapter or a fork or persuading the owner to accept my changes.
– Understanding the existing solution is time-consuming.
– The existing solution is flakey. Making it solid is time-consuming.
– I don’t like the smell of the existing solution (naming conventions, licensing, packaging, “I didn’t write it”)
Except for the “I didn’t write it” justification, you can present a valid case for each of these. Often, the amount of time involved in researching, modifying and adapting an existing solution is far greater than the time needed to create something simpler that solves your specific problem.
So my process is:
1) try to do due diligence in surveying the existing solutions.
2) try to get a sense of what is involved making the existing solution work for you.
3) try to get a sense of the cost of a home-grown solution (both in terms of immediate cost of implementation and long term cost of maintaining it).
4) justify a decision based on a consideration of cost.
5) finally, having done this, make the decision and don’t worry too much about it. All told, everything is always in flux, you can always change course and there’s very little effort that’s completely wasted.
LikeLike
There definitely no hard and fast rules here. I do find developers, including myself, are very quick to dismiss other folks code. It takes time to understand a system or code that was written by someone else, but if you only work with ideas inside your own head you never learn or grow.
I found this to be a good and humble approach to the whole thing http://www.girldeveloper.com/2010/07/your-code-sucks.html
LikeLike