Question for software developers

In My Humble Opinion
1

So I work for a software design firm where we build small-to-medium sized web applications (using PHP) for various clients and stakeholders. I’ve worked for this firm for well over a year, and I enjoy what I do (mostly). Prior to joining this firm, I was a solo developer, so having to develop software as part of a team is something new to me.

Here’s my question:

What is it about our industry that makes it so dominated by otherwise very intelligent people, who feel compelled to take fairly ordinary, simple tasks, and turn them in to complex, convoluted, time consuming nightmares?

Why is it that when it becomes a complex, convoluted, time consuming nightmare, they will still keep singing its praises?

Here’s an example. I’m currently Lead Dev on an smallish web application for a client in the health industry. The app needs to allow users to set certain goals, rate themselves against those goals, allow mentors to log in and leave comments and feedback, etc. It’s basically a form with maybe 100 fields, sitting behind a log in screen.

My support dev, (new to the company), couldn’t believe that I didn’t use an Object Relational Mapper (ORM) to handle my database querying and data modelling. When I asked what the benefits were of using an ORM as opposed to not using one, he described benefits that were at least ostensibly clear (less data replication, easier data portability, etc).

I weighed the pros and cons and decided I was willing to let him try and “wow” me with this new method of handling data, so I left it up to him to write the main data file that these ORMs eventually use to generate databases, classes, etc.

Fast forward 7 working days, and he’s barely got the ORM configured, talking to the framework, and executing some basic data inserts and retrievals. This afternoon we had a meeting so that he could demo his ORM work to me, and he couldn’t even tell me how this particular ORM handles slightly more complicated tasks such as join queries or updates (as opposed to inserts). This was the meeting where I thought he was going to show me a fully-fledged ORM “making life easier” for us and the project. But I shouldn’t be concerned, because joins and updates will be “easy” - he just has to go and spend even more time figuring out how to do them.

Given that’s he’s spent 7 full days on just trying to learn and configure the ORM, I couldn’t help but think to myself: In 7 full days, I could have not only written every SQL query needed by this app 10 times over, but I could have basically completed the entire application (minus the page styling, which we palm off to a designer). But when ever I try to hint this sort of thing to the support dev, it’s just met with lines like “But can’t you see how much better this is?!”. And he’s not even trying to have a lend of me. He genuinely believes what he’s saying. He must be seeing something that I just can not.

On a prior web app in which I worked as co-developer, I let my more experienced colleague take the helm and lay out the core application structure… “Fully OO” he tells me. Fast forward 2 weeks, and we’re still struggling to get the thing ready for the first round of testing, as we struggled to navigate our way through the endless bound of objects, classes and inheritance that was going on.

It was at this point that it occurred to me: “This web app is basically 5 forms with some business rules thrown in. If I did it all in pure procedural, and laid the code out in a way that I deemed to be sensible and modular, I could have it working in less than a day”. I went and explained this to my boss, who gave me a day to prove it. So I did. I built it. In a day. And demo’d it. I was immediately promoted to lead developer, and the project was a success.

But here’s the thing. My co-developer on that project, to this day, continues to tell me how much “better” his application structure was. Because if we did it his way, the application would be much more “scalable” and “reusable” and just “better” (we have run in to no scaling or reusability issues with the app since). And he’s not trying to deceive me. He genuinely believes what he is telling me.

I don’t want to sound like some sort of programming genius, because I am not, in fact I am miles from it. But it just feels like every time a new project comes up, it’s like some of the other developers engage in a contest for who can come up with the most insanely over-complicated way of producing the solution that they can conceive.

It’s frustrating, and I don’t like constantly having to tell my support devs (and sometimes co-devs) that I am just not interested in any fancy new cutting-edge development technology unless they can clearly describe in less than 15 seconds what the supposed benefits are.

I promise this is not a brag, it’s more a plea for help, and a request for advice on how to placate my support devs who take issue with my fast, lean programming style. Is it just where I work? Or everywhere? Or am I really the dill? Thanks.

2

Seemingly intelligent people often think they are clever. In exercising and indulging that trait, they tend to ignore the first part of Einstein’s adage: “Make everything as simple as possible”, with nary a chance of reaching the second.

And it really does seem to be a characteristic of the industry, as exemplified by the tale of toaster design. IMHO, you don’t ever really placate your co-workers…you just repeatedly get shit done in minimal time that works, and then they learn to listen to you.

It took me years to train myself to shy away from being clever for clever’s sake. It’s an ongoing effort.

3

It’s more enjoyable to code the latest cutting edge idea than it is to do it as simply as possible using the ‘old’ techniques. It’s a challenge and you get the pleasure of learning something new and who doesn’t love that?

4

Some technologies make development faster.

Of course, that’s if you already know the technology.

Some people can learn new stuff fast.

Of course, there’s no way to know which person can or can’t until you’ve seen them try a few times.

Some technologies are only a catchword because they have a big company/consortium promoting them, and actually don’t make development faster.

Of course, there’s no way to know whether it is or not until you’ve tried it.

5

Developer here. The advantage of an ORM over raw handwritten queries is reliability and testability. Complex, handrolled queries to create, read, update, and delete are notoriously hard to test, especially when they are more complex than “insert into stuff (stuffid, stuffdesc, creatinguser, createddate) select newid(), @crap, @UserId, GetDate()” and “update junk set gizmo=@myNewVal where ID=@id”. Source: personal experience. You don’t want to see some SQL queries I’ve had to maintain.

6

OO is vastly overrated, most implementations actively encourage coding styles that break multithreading, and god help you if your language doesn’t also provide functions/closures, because then you’ll probably quadruple the number of classes required for a “flexible” API.

ORM can be useful; it can help provide an additional validation layer, it can help write/generate complex queries for you. If it tries to do more than that, it’s usually broken in some way. I’ve never seen an ORM system that is 1) fast, 2) easy to set up, 3) predictable and 4) reasonably complete. And all of them fundamentally break the relational model.

7

How so? I say that explicit SQL sitting right there in my model, in which my eyes can dance over the actual queries being executed against the database server, is far easier to test and especially debug when it doesn’t work as intended.

As for “reliability”, I’d say ORMs are only as reliable as the control files used to define the data.

How so? Why are they “easier” to test if using an ORM?

8

I think it’s a little bit of everything that people have mentioned above, along with a strong tendency to not fit a solution to a problem.

I see this all the time - a developer reads about new technology <x>. It’s going to change the world! It’s so much easier! We need to do everything using <x>!

Then what happens is one or both of the following:

  • new technology <x> is in its infancy, barely works, and is barely documented. Two years from now it might be great, but right now it’s taking hundreds of hours to figure out how to twist it to the task

  • new technology <x> is great and it works, but its benefits are primarily for large projects that involve a group of programmers and testers. As robert_columbia points out, things like ORM really shine when it comes to testing and reliability, but the overhead of putting it in place is much higher. If you’re writing a 5-screen app that can be tested by one person in 30 minutes, the time spent learning and implementing it might not be worth it.

Getting across the idea that the overhead of a technology might not be appropriate for a small project is something I’ve struggled with for a long time. Some developers just can’t get it out of their heads that a certain technology is always appropriate for all projects, regardless of size, scope, or audience. Just the learning curve can be tremendous - “sure, YOU know Microsoft’s new MVVM framework, but the 3 other developers on the project can barely do WebForms and we simply don’t have 3 months for them to get up to speed, much less productive” is a hard sell to some people.

There’s also the Magpie Developer mentality to get around.

9

Oh god yes. I am not now and have never been… but I could hit a couple of them with a piece of fruit without leaving my chair.

Newer technologies don’t always make development easier. On a recent project (where I was a tester) I wanted to add a new database access field to a web page. This involved editing six different files, one to tell the ORM about the database change, fair enough, two add the get/sets to the ‘backing bean’, three and four to add the actual element into the page and the java class, five and six may have been to do with the locking policy and validation, I forget. The IDE does not help you at all in identifying which files to change, you just need to know, which I suppose the developers using that framework do.

I am* an Oracle Forms developer. Doing the same thing in forms would take less time than it would take me to describe it here. Write a database upgrade script, add the item to the form and point it to the new column. Done in seconds. Forms takes care of all the database access, locking, transactions, everything without any coding.

New development here is done using java or php, because they’re free. Oracle stuff is not. But even with a fantastic IDE like Eclipse developing database applications seems to be a long winded pain in the arse.
*strictly I should say, was… the economy took that. Doing a different job now.

10

There are some valid and good reasons for using some of these tools. Yet I think, taking the broad view, these tools are designed for management purposes so that they can standardize people’s work and hire less experienced people in the long run. It is part of an overall trend to industrialize the workforce and remove the craft aspect of the profession.

The result is jobs that require less creativity which are easier for management to oversee and switch resources for. Also, it makes work easier to quantify (even if the statistics are inaccurate, at least they have numbers!).

Something like a good debugger is a must. But some of these design tools are BS, even in a large-scale project.

11

As others have said, it’s just creative people trying to make otherwise dull, clerical tasks more interesting. There can also be some job security issues at play, making something complex enough that one person becomes indispensable in maintaining it. One approach is to find interesting work for the people, and telling them they have to complete the mundane work first before they can work on the interesting stuff.

It is possible you are overhiring as well. Many of the applications of the type you are describing require only basic knowledge of the tools, not heavy duty programming skills. Also, this can be more of a problem with younger less experienced programmers who have not yet had to work their way out of the rat’s nest that they’ve created.

12

I’m not sure what you mean by this - the tools we’re talking about here (ORM, etc) typically have a steep learning curve and require higher level people to implement and maintain. Switching resources is often more difficult, as you must train the person in that particular ORM/technology/whatever before you can switch, or set up your workplace so that you only hire people with that particular skill or send all new employees to train on that skill.

If a given technology really did make it easier for lower-level people to pick up and use, I’d be all for it. But I don’t think that’s what we’re talking about here.

13

I don’t know what you are using, but the point of such tools is reduce the complexity of the task. If there’s a steep learning curve and requirement for high level maintenance, you might as well hand code.

14

Oh no kidding. I recently inherited a project that was obviously on the cutting edge of the previous coder’s skill set. Luckily it’s not too big, but boy-oh-boy is it a mess. Used exception handlers for logic, had major state changes happening all over the code (as opposed to being in one nice happy maintainable function), parsing together huge SQL strings with lots of “select *”, I could just go on and on. And I know, if I were to talk to him about it, he’d just say “well, it works!”, which it does. But it’s just such a mess, and so obviously because he’s never worked on anything more than one-or-two person projects.

15

Per the OP, we’re talking (in part) about an ORM that took someone 7 days to set up and still barely works.

In my experience, these kinds of things sometimes have payoffs, but typically that only happens in larger projects with many developers, where once some guys spends a week or two setting something up, it works, and eventually you see savings over 6 to 12 (or more) months (or however long the project takes).

It’s much harder to justify in smaller projects with fewer developers and a shorter timeframe.

16

I agree with this sentiment.

The realistic systems that end up being “good enough” seem to be systems where all SQL ugliness is still hand written, but it is tucked neatly into a separate package or program unit from the web bits and there are clear boundaries between business layer, database layer, and presentation layer.

Anything with less rigor ends up being a scary mess of horse-choking SQL statements embedded in JSP/ASP code—often made fragile because many bits have to be copied/pasted all over the place and a future developer will probably miss one of the duplicate bits when applying a fix.

Anything more, and the deep ORM stuff just seems to be far too much hassle for smaller projects with little return for the vast effort needed to get it going properly.

17

You make a good point, and my generalization was not meant to apply to every possible tool! Similarly, I don’t think the OP was limiting the question to ORM, I think there was a wider question about tools. For example, I worked for a midsize company that was bought by a mega-corp a few years ago, and our work has changed dramatically. We had an experienced group of developers who were intimately familiar with each other’s abilities, and we wrote high quality code and could turn on a dime. Now our work is exceedingly fragmented and subject to various tooling so that everything can be quantified, with the aim of distributing it to other development teams overseas. In the long run it is NOT more efficient but it can be skewed to present the appearance of flexibility, when in fact it is straight-jacketing people into small, atomized units of work.

I’m not saying that tooling always has this effect, just that I have seen instances of this occurring. Simply mandating use of a tool is not a good thing in itself.

18

I don’t know anything about the real best solution for your app, but just wondering, have you ever had to maintain/update an application that somebody else wrote? (Not talking about working on a project with someone, but legacy code that the original writer is long gone for).

In my limited experience, that’s an experience that makes a lot of programmers understand why some ways of doing things are better. Not that the fancy, complicated solution is always best, but clarity and extensibility is worth a lot in the long run.

19

We call that Expection Handling (as in that you are expecting the error).

Employee employees = GetEmployeeArray();
try
{
for (int index = 0;true;index++)
{
Payroll.ProcessCheck(CurrentPayPeriod,employees[index])
}
}
catch (IndexOutOfBoundsException noMoreEmployees)
{
//no more employees
}

This is bad because if ProcessCheck throws an IndexOutOfBoundsException due to an internal problem, you never find out because the code you wrote mistakes that as an indication that you are out of employees.

20

IMHO we now live in a world where the axiom is: “Everything Must Be Automated!”

Why have 2 clerks making $25K do something manually, when you can have 4 techies making $80K create and maintain an automated system to do the same thing? :smiley: