Coding/Programming nightmares. Those forced on you and the ones you left for others

Miscellaneous and Personal Stuff I Must Share
1

If you’ve dealt with code, you’ve almost assuredly run into a ball of code you’d rather not deal with at all, but you have to anyway. If you’ve done it long enough, you’ve probably been the progenitor of the same for someone else to deal with. You might be the kind of person who actually wrote clean, readable code from birth, but I think you’re likely to be the exception, and you’ve still probably had to deal with the efforts of the rest of us. So, I figure we can share our nigthmares here.

I’ll share one I had to deal with and one I created to start us off. It kind of neatly bookends my tenure at one job.

The one I had to deal with:

When I was a fledgling Unix admin, I ran into problems and irregularities (a bunch of fucking bugs) in a Perl script that our CTO had written that in the end managed mail routing and aliases on all of the 4000 or so systems that our company ran/rented using our standard Linux image. I got access to the unencrypted file shortly before he left the company, and endeavored to resolve some of them.

It started with the worst variable names. and never got any better. IIRC sequentially $A-$NN (yes, all scalars), in less than 200 lines of intentionally bizarrely written code. No comments whatsoever. When talking to another employee, I found out that the previous CTO had admitted he himself had no idea how it worked anymore. I slogged through it to where I had about half the code figured out and commented before I found out that we were going to ditch everything about the old system, anyway. I still dealt with other code that fool created, but none was as nasty as that.

The one I created:

The same company had no money and a constant backlog of DNS tickets that never ended. Since I was the guy who handled most of those tickets, I was tasked to write the customer facing DNS control panel they couldn’t afford to buy that would interface with the APIs of their existing CRM system. I had mostly used Perl and/or Bash in most of my work at this job, but the CRM was written in PHP, and its examples for authenticating were written in PHP. Most of my experience with PHP up to this point amounted to looking at the docs and explaining to customers “this is how you’re doing it wrong” and providing enough of a code snippet to demonstrate the correct way to do it. But, if my customers could do it, how hard could it be?

So, I started to actually learn PHP while writing a reasonably secure, functional DNS control panel. IIRC, it was around 4000 lines of code with comments, etc. in the end. When it was mostly functional and around 2000 lines, a co-worker of mine who was actually a pretty good PHP developer took a look at look at it, corrected a bunch of formatting and said “I cleaned it up some, but it looks like it was written by five different crazy people”. However, even he was kind of impressed about how well it worked and what it did. I switched between OOP and procedural styles where it felt easier for me when dealing with that idea. But in the end, I was the idiot savant coder version of James Joyce. Clever, but hard to interpret. Even though it was littered with comments and descriptive variable names, I still had to kind of wrap my head around how it did things when I needed to return to it and work it through new issues and fix things. One coding style, or having the sense to break the silly thing into modules would have gone a long way.

When I left the company, I checked the docs for the next release of their CRM system, noticed the authentication system was going to change and told them “Don’t upgrade your CRM without contacting me first so I can re-write the auth system on the DNS control panel.” Of course, they didn’t remember that, upgraded it, and broke it, and apparently never contacted me. They cycled through a couple of admins that couldn’t fix it, and then happened to hire a friend of mine who had the good sense to contact me, who I helped fix it in about 4-5 messages after he got to looking at the code (it also didn’t hurt that he’s smarter than I am, but he still had to ask to be sure).

So, any steaming piles of code that other Dopers have had to fix or have embarrassingly left around for others to deal with?

2

My only “professional” program…
Mid 80s. I was going to school hoping to be a computer programmer (never did). We had manual forms to fill out. At the end of the month a co-worker would spend hours converting this into files in a computer and running programs (COBAL) for status reports. He got a different job and I inherited this.
I wrote programs where the data could be entered online rather than writing it out on paper and could run the report at the push of a button.
Whoever inherited this probably saw a lot of amateurish coding, but it was a lot better than what I started with.

3

Coder for 30 years here, recently retired.

The worst I inherited:
A 10,000+ line sensor model written by hand in C. One giant header file (just one), and every variable used in the entire model was global (extern). Every function was void and changed some value in this giant header list, nothing was returned. Not a single data structure was used, just a large collection of arrays. And the arrays which described elements in the program were not even co-located in the header file. Lat and Long might be together, but Elevation would be declared 50 lines later. To make it worse, there was no consistency in whether the relevant elements within each array began at zero or one. Therefore, many of the functions were “off by one” as they traversed their data, and the previous programmer had corrected this by (sit down please), adding “[i - 1]” or “[i + 1]” as needed to every single array reference in the function.

I inherited this mess because it went wonky as the number of tracked elements increased. After I finished sobbing in my cube, I gathered some analysis tools and discovered the overruns were uncountable. Once it got past some number of elements, it wandered out of many of these arrays and everything went to hell.

What I was forced to leave:
Just to get it working enough to start fixing, I changed the compiler settings to lower optimization. Then set about adding dead-space variable declarations in between each of the variables in the giant header file. After several tries, I got it to work a little better by spreading everything apart enough that the overruns weren’t stomping on adjacent data. And… was told by my handlers that was good enough. We didn’t have budget to repair it, so check it in and move on.

4

Ooooof, that’s quite a nightmare, pullin. It’s a textbook of what not to do in C.

5

Decades ago, I designed an A/V/Serial controllable switcher for a Mac-based multimedia platform. It allowed 4 serial-controllable devices (Laser disc players, VCR, CD’s, etc.) to be controlled from a Mac SE. Since there were only two serial ports on the SE, I used ADB (Apple Desktop Bus) to control the box. I wrote all the control firmware in 8051 assembly language. Everything seemed to work perfectly. I burned 200 EPROMS and we went into production. Then, I got a report that the box locked up.
Ugh.

The content developer said he had been away from his desk for a long time, and when he got back, the box wouldn’t work. He eventually determined that if he was using his computer, everything worked great, but if he let it sit idle for several hours, the box locked up. I thought that was weird, but set about trying to replicate the problem. I eventually figured out that if I reset the bus enough times, the box would hang. And, I found out that when the bus was idle, Apple would reset it every 30 minutes or so (why???). So, now I knew what triggered the problem - what I had to figure out was why it caused the firmware to crash.

I figured out that what was happening was that the stack was getting blown, and overwriting something important. This is a very, very common problem in higher-level languages, but not that common in assembly, since I wasn’t using stack frames or anything sophisticated - just ACALLs and JMPs, etc. I thought it through, and realized that really, the only way that this could happen was for me to JMP out of a CALLed routine. But, I would never do that, would I?

Turns out - yes, I would!

I had a number of error cases, and they would JMP back to the start of the code. I was (incorrectly) thinking that this would be fine - after all, it’s the start of the code. I had figured that everything would get re-initialized and would be fine. Which is true, as far as my code went. But, the SP was initialized even before my code started, so it would slowly get incremented, until it clobbered something important. It took one instruction to init the SP at the beginning of my code to fix the issue.

Then it was a “simple” matter to open 200 boxes, remove 200 EPROMs, erase them 16 at a time for 20 minutes, reprogram them, and reinstall them in the boxes.

The box worked very well and we made many hundreds of them.

6

I left one when I retired, but it could have been worse.
The program extracted information from tester output logs, which was used to populate a database of the results. I used the keywords from the log as instructions, and used my microprogramming experience to build an instruction decoder which shipped each line or set of lines off to subroutines to be processed.
The problem was that comments were semantically significant, and the people writing the test programs had no standards. The test log for each product would be different in trivial to them but significant to me ways, and there were misspellings, different ways if saying the same thing, etc. I’m good at constructing grammars of languages, but that was impossible here.
So, the code had tons of if trees figuring out which product and test program revision it was seeing, and decoding the line accordingly.
Given the lack of resources and lack of time, it worked pretty well.
Then I retired.
During a get together lunch my old boss asked me if I would come back for a while to look at it. I told him he’d have to pay me. That ended that.
The whole division died anyway, with most people laid off, so the problem is self correcting.

7

I inherited a few and created a few more during my tenure in dev.

This http://thedailywtf.com is a site dedicated to just what the OP asks about.

Mon-Thu they post a code snip with a back story and let the snark fly in the comments. Every Fri they post a collection of screenshots of FU-ed UIs. They don’t post a new entry over the weekends. The site gets little traffic on the weekends. It’s 90+% devs posting from work.

You don’t need to register to post, just choose a handle, although for the first few posts you can expect to be hidden until reviewed.

Some of these entries are mind-bogglingly bad, others are pretty meh.

8

What I inherited

I have spent the past couple of decades developing and maintaining code for an industrial controller that uses its own proprietary operating system. The original controller code was developed on a PDP-11, got ported to a VAX, then eventually ported to be built on a PC. While the PC compiler is fairly nice to work with, the original compiler had a LOT of limitations. One limitation was that it only bothered to check the first 6 characters of variables, and would link up to the first defined variable that it found that matched on those first six characters. In other words, if you had VARIABLE1, VARIABLE2, and VARIABLE3, as far as the compiler was concerned, they all matched the first six letters (VARIAB) and were therefore all the same variable. Also, if you put an apostrophe in a comment, it assumed that this was the start of a text string, and would ignore everything until it found another apostrophe to match it with. Most of the code was written in x86 assembly, with a small amount of C. And, keep in mind that all of this was written before the ANSI C string library became standard.

Needless to say, the short variable name, cryptic compiler weirdness, and other things resulted in some nearly incomprehensible spaghetti assembly code. Much of the C code isn’t much better. Imagine dealing with very complex code with variable names like A, AA, and AAA, and almost no comments at all about what those variables actually are.

To make things even worse, because this was written on an 8086 with limited code space and limited memory overall, the original authors often played games like making the DS register temporarily point to something other than the task data segment just so that they had faster access to certain bits of data (to you non-assembly programmers, a general assumption of most assembly code is that the DS pointer on an x86 processor is actually pointing at the data segment so that you can access data segment variables.

This embedded controller has gone through numerous revisions and upgrades and bears little resemblance to its original self, but a lot of the original code remains. For example, certain data structures are sized in 256 byte chunks because that was the sector size on a PDP-11. Most people in the company I work for are afraid to touch it, and with good reason.

I think there’s something a bit wrong with me, because I actually enjoy working on it. But it is probably some of the most incomprehensible and complicated code in use today.

My Contribution

The controller has evolved by leaps and bounds, but since it is an embedded controller even the most modern version has some significant memory limitations. I have spent a lot of time going through the code and turning it into horribly unreadable spaghetti code just to save a few bytes here and there.

9

If you’ve ever heard of Forth you can imagine the nightmare I ran into. When the codebase was started there was a lot of design documentation and the code was heavily commented. But it was easy to see the point where it went out of control. The first step in fixing or changing anything was a real serious evaluation of how long it would take to redo it with some planning aforethought. Unfortunately too many intertwined areas of code could not be redone. The solutions kept making the problems worse, functionality became dependent on identifying the calling code off the stack at the time of execution.because that calling code was too complex to change or rewrite.

10

One of the biggest projects I’ve ever done was in Forth, in the mid '80s. I loved it. But the temptation to misuse it is just awful. In Forth there are almost no restrictions in the names you give variables or subroutines. For example you can program 2 to return 3, or you can use names like l|l||l|ll|II|Il (that’s a bunch of lowercase “L”, uppercase “i”, and pipe characters). There is also a stack for numbers (remember RPN?) and a stack for return addresses, and you can move things back and forth between these however you wish. I kept toying with how to write the most impossible to understand version of “Hello, World” in it.

11

Aww, I wanted more detail.

Yeah, I’m aware of the site, and I’m a bit of a fan. I’d have to be a robot to have been a Unix admin for more than a decade and not be. :clown_face: But that place kind of lacks the “We have met the enemy and he is us” feeling that my experience gives me. I saw quite a bit that made my mind boggle as to how it got that way. As I went along and created things of my own, I inevitably left a few creations that bewildered my successors. I might be able to explain it with a lack of time/resources to have done it better, but just about everyone else could use the same excuse. Plus what I created is still probably more difficult to deal with than it should be, and I knew better.

Heheeheh, I’m not sure something is wrong with you, I’m pretty sure it’s actually a valuable skill. But you certainly don’t shy away from difficult problems.

There was at least one part of that image that I just stared at in horror when I saw it. It monitored a directory on the server (writeable by the user running the web server), when it saw a new file in that directory, it read its contents in as shell commands to be run, AND THEN RAN THEM AS ROOT WITHOUT ANY SANITY CHECKS. Upgrades to the systems were piecemeal, so there were many different generations of this code running around that I had to decrypt and examine for different reasons. None of them had any attempts at fixes, but there was one version that had been audited and commented by someone else before I embarked on my new errand that said “This is beyond horrible. $NameOfControlPanel needs to be re-written to not expect this to work. Right now, I don’t have time to do anything about it.” I wrote a sub-routine that at least checked a list of blacklisted commands that the control panel couldn’t be expected to actually need, but that was paltry defense against the awful hole it was. Similarly to the previous example, it was part of the image that was ditched shortly after I got access to it and started talking to anyone would listen to me about the nightmare it was.

Something that got ditched from that image quite some time before I got access to the source that relates to my shame:

The image managed DNS changes in a sort of similar manner. The web server or CLI client would write their DNS changes out to a file, and the DNS servers would log in periodically to retrieve the file and make the changes contained within through a set of scripts. It was a pretty stupid system which similarly to the above system would blindly run any DNS changes that were retrieved. This would create problems when customers would do totally normal things like create a test domain by the same name as the active domain on a separate server. This system would pick it up and happily send their traffic to the test server.

That’s bad enough, but these weren’t just the authoritative DNS servers for our customer’s domains. They were also the caching DNS servers for all of the machines running that image. So when customers did things like create an entry for domains like yahoo.com or gmail.com, loony issues cropped up.

I was able to kill off that system about a month after I was able to figure out what it was doing, and then had the time to stop tracking down stupid DNS issues so we could roll out caching DNS servers and switch all the systems over. However, that meant I was on the hook to write something better. So, I had to write the DNS control panel, which ended up with all the stupid different programming styles. That wasn’t forced on me, and I really intended to at least unify its style and structure it better before I left the company. But their horrible management style finally left me as the only Unix admin again, and I finally bailed on them.

12

I didn’t technically inherit it as I was hired to translate it off of an archaic Xenix network architecture to the more modern and robust MSDOS and Novell 3.12.

This was in the early days of Windows 95 if that wasn’t enough info to narrow it down. GET OFF MY LAN YOU YOUNG HOODLUMS!

The existing code base was Xenix BASIC. For those not familiar, most variants of BASIC will let you concatenate multiple commands on a line separated by colons. This is mostly to allow short code blocks to occupy one line when it’s obvious what the block should do. Or to allow adding code without having to renumber the entire program. Oh? Didn’t I mention that this was before named code blocks and every line of code had it’s own line number? GOTO 1570. GOSUB 3000.

Not saying the original coder wasn’t smart, he taught himself to code after all. But every line was near the 255 character maximum length and the line numbers were usually +1 sequential. I was never so grateful at not having to actually maintain that monstrosity. Analyzing it to figure out what it was supposed to be doing was migraine inducing enough.

Fortunately I can say I haven’t left any major dumpster fires behind me other than some poorly commented routines that I was going to get back to.

13

The company I worked for at the time had developed a machine that calibrated one of our products. Basically, you placed the products on the machine, hooked them up and ran the calibration program. The machine used a laser, a stepper motor, and talked to the products through our proprietary serial protocol over RS-485.

The hardware was excellent. They gave the task of creating the calibration application (in VB6 - DON’T JUDGE!) to one of our programmers overseas who 1) totally didn’t understand how the products and the serial protocol and RS-485 worked and 2) had no idea how VB6 worked (not really his fault, he never worked with it before).

Time to start rolling out the product and it the calibration machine would only work sporadically. I was tasked with “make it work! hurry!” I did manage to make it work by putting in a bunch of delays in the code and other horrible bodges. There was no way to make it correct without re-writing the whole thing. Then they wanted more changes, fixes, and added features, etc.

I was downsized from there in 2009, and before I left I apologized to my fellow programmers there for leaving them the hot mess. At least it was less of a hot mess that it was before.

14

I can honestly say that I’m not ashamed of any code I wrote as a professional programmer. Just ignore the horrible spaghetti code I wrote as a student, but since those card decks have long since been recycled, that won’t be hard.

Anyway, my first job as a programmer was at a very small operation. The guy running it also designed the system (it was a new system) and wrote the specs. There were two other programmers (I think they only worked part time) but after not too long, they were let go. I never asked, but thinking back on it, the boss probably realised I could write all the programs he needed.

There weren’t very many bugs in the system, but one cropped up in one of the few programs one of those two other programmers wrote. Since I was the only programmer, of course I had to fix it. So I sat down to figure out how that program worked. Now this should not have been a complicated program; it just generated a report from a database. But i studied that code and studied it and finally thought I figured out how it worked. And at that very point, someone walked in the door, said something to me and I lost it.

At that point I realized it would be faster for me to just rewrite the whole program, which I did. Took me maybe half an hour. Didn’t even tell the boss until some time later.

15

Forth was not only low level, it was self compiling, that took it far past redefining the value of 2 as 3. That’s just a dumb Fortran type of mistake, you could change the entire language by modifying the lowest level code definitions. But in this case you can definitely blame the player and not the game. I was about to say it takes discipline to work in that environment but that misses the mark, what it takes is the sense to avoid going down the road with the big huge sign in bright neon letters that says “Bridge Out - Dead End Ahead - Last Chance to Turn Around”. Pretty much the same as any other development platform except Forth is right at the edge of the cliff most of the time.

16

One of my first jobs was a bit like @Projammer’s story but ~20 years earlier.

A wholesale company had had a generic accounting / inventory package written in BASIC for the Basic Four series of minicomputers. Which had become obsolete and too small for their growing biz. So it was time to port it all to a new host, the HP 3000, and a new language, HP-BASIC. And best of all, from a quasi-ISAM file system to HP’s latest whiz-bang non-SQL database, IMAGE.

A thousand+ pages of uncommented massively patched BASIC spaghetti? Check. All variable names limited to A1, J6, etc.? Check. All program module = file names limited to AA, BY, G7? Check. A radically different OS, command line, and file system? Check. A new and poorly understood DBMS? Check. Significant enhancements invented on the fly as the conversion effort was on-going? Check.

Immediately upon “finishing” the project they decided to market it to the rest of the burgeoning HP user community. Which meant a lot of “This is coded for how we run our business.” suddenly became an obstacle as we had to try to retrofit more flexible, configurable approaches (or at least not insane, organically grown, shall we say “idiosyncratic” approaches) to how other customers ran their businesses.

As Patton once said of El Alamien: “It was a shambles.”

17

In the 1980s, I worked for a very small engineering firm that used BASIC for their programs. The boss/owner wrote the original programs, and his coding sucked. (Note: I cannot claim to be a great programmer.) One day, I was following code and saw a GOTO statement that went into the middle of a For…Next loop. I let out an audible scream. I think the code may have worked, but that was purely accidental.

18

Ha. It would be cool if he’d managed to implement a variation on Duff’s Device but somehow I doubt it.

My job has involved working with a multi-million-line C codebase which has been growing organically over the last 20+ years. Despite the company doing its best to enforce coding standards, there are inevitably a few things in there that are best not disturbed. The worst I ever ran into (and admittedly this was in internal test function not unleashed on unsuspecting customers) was a function that depended on trailing whitespace. It will probably come as no surprise to learn that I stumbled over it by editing an innocent-looking header file and then wondering why a completely unconnected test was suddenly failing.

But the biggest WTF was when I was very new and inexperienced and found myself being asked to fix up some failing UTs, ostensibly as an introduction to a particularly gnarly function.
Opening up the source file of the offending test, I was confronted by:

test_c7

followed by two empty comment boxes where the function description and arguments list should have been, followed by the two simple and enigmatic lines:

run test_c5
MAYBE run test_c6

Feeling slightly desperate, I searched for test_c5 and was duly confronted by:

run test_c3
MAYBE run test_c4

And so on. There wasn’t a comment in the entire file, and every test did nothing except call (or MAYBE call) tests lower in the list. At this point I ran up the white flag. As far as I know no-one ever figured it out and the code went on doing … whatever it did until that whole component was torn out and re-written years later.

19

It gives me some comfort to hear that others have had to deal with the same types of nightmares I’ve had in my experience (almost 40 years now). It is also interesting to read these stories, and “date” them based on the languages/systems used. And to see how the industry has “grown” or morphed to (try to) prevent programmers from doing the crazy sh*t that got us all into trouble in the first place.

My inherited example is similar to pullin’s: the early days of C. I joined a company and worked on a project that was a spin-off of another project. Part of the problem was that this “base” project wasn’t quite complete, and I found out why - strange, bizarre bugs from seemingly out of the blue (unrelated activities affecting other stuff). The group that wrote the base project took great pride in having “no globals” as the use of them had gotten an earlier project into lots of problems (and rightly so). They did have structures, but that is where the problem was. They had this big structure (made up of arrays of smaller structures, etc.) that…they…passed around…to practically every function. In other words - one big structure-global. And as with pullin’s experience, rather than handle the arrays correctly (with defines and such), there was a lot of “i+1” and “i+2” types of referencing, and so it led to corrupting of parts of this huge structure.
I pushed for breaking up The structure into structures that were more “constrained” in their use, but the use of The structure had gotten so convoluted, it would have required too much change. So we ended up just fixing the bugs as best we could.

As to what I left for someone else, I worked with an early DSP processor for a couple projects. It came with a “C” compiler so I thought “piece of cake”. But the compiler had some major flaws to it. Part of it had to do with this processor having an exposed pipeline, and the compiler didn’t always account for the delays in reading/writing to RAM correctly. It also “padded out” the delays by merely inserting NOPs, so your code wasn’t always the most efficient (yet we were trying to get the most out of these processors). So the process that I used was to 1) write a function in C, 2) compile it, then 3) take the generated assembly and modify to fix the compiler errors, and kind of “parallel-ize” the processing by replacing the NOPs with operations I would need further down. So the result was this assembly code that was marginally comprehensible. I tried to get creative with indentation to separate the “paths”, and did my best to comment what was going on. But I know that even if I picked up that code, it would take a while to understand just what was going on.

20

Hehehe, yes. Both the heroic “I can fix this” and the practical “I know the input and the output. - screw it, I’m going to re-write it.” stories are comforting. And yes, being able to date some of the activities by the language/systems used is fun. I never used punch cards, but my dad did. I learned some BASIC from one of his college textbooks, in fact.

But, now that you mention it, I feel like a definite young’un. Nobody’s mentioned their Ruby or Java nightmares yet. Other than the folks who mention BASIC, I think I’m the only one dwelling primarily in interpreted languages. I know enough compiled languages and x86 assembly from school and/or just fooling around to keep up with most of the discussion so far, but the chances I’ve had to use them at work have been rare.