Well that was a little drama

Thought I had lost the databases for not only this, but also the subter.com. Both sites were throwing a “error connecting to database” message. My first thought was to check the status message displayed on my host’s webpage and I noticed that two different file server clusters were having “issues” recently and figured that my problems were probably related to that.

They weren’t.

I waited a bit, hoping the problem would resolve itself. It didn’t. I sent in a support ticket, which is usually enough to get any issue resolved with my hosts. I received no response. I came into work this morning and saw that both this site and subter were both still down and I had yet to hear back from my hosts. So, I figured, I worked tech support at a web hosting company for several months, I’m a fairly competent guy, I’ll do it myself.

The first thing you learn when working tech support is that the vast majority of mistakes are user error. The acronym frequently thrown around is PEBKAC (Problem Exists Between Keyboard and Chair), and you can usually diagnose the PEBKAC calls within the first 15 seconds of the conversation.

If you know, you know.

I sat down and started thinking about everything that I’d done server-side recently, to try and determine what I possibly could have screwed up. The only thing that came to mind was that I had recently removed a website that I’d been hosting for some other people. Turns out, the database server was lumped under that name.

My first fear was that in using the auto removal tool for that website, I had taken out not only the database for this site (no big loss, not much here) but also the database for subter (which has quite a bit of quality writing on it). I jumped into the database server and lo, the databases were still there. It wasn’t my fault, or so it seemed.

I started banging out a rather angry email to my hosts about how a pair of my websites had been offline for close to 60 hours now when I started thinking about DNS. You see, I started this saying at the hosting company I worked for, “DNS is magic.” The saying stuck because, for all intents and purposes, it is. The process is completely transparent to not only the end user, but frequently the users in the middle. I realized that the hostname for my sites’ respective databases included the domain name that I had recently stopped hosting. Common sense snuck in – if I were a host, and someone stopped hosting a domain with me, I’d strip that domain name from the local DNS system instantly.

I found a second database hostname for my websites, chanced them out in the CMS’s configuration file and a chorus of angels opened up above.

The sites were saved…and then the squirrels attacked.

Social