|
|
|
incompetence in everyone
Friday, February 5 2010
There was one vexing issue still remaining for any database synchronization process that could be expected to operate behind the scenes and not require gross modification to the web application whose data needed to be forked into two different databases: updates. Updates to a row don't result in dramatic detectable changes to a database. Without access to the logs, some such changes can only be discovered by scanning every column of every record of every table and comparing each to a saved copy dating to the last synchronization. Obviously this is impractical. A better solution would be to intercept every database change in the application and either log it or fork the data at that point. This would be easy in an application with a wrapper function around its database accessing code, but the application I was working with had no such wrapper function (and even if it did, I would prefer not to write code in Cold Fusion, the site's server-side language). It turned out that the easiest way to capture such changes was to intercept every submit from all of the site's forms. I could write a single intermediate web page that could catch the form data from every form, analyze it, log it, and then submit it again to its intended destination. In fact, I could write this intermediate page in PHP even though the pages on either side of it in the workflow had been written in Cold Fusion. Then I would modify the "action" of all the site's forms and add hidden tags to pass on the urls of the original form processors. This sounds like a big, complicated job, but it proved relatively easy. Soon enough I was not only logging the data, but I was also successfully determining, based only on the names of the fields being passed, what tables were being modified. (It certainly helped that the site was consistent in keeping the names of form inputs the same as the names of related columns in the database.)
As I was experimenting with this new form interceptor (which I'd written completely generically with a view to use it other applications), I discovered a fundamental problem in a data migration that had been infuriatingly difficult but which I'd thought I'd been able, in the end, to pull off successfully. I speak, of course, of the migration of hundreds of megabytes of data from one Microsoft SQL Server to another, a process I've previously compared to "coping with refugees fleeing a war zone" (in contrast to the many the completely-automated ways to move MySQL databases). Today I found the application was failing when I attempted to insert data in the USERS table because, well, the indentity (autoincrement) trait of its primary key had been lost. It seems that when the SQL Server Import and Export Wizard creates a table in an export, it not only loses the restraints, it also loses the identity attributes. Fine, I thought, I'll just turn the identity attibute back on. In MySQL, after all, this is a very easy thing to do with an existing table. But not so with Microsoft SQL tables. To flip that one little attribute, I would have to use Enterprise Manager (which automates what turns out to be an extremely complicated process), but the crappy shared hosting I was using wouldn't permit the level of access necessary to make this change. (I even called GoDaddy about this, though with matters this esoteric and technical tech support is as likely to misinform as it is to inform.) Believe it or not, I ended up having to blow away the table, rebuild it from scratch, and then attempt the migration (a time-and-bandwidth-consuming process) once again. But even that failed, because I couldn't get the goddamn "IDENTITY INSERT" thing in the wizard to work. For awhile there it seemed like I was stuck with an unfixable web application: I required identity primary keys, but I could find no way to actually migrate data and have them. Then, for some reason, I tried working from the other direction (doing an import instead of an export) and this worked. The whole process was an infuriating disaster and it's my firm opinion that someone at Microsoft should be working at Taco Bell instead.
Not that there isn't a certain amount of incompetence in the lives of everyone. Today, for example, I went into the woods to cut down a dead oak tree but it fell against another tree and got hung up there, dangling dangerously high above the Stick Trail. Hopefully we'll have another day of terrifying winds and the damn thing will fall.
Ray came up from Brooklyn this afternoon with plans to spend the night with us and then drive around to the local restaurants to hand out resumes. He'd been working for a book publisher for the past couple of years but was laid off several weeks ago. He hopes to get a low-stress/low-income restaurant job with a view to eventually moving Upstate. He's actually quite experienced in the restaurant field, having worked as a waiter, a bartender, and also in various kitchen jobs. He's also once worked as a school teacher in a rough neighborhood but had to quit after the stress led to a life-threatening autoimmune condition.
The three of us watched teevee and dined on a vegan pizza (Gretchen having just received a stack of frozen pizzas in the mail).
For linking purposes this article's URL is:
http://asecular.com/blog.php?100205 feedback
previous | next |
