My Comedy of Errors

Written on October 19, 2008  |  by RP Mickler  |  Leave a Comment  | 

It’s a horrible tale. A tale of frustration, anguish, and very little sleep.

Our story begins a couple of months ago really. One of my clients gives me a call. They’d like email, calendaring, appointments, and I said I could so help them with that. So, I went online and downloaded the trial of Exchange Server 2007 for their Windows 2003 Server. I installed the Exchange environment and switched mail over; all was swell. Well, time had come to get the Exchange installation licensed this week so I went to get the volume licensing squared away only to find that I couldn’t get Exchange 2007 in a 32bit distribution, so I called up Microsoft’s open licensing division. After two hours on the phone and, eventually, my own looking into the subject, I found that I made a horrible assumption. That Microsoft would be distributing licensed copies of Exchange 2007 in 32bit. Totally wrong – they’re only selling it in 64bit, and the 32bit versions only exist for, well, trial and evaluation purposes. Yikes! Well, the licensing was just the same anyway, but I had to pass along the bad news to my users and tell them I needed to strip out Exchange 2007 for the previous version, Exchange 2003, and it shouldn’t take me all that much time. That, right there, was the kiss of death.

So, this weekend, I go to run a simple utility called Exmerge which would allow me to extract the database of everybody’s email to individual containers that I can use in Microsoft Outlook called *.pst files. It’s a common utility – one that’s been around with Exchange forever. Well, I found out, Microsoft discontinued support for Exmerge in 2007. After 2 hours of additional research, I found the commands I needed for Exchange’s new management shell to accomplish this.

Well, except after another hour of fiddling with it, I found out that the command that I wanted really wasn’t available until Exchange 2007 SP1. You see, Microsoft apparently thought that this whole idea of exporting data to files was crazy-talk until the universe screamed and they included that functionality in the first service pack. Well, okay, so I download the service pack and run through installation, and I had to spend another 2 hours troubleshooting that.

Okay, so, I now have Exchange 2007 SP1 on the server and the tools I need are installed. Great! So I go to run my command, but I still can’t export. Why? After another hour of research, I find out that this will only work with Outlook 2003 SP2 installed. Well, that’s a client application and I don’t have that on my server, so I have to install it over a remote connection which takes upwards of 2 hours. Finally, after something like 7-9 hours, and it’s 3 am in the morning, I finally get the database extracted to individual PST files.

Whew! Piece of cake! Woops. That killed me.

For you see, now I needed to uninstall Exchange 2007. That didn’t go very well; the process bombed on me multiple time. Four hours later – I kid you not – I still can’t get Exchange 2007 to cleanly uninstall, and its Active Directory updates are confusing the ForestPrep portion of Exchange 2003, so essentially what I have here is a trashed Active Directory database.  Loathing what I would have to do next, I realized the bitter truth: I would need to DCPROMO the server to make it a standard box, then, DCPROMO again to re-create the domain and a brand new Active Directory schema.

Holy crap, Batman, loads of work. So it’s seven AM. I get started. I start backing up files and my user’s profiles; I start checking services; auditing my permissions; getting ready for basically nuking 30+ users, their passwords, and all of their data. So, I do this, downgrade the box, then DCPROMO it back to a Domain Controller for a new domain. It worked! I was able to install Exchange Server 2003 cleanly, get IIS to map correctly for the remote services, and installed all SMTP routing for email. I re-created my user accounts and my mailboxes. Now I was ready for Exmerge! It’s noon, on Sunday, I’m now 36+ hours into no sleep land and some 17 hours into this project.

Well, Exmerge fails. I spend another two hours verifying permissions, and then I learn the next horrible truth: I used Outlook 2003 SP2 to create the *.pst files with all of my user’s email. Well, Exmerge only understands Outlook version 2002 or _less_ and can’t read the 2003 file. Microsoft’s brilliant remedy on their technical articles: open Outlook, create a new *.pst in the older version, import the stuff from the subject *.pst, and save all of it to the older version. So I manually DO this for 30+ users.

Now… NOW… I’m ready to roll! I kick up Exmerge, get it to import, all of my user’s containers get their email back. I ran the file restore and brought their files back to their profiles. Yes, it’ll be a headache as users need to re-log in tomorrow (Monday morning) using a new credential and a new password, but hey, 20+ hours later over the weekend, they have a working server.  The server works; the new Active Directory is sound; mail is routing; my client is a reasonably happy camper.

Me? I’m looking at ~44 to 48 hours of no sleep. While I was doing all of this, I was trying to grade homework assignments across four classes to kill a little time. Wow. What a weekend. I need a nap.

R