Well I finally got around to taking my car apart on sunday and discovered that there are some dumb people out there. When I took the part off I noticed that the o-ring wasn't quite right... it was way too big. The proper one could fit nicely inside the one that was used. No wonder it leaked. After getting the part cleaned up I noticed that the flange was cracked, so I had to get a new one. I was dreading that but it turned out to only be $15 for the part.
So yesterday after work I put the new part on with the new o-ring and a new o-ring for the sensor that is in the housing. I also picked up some gasket maker that is specifically for thermostat housings because the block is quite pitted under where the o-ring has to go. I'm going to put anti-freeze in it tonight and hopefully all is well.