About the downtime Wednesday night

[Elad]

DU went down around 10pm eastern time tonight, and we were unable to get back online until approximately 1:30am. Our main database server crashed, and it took some time to get it rebooted.

Over the past two weeks or so, as you may have noticed, we've been experiencing some issues with the site, where you may be unable to connect, or get many bugs when posting or browsing. We've been doing our best to deal with these issues as they come up and working to figure out the underlying cause. Without getting into too much technical detail, the issues are all database related, and we believe the problem lies in the server itself (potentially a piece of hardware going bad). We will be moving as fast as we can in cooperation with our web hosting company to isolate the source of the problem and come up with a solution.

In the meantime, we ask that you bear with us while we work these problems out. We are just as frustrated with them as you are. You can help by always clicking the "report" link if you get a bug report, thus making us aware of particular problems quicker so we can fix them.

Thanks for your patience. We'll post updates as we have them.

Elad
DU Admin

[Patsy Stone]

1. Thanks for the info

and for just being you. :thumbsup:

[Cha]

72. Exactly!

:loveya:

[NMDemDist2]

2. thanks for your hard work Elad, now go have a beer and go to bed!

:hi:

[Hekate]

8. Yes, do that!

And thanks.

Hekate

[Gold Metal Flake]

3. That's odd...

Because I caught this guy in the server room.

[UrbScotty]

17. Looks like Dick DeVos (nt)

[ChoralScholar]

33. HA! "Oh, it's a big pretty white plane with red stripes,

curtains in the windows and wheels and it just looks like a big Tylenol!"

[aquart]

4. Good luck!

We don't realize how much we depend on DU till we don't have it.

[Ron Green]

5. Nevertheless, thanks for running the best site on the 'net.

:yourock:

[originalpckelly]

55. I know, DU is the best discussion software/site I have ever used...

Pretty amazing when you compare it to the shit Rim Job sucks his folks into using.

[blm]

58. Agree - maybe an early fund drive can help?

.

[Cameron27]

6. Thanks

:thumbsup:

[lildreamer316]

7. Thanks again for all your hard work.

It is much appreciated by this homebound housewife.

[mohinoaklawnillinois]

9. Thanks for the update Elad.

I was having visions of an eville hacker bringing down my beloved DU...

[orleans]

20. so was i. maybe it's something in the water in illinois. (no--never mind

i drink bottled water)

???

[la la]

41. me too--

I've been trying to get on all night.

Thanks to you all for the great work that you do. The internetS tubes is a hellish thing at times!

:toast:

[ancient_nomad]

10. Thanks Elad!

Sometimes one doesn't appreciate things until they're gone. ;-)

[TahitiNut]

11. Hang in there, Elad. You do good. Thanks for the info.

It's as I guessed. (Maybe I haven't lost all my marbles in the eight years since I was active.)

[Eurobabe]

12. You guys rock! n/t

:toast: Hang in there!!

[LSK]

13. I wish you the best of luck on finding the problem

I work in IT so I know what a pain in the ass those "unknown" problems can be.

[The Doctor.]

14. Over three hours without DU!

Imagine the suffering! :cry:

Thanks for being all over it. You guys rock!

:yourock:

[yewberry]

15. Thanks Elad. Understand, however,

that vegan programmer chow can be withheld.
:silly:

(Thanks for the heads-up!)

[TahitiNut]

27. "vegan programmer chow"???

Times sure must have changed. We survived on coffee, cola and Hostess cupcakes.
(God invented vending machines for programmers.)

[yewberry]

78. This way is less work for Skinner.

He can just toss a bowl of VPC down in the cellar and not even worry about unchaining Elad.

Much more efficient.

[eleny]

16. thanks for working so hard for the site!

[emlev]

21. I second that!

[emilyg]

37. Third Here

good to be back.

[Trajan]

18. Thanks Brian ....

You know : Spring is just about to burst forth in the NW ....

Sounds like high time to meetup with your DU friends in Stumptown .....

Again .. thanks for your hard work ....

[Oeditpus Rex]

19. It's bound to happen sometimes

We appreciate all your efforts, guys. :toast:

[SeattleGirl]

22. Thanks, Elad.

Both for the info and working so hard to fix the problem. :thumbsup:

[JohnnyLib2]

23. Thanks for everything you do.

===withdrawal symptoms were serious :hi:

[badgerpup]

71. Thanks, Elad!

For all you and the programming team do...
and for 'splaining what went wrong last night. I was almost thinkin' it was Big Yellow Taxi time...:scared:

Yeah, so I'm paranoid. :hide::yoiks:
Just because yer paranoid doesn't mean they're not out to get ya...
and if they are out to get ya, then being paranoid is just good thinking.
(second half attributed to WKRP's Dr. Johnny Fever)

[Divine Discontent]

24. thank you

while I have a chance to say some constructive feedback. I think the homepage is really cluttered. Just my opinion, if most other love it, never mind. :) I like the 'news site' feel to it, but it's just so 'full'.

thanks for your hard work in getting your site back up for us!

[nicknameless]

25. Thanks for all your hard work, Elad

It's very much appreciated!
:toast:

[CaliforniaPeggy]

26. My dear Elad!

I truly do appreciate all your hard work...

Thank you for this post, explaining the situation...

DU is the best site ever!

:applause: :applause: :applause:

[WiseButAngrySara]

28. Thank you! and Bill Moyers! ....n/t

[nam78_two]

29. Dammit

And here I was hoping it was because of super high traffic :D. I have noticed a spike in new members (I think)....

[Lugnut]

30. Thanks Elad!

Much appreciation for all you do.

[FloridaJudy]

31. Whew!

I was afraid Homeland Security had come after us, or something equally appalling.
Thanks for managing the best site on the web.


Now give yourself a pat on the back, and go do something relaxing.

[mwdem]

32. Thank you so much!

What a panic to not be able to have the site come up when I tried to log on. :scared:

[CC]

34. And here I thought it was a combination of

KO's special comment and Moyers that brought DU down tonight.:evilgrin:

[mnhtnbb]

35. Good to hear it wasn't Agent Mike who did it!

[illinoisprogressive]

36. I was thinking of sending a donation to you in case you need to replace anything

[Alexander]

38. Thanks Elad.

I know you guys are doing your best. :-)

[MissMarple]

39. OK, I will be more cognizant of possible, future potential errors.

:)

[demobabe]

40. Poor Elad... hardware problems suck

You've got a cluster of servers too, don't you? Three of em? Fun fun.

Will make sure to submit those error reports - I was ignoring them up til now.

Anything else we can do to help?

[Nicole]

42. Thanks for the update.

I've been getting the bug reports pretty often. I figured you were probably being buried in those things so I haven't been clicking on the report link. I will click on them in the future.

FYI Yesterday I got about a dozen of them while attempting to do a simple search.

[rumpel]

43. long live DU

how frustrating it must be...after all DU has grown so much - thanks for the great work.

I vow to report the problems as well - I usually think it's my computer or browser or setting...

[NanceGreggs]

44. Don't it always seem to go ...

... that you don't know what you've got 'til it's gone ...

[DKRC]

45. Glad to know it wasn't just me. Spent the time watching the Moyers report on PBS. nt

[tavalon]

46. No problem

I just went somewhere else and did something else. I did wonder if DU had gotten heavier than usual traffic to cause it to go down.

[byronius]

47. Upon your capable shoulders rests this beautiful challenge.

I mean, talk about your ethically-viable work. You're like, the guy fixing the machinery that Saves The World.

Seriously.

[pnorman]

48. Brother Elad, I just recently posted the below on the DU Lounge:

----------------------------------------------------------------------------------------------
Lengthy DU lengthy shut-down yesterday

I'd been following the HUGE traffic on the DU Panic Room, during the recent outage {What else was there to watch?). One posting there suggested that someone from the DU team, be tasked to post official announcements there about problems and updates. Seemed like a reasonable suggestion.
------------------------------------------------------------------------------------------------

Perhaps I should have posted it elsewhere, but now that I've gotten your attention, please give it some thought.

pnorman
On edit: In connection with something else, I had just emailed to a friend: "Overall, I don't regret being TV-free for all these years. But I wouldn't last more than six hours without the internet!"

Being TV-free was NO exaggeration, and it's been that way for at least 8 years. And DU has been a MAJOR part of my internet activities. I check it COMPULSIVELY each day --- sometimes a couple of dozen times a day. I well remember when DU was hacked by Freepers on that 2004 election day ("All Your Bases Are Belong To Us!"). I'm over the anger, but I can still recall the feeling of helpless panic!

[Cessna Invesco Palin]

49. Maybe you should switch to a new ISP.

I hear Smarttech has some rack space available. :evilgrin:

[rasputin1952]

50. Thanks for the Heads-Up...I thought you might have decided

I was a troll...:D

Good luck in tracking down the gremlins...:hi:

[wildhorses]

51. i bet you need to get bigger tubes --

;)

thanks for all you do

as for patience, you got it:patriot:

[HamdenRice]

52. Is there a DU "WTF is going on w/ DU" site?

I remember once when DU went down and some DUers said that they went to a site (independent of DU) where people go when DU is down to find out what is going on -- or at least to guess among themselves what is going on.

Is there a DU refugee center like that somewhere?

Thanks in advance.

[Cobalt Violet]

57. The DU Panic Room .

It's a yahoo group. It just helps to know that I'm not the only one when these things happen.

http://groups.yahoo.com/group/DUPanicRoom/?yguid=134845070

[HamdenRice]

60. Thanks! That's what I was thinking of nt

[tandot]

62. I had about 100 WTF emails from DUers in my yahoo email account as I logged

on this morning. So, be prepared :evilgrin:

[Rude Horner]

53. Thank you, Elad. We understand

We know how much work you guys put into the site. Thanks so much.

[annabanana]

54. As we dealt with the "DU DT's" in the Panic Room

We agreed that you guys are the best and that DU has become damn near indispensable in making "sense of it all".

[Cobalt Violet]

56. I woke up with 168 emails in my inbox from the DU Panic Room.

So I figured it was down for a while.

[OmmmSweetOmmm]

59. Thanks so much Elad. Thanks too for the DU Panic Room, a comforting place to know

that I was not alone!

[mod mom]

61. Thanks for your efforts!

I went to bed early last night. ;)

[Lone_Star_Dem]

63. I feel you've been keeping us well informed on the issues

I've seen postings by Skinner and you both since the problems began. I thank you all for all the hard work you put into this site and the way you feel a need keep us informed on what's going on.

Glad to know that you would rather we report on any bugs we encounter. I admit that one day I went a tad crazy sending error reports and felt a bit guilty after the fact. Now I don't feel as much like I was abusing the 'report this bug' feature due to the fact that I was having a crabby day. :blush:

I hope you're able to resolve the issues with a minimum of added effort for everyones sake.

Thank you again for providing us with this site, and for all the hard work that it takes to keep it up and running.

[bvar22]

64. Thanks for the info.

While you're cleaning out the tubes, keep your eyes open for Rove's E-Mails and the Ohio election returns from 2004.

[yurbud]

65. Thanks for DU--it's been a sanity saver

[lonestarnot]

66. We know you be takin' care of bidness.

:toast:

[proud patriot]

67. Thanks for the update Elad

and thanks for your hard work , I sure do appreciate it :hi:

[northofdenali]

68. Just as long as you don't disappear altogether -

us longtimers will always hang in. :hug:

[IsItJustMe]

69. Hell I thought chimp had the NSA shut it donw. Wouldn't put it past him. LOL

[ProudDad]

70. Are they your servers or the Host's?

Could you use an emergency donation from us all to help with any upgrades?

[AikidoSoul]

73. Once DU shut down there was nothing worth staying awake for

We've noticed computer related problems on several computers that occurred after the last few Windows critical updates. Microsoft screwed something up, and then attempted to patch some of it with the next update download -- but there are MS induced issues that need more direct, manual interventions.

Nice guy Mr. Gates...I'll bet this cost the business community millions.

[Madspirit]

74. Thanks for the Info!!...n/t

[LuckyLib]

75. I was watching Bill Moyers while frantically trying to log on DU,

and I'll tell you, I had my tin-foil hat front and center!:tinfoilhat:

[Dr Fate]

76. No worries- thanks for all you do. n/t

n/t

[Forkboy]

77. Yeah right...you guys are really in Barbados spending the donation money.

I just hope while you guys are yukking it up down there drinking drinks with little tiny umbrellas in them that you pause to think of the suffering of those of us left here.



I pity you...

:evilgrin: