A discussion about the major site problems yesterday/today

Well, it is worth discussing. This was the most major problem we've had on DG since we began the site in the summer of 2000. The problem was so severe and so long lasting and so visible that it was just a disaster, all around.

The good news is that everything is in fairly good shape right now and, as far as I know, very few people are having problems with the site. Things should stay that way.

So, what happened? Here's the (lengthy) answer:

I am always looking for ways to optimize DG and make it run faster. I think a fast experience is very important. So, when I launced the new search engine yesterday I found the server was actually running quite slow, so I started looking for areas of optimization.

Among other things, one change that I made was a programming change (warning, technical discussion follows):

PHP has a function called ereg() which lets you search a string for a value. It works like this:

ereg(needle, haystack)

But ereg is slower than a different function, strpos(), which does the same thing.

So, I changed all the functions from ereg() to strpos() and actually saved a millisecond of loading time in the process. The problem is that strpos() takes arguments in reverse, like strpos(haystack, needle) and therefore every value on the site was reversed.

So at this point you can see where the problems began. The biggest manifestation of it was the "Permission Denied", because I have the system programmed to refuse service to certain computers. Well, when it was checking to see who you were, the system was seeing the exact OPPOSITE of reality and therefore disallowed almost everybody.

Well, I didn't notice this when I uploaded the change (for a variety of reasons) and for over 3 hours the site was like that.

When I got in the office at around 4:30pm yesterday, I found and quickly fixed the problem.

After that, however, there were still issues of caching. People's browsers were still caching the pages and therefore they were still seeing the error.

At this point, I think almost everybody is caught up and able to use the site to its fulness. There really shouldn't be any more open issues, with very few exceptions and those will work themselves out over the day.

So, there ya have it. I hope the next disaster like this won't occur for at least another 5 years. :)

As a side note, I want to thank everybody who gave us reports, and particularly those (almost everybody) who was kind and gentle with us. When you have such a disaster, rude comments are not at all helpful and I am thrilled to say that the membership was extremely gentle and kind with us during the experience! Bravo!!

Dave

Still can't get in using my usual browser inspite of having cleared the cache several times. Fortunately, I have two other browsers on my machine and they both work.

Ann

I'm still seeing something weird. If I go from thread to thread within a forum it is fast, but if I jump to a different forum there is a considerable lag.

And Dave, what's with the counter at the bottom of the pages? What's that for?

Thank you for the explanation of what happened. I didn't understand a bit of it, but thanks for trying.

mg

From "Home page" to "Hosta" Forum: Render time: 5770ms

From "Hosta Forum Home page" to thread within Hosta Forum: Render time: 1270ms

Does this mean anything? Other than I am impatient, lol?

mg

Dave,

I consider myself fairly computer literate, and that goobly gook you posted made me appreciate this site all the more.

It is easy to take what we have for granted. And therefore it is easy to be annoyed when we experience anything less than the stellar workings we are used to here. Personally, i am amazed at the speed and fluidity of this site. On the rare occasion that we experience problems here, maybe it is a sign that the universe is telling us to get off our bottoms, quit staring at the computer, and get outside and weed. :)

Melissa

I just say thanks for doing all you do to keep and maintain this site with the care that you do.

Hands on and caring all the way.. Koudos to you.

The troubles we had I knew would be short lived and corrected just reported them for information your way if needed.

Thanks again wonderful job as always,

Anne

Thanks Dave, you're the best. You've got us all spoiled for any other place.

Blooms you got that right!!!!

I am VERY impressed that Dave not only dealt with all the problems and all of us having major withdrawal symptoms, but he also got the problems fixed and managed to respond personally to my e-mails in regard to the problem.

Thanks. this is a great site.

Ann

Im happy tha I wasnt booted and my crying whiny email got answered so fast and got back in here. Whew, I was freakin out big time trying to figure out what I did, LOL.

Thanks Dave for all the help and believe it or not I could actualy comprehend what you were saying, now ask me how to do what you did and/or how to fix something like that and youll get the deadeyed look of a computer illiterate blonde who only had one cuppa coffee.

I am not into programming and such stuff, but I do know that with the major changes you have done to make this site what it is, that we HAVE to expect some bugs to surface and have the patience with it. My reports of what I have encounter is just to make it easier for you to fast figure out what is wrong as you always do.
The only time I freak out is when the chat is not working :0)) but MG and I just start the chatannex then, rotflol hope YOU don't mind us doing it.
I don't expect you to be online 24-7 and I know that you don't have four arms and 2 pair of yeys even if we sometimes wish you had. So we can wait! as the saying goes
If you are waiting for something good, you can never wait too long.
Janett

No rude comments here - just a note of appreciation for taking care of the problems so quickly! I know myself, no freaking out - just going into withdrawal :)

My only complaint, and not really a complaint, just frustration - still no photos posted with the Newsletter? I look forward to the comments and photos every morning - but I'm a patient person :)

The lack of photos is my fault. I thought I saw a batch go through successfully yesterday morning after Dave had fixed the problems. So as I usually do, I sat down about 10:00 last night to release the photos for the day - and encountered the SQL error that appeared the night before.

It was too late to have Dave fix it in time for the photos to go through the nightly batch processing (In retrospect, I really should have released them earlier in the day while he was in the office to trouble-shoot.)

I'm about 99.999% positive that Dave will be able to fix the problem in a jiffy, now that he knows it still exists, and we should have plenty o' photos in tonight's newsletter.

pretty close to normal, links on member pages not working right now

You'll get it back in to shape Dave - I have faith

Thanks Terry, EVERYTHING you and Dave accomplish is simply amazing. I really appreciate the update!

Things are working snappily now.

Yes, we had to wonder yesterday just what post/posts had got us on "permission denied" Perhaps the banned members could have gotten on then LOL!

THanks for the explanation dave.

at least now I know I wasn't being a bad girl:)

Like coffee, withdrawal can be painful. Now that we're all drinking our coffee again..... I guess you wouldn't have known about this issue if you wouldn't have tried. I've fiddled with PHP but frankly don't understand it and mess it up and admire those who "speak the language". I certainly appreciate this site and all the work behind the scenes for the community. Thanks!

hi terry i wish i could figure how to upload a photo faster on my end it takes fifteen minutes to preview.......but at least when i hit send it gos quickly.

Sounds like the photos are too large. Try resizing them to no larger than 800 x 600 pixels (680 x 460 is even better.) Also try reducing the resolution and/or saving them as .jpg files - all those will help reduce the time it takes to upload a file while maintianing sufficient quality for web purposes.

Bad news - at least for our newslettter readers and those waiting to have their photos released.

I think Dave took a well-deserved day off, and hasn't yet seen my note. So the problem is still preventing me from releasing photos. Sorry everybody!

In the Daylilies Forum ~ although I have refreshed the page, I still see several "Unauthorized Access" where the pics should be.

I was in San Antonio all day yesterday and should be working in the garden most of the day today (I can't let a 75 degrees day with all sunshine go by unnoticed!!)

Links on the member pages are fixed.

Terry: the PF Images queue seems to be working fine for me. Please give me a step-by-step way to replicate the error...

Moby: can I get a link to a page where this error is occuring?

dave

Thanks Dave the links work, have fun outside.

Dave, it's the Daylilies main page. I asked another DG'er if she saw the same thing and she did not. http://davesgarden.com/forums/f/hemhead/all/

This is what I see on several of the threads instead of the pics originally posted ~

Moby, when you see it, do a "Shift-Refresh" and see what happens.

(Hold down shift and while holding click on the refresh button.)

dave

OK, I did that and they're still there.

Whew - Dave got the SQL error resolved for PlantFiles, so photos have been released tonight - yeah!!!!!!!

I just received a private email from Jasperdale asking if I've had any problems getting on DG. Seems he keeps getting "Permission Denied" when he tries to go to DG.

Could you privately email him with some help please.
Thanks, Donna

All the permission denied errors should be 100% over by now.

dave

The number on the counter changed too.

What is up with that thingy? Why is it there?

mg

It's an experiment. It tracks how large DG is, based on a third party estimate.

Right now, in the entire world, there are only 17,554 websites that are larger than DG. The lower the number, the better.

dave

*eyes really big*

WOW! COOL!!

It will be interesting to see which single day gets the most traffic, and which week and month too.

Is the a link available to see the rankings? I'm wondering which are the top 100.

mg

Gayle, if you go to Alexa's site (they work in conjunction with Amazon, but they also have a separate site), I think they do have a list of the very top companies. There's actually quite a bit of information you can gather there, on any website that is sufficiently busy enough to garner a rating by Alexa.

I notice someone decided to add a new rating for us on the Alexa/Amazon site within the last few days - whoever it was, thanks for the kind words!

Your welcome Terry, hey I forgot all about being Rory Bellows on there - one of my Simpson's aliases!
Al
aka Rory Bellows
aka Joeyjojojrshabadoo

Jasperdale is back on DG, he had a friend help him.

Terry, thank you - I'm a happy camper now that the PF pictures are back in the newsletter :)

Uh oh... is problem happening again? Got a whole bunch of old messages (some 3 years old) highlighted on my page... and I have not been able to add a photo to a message stream this afternoon. Bummer

My watched threads aren't going away after I view them either.

The watched threads issues should be fixed now.