New server update

Just a quick update for those of you who are interested in following the progress of the server troubles we've been having since April 30th.

The new box was installed and transitioned to this past Friday, 3 days ahead of schedule.

The new box is doing great - very stable with few problems (the few problems that have come up were dealt with quickly as I became known of them).

It is slow, however, and the reason is simple: The Plants Database. Most folks don't know this, but the PDB is an absolute juggernaut and received over 120,000 different people accessing it each month. This by itself is not the problem, however. The big issue is that the PDB has so many complicated features that the server must handle, that it slows the whole boat down to a crawl when a lot of people are doing complex browsing and querying of our database.

The result of this is that there is much work for me to do on the PDB to make it optimized to be able to handle these features with the large amount of visitors. DG will be slow as a result, until I get the PDB nailed down.

One possible solution is to make the very best features available (the ones that use the most CPU - the big browsing features, advanced searching, etc, available only to subscribers, while non-subscribers get a more bare-bones "pictures, plants, and comments" PDB. It's worth a thought.

Dave

Way to go DAVE!!! you and the gang are doing a great job...
your solution to have the best availabe only to subscibers is good, however, dont we want to lure them in with that feature?

Dave, does this go back to the search engine issue that we experienced a few months ago (when you temporarily switched over to Google for some relief?)

Thinking outloud here:

1) IMO, cutting back the advanced search features would be fine, especially for nonsubscribers - do you have some stats on how many people use them, anyway?

2) Would it help to display only one or two images per entry, with a "click here for more images" for those who really want to see more photos?

3) Would disconnecting the link to the PDB thread for non-subscribed members have any effective, positive impact on the site's performance? (Could it be done so that whoever submits the photo, regardless of subscription status, is still able to access and post, but all other non-subscribers don't see the link?)

Terry,

Options 1 and 2 won't do a thing - those are very "light" features (in that they may be heavy on bandwidth, but are light on CPU).

CPU is where our bottleneck is right now, despite our having a very fast and expensive server up right now. The advanced searches (and even regular searches, to a lesser degree), as well as the big browsing options (browse by family/genus/species) are the real CPU wasters.

/search.php
/advanced.php
/b/
/latin/

are the big culprits. Coming in after that is the /go/ page (the plant's main page) which is quite heavy with all the extra information (botanary hooks, comments, ratings, checkboxes, etc). It all adds up to sometimes take up to 15 seconds or a little more to serve the page. While it is pulling it out of the database, all other transactions are locked and waiting in the queue.

By keeping some of the most heavy advanced features for only subscribers, we won't have the problem of hundreds of thousands of regular folks coming in and taking all the resources.

Again - I'm just trying to find a way that the entire server doesn't get dragged down. I'd like to provide some level of service to our paying customers, even if it means providing a bit less to our non-paying customers.

Dave

I'd DEFINITELY take off the Latin lookup link (where are people getting to it from, anyway? It's not even on the PDB page...)

And if it helps to take off the link to the "more information" (in Botanary) for non-subscribers, that makes it less useful, but could be okay...would the pronunciation still show up?

Still thinking outloud, (and knowing I'm NOT a programmer) would it be worthwhile to reprogram the search to have two (or three) search fields, and users can choose to fill in one of them:

1)common name
2)botanical name
3)cultivar name

That way, the search would only have to chunk through one (or two, in the case #2) name field(s) rather than all of them.

We'll have to see. I'm devoting some serious time to the PDB optimization today (it's all I've done today so far) and I'll have to find the best way to do things. A lot of it can be handled with better programming (optimization isn't a priority when you have a small database, but it becomes crucial as the db gets bigger).

We'll just have to see. I've got some serious re-writing of a lot of code to do.

dave

I think that the way you go about discussing the back end of this site in the open so DG's subscibers can understand the complex process of developing the underlying infrastucture is totally first class.

Adam.

Personally, I think that a more limited non subscriber option is a reasonable one. Right now the amount of free information exceeds what is available on many paid members only site and for you and a little under a 1000 subscribers to foot the bill for almost 47,000 people is not realistic let alone those who hit and run! Your little hobby site has become a business and should afford you at least some profit in addition to paying it's expenses.

I think the site is large enough now to continue membership growth without giving so much away for free.

Just my opinion!

I agree, Zany. And it might make subscription more appealing to the frequent PDB visitors.

I agree Dave and Zany.
To be honest: I agree most much :-)

People can pay that little money for that great information site.....
YMHO

I would just like to say that you are one hell of programmer analyst if you are responsible for all that I see on this site. I don't know what kind of a gardener you are but you're bloody brilliant in the tech department. This is most easily navigable and informative site that I have used for any subject on the web. And you seem to have the most techno literate and capable membership as well. Does anyone but me not have a digital camera?

jagonjune, I don't have one! Still wishing, though. And I agree with your description of our webmaster, what a master of many things!

All in all, this transition has been unexpectedly smooth. I love it that Dave has thought of us every step of the way, trying to make sure we have all that is possible in the midst of mega problems.

Hi Jagonjune, I dont have one either. I take pics and then take the film to be developed and order it on a CD or diskette. Sometimes I dont even get paper pics, depending on what is one the roll. I am still just learning, though.

Aimee glad to see you are back.

Dave, I agree, let the paying customers have the most benefits. It will make it more tempting for serious gardeners to join, and for those who are already members, it
will be a reason to renew.


Edited because I can spell but cant type.

This message was edited Monday, May 19th 10:43 PM

I know others are saying it, but DG is running so slowly, even at off-peak times here for us, and we have a cable connection. If subscription-only access to the PDB's processor-intensive apps would help things overall, I'm all for it!

LOL, GW I was just thinking the same thing! Paying for high speed acess is a strain on our budget but worth the expense, gut now I am loading DG at Dial up speeds and can't imagine what dialup users are going through!

Groaning here! Now you know, Zany!

I sure was shocked at the low number of paid subscribers--just does't seem fair for all the work you do so well, Dave.

I was surprised too - I had guessed AT LEAST 5-10 percent.

Just think - all those peeps who dont know what they are missing!

I try to promote it as much as possible.

Dave, I know you are very busy right now with getting everything lined up the way you want it. I just want to pay the $15.00 for my daughter to become a member and i don't know what the procedure is to pay for someone else. Or anybody who know what i should do. Thanks, Donna

Rutholive -- I replied to you in http://davesgarden.com/t/381268/ but there have been posts since then and it might have gotten lost in the shuffle.

You must first sign your daughter up as a member, making up a member ID for her and entering an e-mail address (she can change the e-mail address later if you want to put in your own e-mail and therefore surprise her). Then, once the member name is created, you can use the subscription http://davesgarden.com/subpreferences.php to sign her up (you'll get a chance to say who it is you wish to subscribe).

This message was edited Tuesday, May 20th 12:23 AM

I'm new here so don't know that much, but I didn't have any trouble paying for the membership I've spent more on books that didn't tell me what I wanted to know. The savings from all the plant knowledge will easily make up for the cost of the site. I just started gardening and don't know anything. Without this site I would've lost a lot of the plants I just spent money on. I do have a dial-up modem and it has been slow since I joined the site but knowing what the cause is makes it easy to put up with any delays. Dave, you're working so hard on this site I think it's the best one I've found on any topic and definately the easiest to use! When I joined the plant data base wasn't even working. It was a thrill to finaly see what everyone was missing. I do have a dig cam but don't know much about using it was thrilled to have one of my pics be forum of the day pic! This site is also easiest I've ever tried to add pics too. I can't even send pics thru AOL to friends half the time cause their picture thingy doesn't work right. Rambling again! Sorry - basically just keep up the good work Dave we all appreciate it! Tracy

I totally agree with limiting some use for the non members if that will help the problem. It might also be an added attraction so more people will become members. As a new member I can tell you this site has others beat hands down!!

Yes..I agree too..I think a lot of people won't subscribe because they can get the info for free..and don't want to pay to be able to post..
The 15.00 U.S. (about $25.00 in Can.) is the best money I have ever spent..it is only the price of buying one magazine subscription for a year..filled with ads of course.
Any time I have a question or want to share something..I get imediate response..I love that!
Thanks Dave. By the way I have dial-up..and it has been slow..
but worth the wait.LOL
Cheers,
Carol

if you give away the milk, no one wants to buy the cow.

Yes, you must limit the subscription to paying people, because we really care about plants enough to pay for the database. Many of those other people are just casually surfing around, it does not mean that much to them.

DG has become so important to me since I've joined in March.

The $15 spent was a wise investment, considering all the plants I've bought! The GW has been especially helpful!

This site has a wealth of information to so many different topics, and is well worth the annual subscription

I'm in agreement with the above posts!

Dave, thank you again for sharing what goes on technically! That is, IMO, a true sign of a generous heart! You've earned your wings, Dave!

What's hard is, people have a perception that everything on the internet should be free. That's starting to change as more sites, such as many of the major newspapers, are going to a subscription basis. We all need to get the word out about DG. Maybe we could write our local papers' gardening writers and try to get some interest going for articles.

This message was edited Tuesday, May 20th 2:02 PM

The internet is a big business and while many things start free, Like Webshots, several phone /msg services, etc.. as they grew and people became more familiar with them they began to charge for their services. Small personal sites that just use a very small amount of memory are usually kept small beacuse of the costs involved when they grow like DG has. Dave's little journal and one forum has grown by leaps and bounds. He has had to block it from some search engines to avoid overload in the past and upgrade upgrade upgrade to keep up with the demand. When he went to a subscription basis many of the most valuable assets on the site were and are still available to non subscribers and somethings like the PDB are contributed to by many non subscription holders. But if the features like Search are limited or removed from free access it will speed things here and possibly increase subscription traffic as people see the list of things they are missing. This site is large enough and well known enough now that it should be entirely self supporting AND afford it's owner some reasonable income and compensation for his work. By limiting some access Dave has stated the site could pick back up to speed. That would be great for us subscribers and much appreciated!

Zany: You have a fully complete view of my thoughts on this matter. I could have written what you posted.

Dave

Zany
Very well said. I second your thoughts.
Its well worth the money.

Paul

Very well said Zany!

I also think you bring up a good point, there are many non-subscribers who make contributions to the PDB and DG's similar features. If you restrict the features entirely to subscribers only then we lose the best advertisement we have for DG.

I think that restricting part of the access to the free features is a great idea, the occasionals can still use it and, perhaps, even if they don't subscribe, they may well be telling people about who will.

Dave, just do it...:-)

ditto
and thanks for fixing my forehead Dave. teehehe
kokopelli

I am glad I was able to put it out there. I just know that is how I would be looking at it if it were my site and as a member. Whatever you decide to do will be fine and dandy with the Zany one :~)

I became a member with dave's because of the data base. If I had not run across it on a internet search for a unknown plant I now could not enjoy this wonderful site.

Is there a way to set up a preferential search from within Dave's for members? And a separate one which could be accessed from any given search engine?

Iam no cyber-ezpert. But when the open-access one would be too busy, sorry Charlie. But members who entred form within the DG site could still hae avaiable access.

I am thinking of http://www.mexconnect.com where I have been the food forum moderator for many years. That website and this one have much i common. When David (equivalent of Dave here...hmmmm.....Dave - David?) decided to go to a pay site, he set up a new web addy http://www.mexconnected for paying members, and mexconnect remainned ro no-payers, with a much more limited access.

Paying members have full access to al current features and past archives.

Just fyi. Dave if you would like his personal email, would be glad to give it to you off-forum

Dave I agree, just do it.

Time to fess up. I did it immediately after first posting it here. :) As a matter of necessity, I had to pull those features for the time being, so I did that by just pulling it for non-subscribers. They currently get a "temporarily unavailable" message if they try to access it.

I had no choice but to do it. All those people converging at once on the PDB slammed the server too harshly. Things cleared up quite a bit after I limited those big features to subscribers! :)

Later on, if Vols, Trish, and I do ultimately decide to go forward with this, I'll clean it up and give a nice presentation and such, with an interface to a subscription area.

The PDB will look and feel the same as before for non-subscribers, but they won't get the advanced search, perhaps only a few regular searches per day, no browsing by family/genus/species, and they may be limited in other areas too. But, because so much data comes from non-subscribers, I want to be very careful not to limit them too terribly much. I know they would be understandably upset to be limited from access the data that they put there!

Dave

So have any of the PDB non subscribers emailed you yet about the change? Or are they like the rest of us and just happy to be back up and running so they really haven't had time to realize it?

Well from my end I having a ball zipping from thread to thread like before. Good going Dave. Thanks

I have tried several times to day to get in the plant database to lookup woodworm astemesia sp & some other new plants I have with no luck. I get the temperary unavailable message. I know I paid the bill *LOL*
Sugar