Watchdog -Criteria for Top 30

After having 2 out of 3 transactions go bad with a Top 30 company I am wondering what is the criteria used to establish this rating? I have incorrectly assumed that the Top 30 were the best companies but now after reading so many negative experiences and neutrals which should really be negative in my opinion I am wondering if there is a need for a Top 30 designation. A lot of companies have gotten my business just because they were in the Top 30 and most deserve it but there is one in particular that has so many negatives I don't know why they are rated so high.

Dorothy

Hmm, that's very odd. The top 30 is completely created based on a computational algorithm that looks at the reviews and scores the companies based on them.

I don't see a recent review by you. Which company are you talking about?

I wrote a review on Buried Treasures today. After posting mine I read thru other postings and can't figure out why this is a top company and am suspect to companies that are not on the Top 30 now. My thinking now is if they are this bad and on the top what in the world are the others like.

Dorothy

Well, things may break down when you use a heuristic algorithm just as we're using to generate the top 30. My desire is that the top 30 really does represent the very best of the companies in the database, and I think it's done a decent job of that over the years.

I see that BT has received 10 negative comments in the past year (8 of them in the past 6 months) and 36 positive reviews during the same timeframe. There is a giant penalty for negative comments, and if even more come in then this company may be bumped off the list.

Maybe it's time that I once again looked at the algorithm that I'm using.

In the meantime, I think the rest of the companies on the list look pretty good.

When you give feedback under Watchdog the changes are made to your previous postings about the company. Unless there is another way I don't know of. I changed my previous feedback rating and added reasons today.

Dorothy

Yes, and by changing your review to "Negative" you are helping to effect the ranking.

I still don't see where my posting is visible or changed the number of negatives. It was at 10 before I added mine and I still see only 10 negatives. Anyway, I won't be throwing any money their way anymore.

Dorothy

The 10 negatives that you're seeing at the top of the page are just from the last 12 months...if your original review was from longer ago than that then I don't think it wouldn't show up in the last 12 months counter because all you can do is update your old review. If you scroll down a little and look at the total number of their negatives (which is currently 21) that is the number that should have gone up after you changed your rating to negative.

How long ago was your original review? If it was more than a year ago, it won't be displayed as a new negative in the table near the top, which records the most recent reviews for the month, six months and one year, but instead, it simply replaces your original rating with the new one in their overall count.

Ooops, ecrane3 and I cross-posted. Yep, what she said ;o)

Thanks for clearing that up for me.

Dorothy

Admin, Dave, Terry,

I don't think that is right. Nearly 1/3 of the feedbacks are negative and they are in the top 30?? I don't care how many positives they have, 10 negatives in 12 months is terrible. Maybe they need to hire more people. I have to aqree with Dorothy. I think whatever process you are using is off kilter and some pretty good companies might be being left off instead of these bad ones.

Jeanette

The problem is, that this company has received 429 positive comments over the years. That history weighs in their favor and despite having bad reviews this year, the older comments continue to push them into the top 30.



I completely agree, and I've spent my morning looking at this. I'm going to continue to hammer at this until I have a good solution (and I think I'm getting close).

Maybe they are having employee problems, or something to cause them to go downhill this year. But, now is what we are dealing with in ordering plants from them. Not "over the years". Maybe those years are being given too much weight.

Jeanette

This is the crux of the issue.

I have adjusted the algorithm to more thoroughly punish companies' scores for recent negative reviews. I also weighted the scores even more heavily if they occur in the last 6 months.

Moreover, if a company has too high a percentage of negative comments in the last year, they are automatically disqualified from being in the top 30.

So, looking at the top 30 right now, it looks pretty good to me. The company in question is out because of the high percentage of recent negatives.

That sounds much better Dave. Especially for us who use the Watchdog to determine who to buy from. Thanks for the thread and listening to what we have to say. Nice to have someone listen. Otherwise the site would be worthless.

Jeanette

"For gardeners, by gardeners". ^_^

I also use the Watchdog myself to determine who to buy from, so I'm also keenly interested in keeping the rankings as accurate as possible.

Dave, have you considered adding a column after the vendors name in the top 30 as to what they specialize in? i.e. trees & shrubs, perennials, seeds, houseplants, roses, bulbs, etc.

I think most of the vendors listed are either East coast or Southern, and since we on the West coast are paying more money for postage, it might be helpful. Yes, you do have a lot of users from the West coast other than California. And, we do buy from those vendors.

Just a thought.

Jeanette

Thanks Dave. Those changes really help.

I was wondering if there was a way to show the revised updates as new separate transactions because that is what they really are? And keep updates for updates on the original transaction not for new ones?

I was looking at dobra1629 new updated review and it is buried on the second page of Buried Treasure's reviews and all the way at the bottom of her original review. Kind of out of sight though her unhappiness was based on a current transaction in September.

As Liz and Terry mentioned, dobra1629's new review does not show up on the "Most Recently added ratings" list on the main page for the Garden Watchdog either. Or the recent 6 month or 12 month box under Recent Reviews on the sellers page either.

A new review would put it on all the current lists where it would be easily seen.

If there is no way to do a new review, could at least the old review with the new update be bumped up to the top of the reviews so people could use that new information in their decisions? And also effect the last 6 months and 12 month listing?

Also I noticed Buried Treasures also has 2 Garden Watchdog Top 5 plaques on their page. Maybe they still are in that grouping though.

Thanks Dave for looking at this!

In adding to Kell's post, there is no indication when, what year they received the plaques. Could have been 30 years ago. Lots can happen in the last year as your figures show.

I think the Watchdog is more than to "punish" a company. It is to help them improve, and for us, the users to decide who to buy from.

I am still going thru Kell's post to figure out what she means. Reserving my right to discuss. lol

Jeanette

The Top 5 isn't well documented, but it is an award given for a given year. So, in January of 2010 we will be taking all Top 5 awards away and repopulating the Top 5 (of course, I imagine many companies will just continue to hold the honor).

So, if you see a company in the Top 5, that means that it was awarded this year and will remain through the remainder of the calendar year.

I think you mis-read me above. I never said it was here to punish a company. Re-read the post above and you'll see more clearly what I was saying.

The Watchdog does far more than simply reward or punish companies.

Kell, I see and hear you and I've thought through this a lot. In the beginning, I didn't want people to be able to review a company more than once as I saw that as a possibility for abuse. So I designed it so that you can go alter your review and add more information to it.

But then all these years later, companies who we once thought were great are now not so great (and vice versa) and we want to be able to change things to reflect that. Bumping up the date on the comment to bring it to the present is probably the best solution I can think of, and I'll make that happen.

lol, I stand corrected Dave. I think the revisions will make it much better. It certainly takes a lot of thought doesn't it? Thanks

Jeanette

I see your point, Dave. I guess it is true some people may get carried away if they feel like they were really cheated. I am still on simmer over being cheated by a grower. Cheated then ignored. LOL

But I would think that my vendetta would be fairly obvious if I kept saying I bought more plants from that guy and then kept rating each transaction as bad. You would think after being burned once, that would be it. Though you have to worry about the reverse too don't you? They are your friend so you write lots of positive reviews.

Maybe it would work if you can only write 1 positive review and then follow up positive remarks on same post for future positive transactions but if you go from a positive to a negative you are allowed one more separate review.

Maybe that is too complicated. LOL

Thank you so much for fixing it, I am sure you know what will work best.

Kell, I think the problem is that not everyone reads the reviews. So if someone went to the extreme and posted the same thing over and over again--whether positive or negative--it would rack up the positive or negative "points" for that company. We made some changes a while back to encourage people to read the reviews instead of rely on a numerical "score" and that seems to have worked. But letting people post over and over again would really open the door to "beating the dead horse" syndrome ;o)