The system DOES need some re-alignment !! (dooyoo - Members' Opinions On How to Rate And Comment)

Member Name: sidneygee

Product:	dooyoo - Members' Opinions On How to Rate And Comment
Date:	05/03/02 (94 review reads)
Rating:

Advantages: Yeah ?

Disadvantages: Oh Yeah !, SAY OH YEAH!!!!, OOOOH YEEEAAAH!!!!!

When I wrote this review, I posed the question "Does the system need re-alignment ?

Recent developments have certainly persuaded me that the system DOES need attantion beacuse in the current Anarchy of DooYoo, a number of excellent writers have ceased writing and/or left the site completely. In my own case, I have admitted that pressure of werrk (I NEVER use 4 letter words) is the major cause of my not penning any new reviews. However, the pernicaty behabiour of one or two members (which would take up time to answer through the inevitable comment thread) has cveratinly also been an influence on my current usage of the site.
I have provided an update dealing with this predicament at the end of this review.


So !!!
Rating a DooYoo opinion ?
Dead easy is't it ?
After all, pretty obvious isn't it ? :-

"Very Useful" (VU); "Useful" (U); "Somewhat useful" (SU); or "Not Useful" (NU) – all very straightforward..... Or is it ?

Right, there is guidance but do we need it ?

Get away with you! Of course we don't !!

We can each decide the "usefulness" of an opinion to each of us....

We are NOT idiots – WE CAN make up our own minds, can't we ?

Well OK, so how many have ever read the blab given in the DooYoo Ethos section ? If you did, it was probably right back when you started and it ain't really that informative, I can tell you.

It starts off with what we all know –

"Dooyoo is a revolutionary concept and a uniquely valuable resource for people wishing to make better decisions in life. The medium of the web has allowed us to communicate on a scale never before seen. It is for this reason that the dooyoo community relies upon its members to be honest, decent and democratic. Please read our guidelines and understand that dooyoo will serve you for as long as you ser
ve dooyoo."

And that "Only registered members of the dooyoo community can rate".

But what about the rest of it, particularly as regards 'rating' ?

The "Best" In The List?
It claims that "rating opinions helps to ensure that the best opinions can be found quickly by other users: the most popular opinions appear at the top of the list?".

This "list" is obviously that "list" of opinions in the Category, listing the opinions on a particular subject in an order of 'merit'.

But naaah!!! Come off it !! That ain't really always the case is it?

The inner sanctum of the near-canonised generally tend to make it up the top of the list, sometimes irrespective of the real quality of their opinion .... to be perfectly honest, they are not always "the best opinions".

Usual Drill – "No Names" - but there are about 30 or so, I reckon – for the sake of argument, let's just call them 'the Usual Suspects'.

You don'y believe me ? Just you have a wander around the site and look at the lists of opinions in different categories. It can frequently be a revelation. Please don't restrict your reading to the 'Suspects' at or close to the top of the list. You can often be very pleasantly surprised at the quality that is there from those you may not ever have heard of and, by comparison, those of the 'Suspects' can on occasions seem quite lightweight. Also it is useful to look at those 'suspects and examine their VU hit-rate. That is the ratio on "non-VU to the total number of ratings" eg : 4601 total ratings - VU ratings 4489 (97.57% VU). Just check on the 'usual suspects', you may just be surprised.

OK, this is not always the case, since the 'Suspects' have each produced some right 'Crackers' in their time, but as wit
h all of us (including even famous journo-reptiles and authors), we can have our off-days, and the positioning at the top of the list because lots of members have read/rated may not be fully justified.

Indeed, often an excellent (sometimes) crowned opinion will appear quite low down in the hierarchy and - believe me - it can be a real 'gem'. Much more informative, better researched, entertaining, or even more 'moving' than those by the 'Suspects' at or near the top of the list. Just try browsing a bit on occasions.

And whilst we are on the subject, have you worked out how the positioning of these opinions in the list is controlled?

The lists are basically in order of number of member reads but grouped, so that those with an overall 'Very Useful' rating are listed first, then those with an overall 'Useful', then those fewer with an overall 'Somewhat Useful' rating and the one or two 'bums/churners/plagiarists' with an overall 'Not Useful' rating. Thus if someone has only just got an overall 'SU' rating but with loads of reads, should someone rate the opinion 'VU', then that opinion will go 'shooting up the list'. In my early days when 'learning the ropes', I amused myself by doing just that – over-rating or under-rating an opinion to alter its position in the list, flipping it over from 'SU' to 'U', or even from 'U' to 'VU', or any of the other permutations.... Well, I lie – but I really have seen my rating of an opinion make the difference in its overall rating, and thus position in the list. So opinions at or near the top of the list may have far fewer numbers of 'VU' ratings than 'better' but less-read opinions further down. Even rating an opinion as 'NU' or 'SU' (as long as it doesn't alter its overall 'rating') could move it up the ratings list
since it would have more 'registered reads'.

Be much more effective if it was done on a points basis, so that, say VU = 2, U = 1; SU = minus 1; and NU = minus 2. Be a much fairer system wouldn't it and perhaps leading to a better rating strategy by members.

"Usefulness"
The Ethos goes on to argue that :
"... the level of 'usefulness' should reflect your view of an opinion's ability to inform the next user about the product or service. If the opinion is not on a product or service, please rate according to how well argued and presented the case is."

Well, that sounds reasonable, I suppose. But, in reality, it can be very difficult to be objective and it is pretty obvious that DooYooers all operate their own criteria (which may be quite idiosyncratic).

Look at your own opinions and those who have rated your opinions against the tide there is a range of idiosyncratic behaviour. Indeed, whereas I cannot complain overall about the ratings that my opinions have received, some of them leave a bit of a nasty taste. The proportion of 'VU' ratings that I have is over 97 per cent. Within the "Not-VU" ratings there are just eleven 'SU' and nine 'NU'. Arguably, I reckon that three of the 11 'SU' and five of the 9 'NU' are 'retaliatory' where the rater has objected to my low-rating of one or more of their opinions, and retaliated – not by rating the same, but either 'just' lower or 'just' higher. Interesting that these occasions are characterised by there being a gap in between the retaliator and the next lowest rating. Thus, with these 3 'SU's, there are no 'U's, and with the 5 'NU's there are no 'SU's. Veeeery interesting,... The problem is in proving that the ratings are 'retaliatory', particularly where the rating is slightly different, and since there ar
e one or two DooYooers who have had alter-egos that I reckon they have used purely for retaliation. Now that really should NOT be allowed, should it ?

The other four 'NU's are from those who just disagreed violently with some aspect or another of my opinion, and as for the rest of the SU's – who knows ? At least four or five of them were perfectly justified on my very early opinions. However, surely you should not rate an opinion as 'SU' or 'NU' just because you disagree with it ?

Generally, if I do not agree with an opinion and find no real new merit in the argument, I will consider just not rating it (leaving a comment behind). I am loathe to do that, because it is only by rating that you can keep track of how much reading you have done of other opinions. If there is a reasoned argument with valid debatable points then, even if I don't agree, it gets a 'VU' or a 'U' rating. So, we each practice our own little foibles when rating.

But does Dooyoo really help us in this matter ?

Well, in the Ethos, the Op-Meister reckons that the rating system we adopt "... should reflect the following criteria":

VERY USEFUL : covers every aspect of the subject in considerable depth, and makes both the positive and negative aspects perfectly clear. You would make a purchasing decision based on this opinion.

USEFUL: a good opinion which states the case clearly but perhaps focuses too much on a single aspect of the topic. You might make a purchasing decision based on this opinion, but you'd look for more information first.

SOMEWHAT USEFUL: an opinion that would have been better with more depth or a more balanced coverage.

NOT USEFUL: the opinion is very one-sided and lacks sufficient depth to really do justice to the topic.

Now I dunno about you, but I reckon that very few (apart possibly from a-true-ben, grahamt, ermintrude and
blackjane) who seem to adhere to this system. So surely we must ask if these guidelines need updating? In any case, how good are they ? In my honest opinion, not good at all. Perhaps these criteria were reasonable when Dooyoo was in its early days, just dealing with assisting in the making of purchasing decisions but hell, the site now has more dimensions than Sci-fi, and I reckon they are well-past their sell-by date.

DOO WE NEED NEW GUIDANCE?
So what do I object to most about these criteria ?

Well I cannot accept that an opinion should cover "EVERY aspect of the subject in considerable depth" to be rated as 'VU'. For goodness sakes, particularly where there are a number of opinions on the subject, the very basic information surely does not need to be re-iterated every time ?

I will give an example here. A long time ago, I wrote a carefully-considered opinion on the RSPCA. The Charity had received an amount of criticism from a variety of writers who generally objected to the Charity's pragmatic approach to dealing with injured wild animals. I tried to give a balance to the overall argument and in doing so gave a fairly detailed account of the dealings that I had had with the Charity when I was a Public Analyst. I felt I was able to provide something to try and restore the balance in favour of the charity. Overall, a little disappointed to get several 'U's from those apparently 'toeing the party-line' and some 'ribbing' in the comments thread from one.

Now those from the same mould also kick up stink and rate 'NU' if they feel that an opinion has been placed in the wrong category or does not comply exactly with the requirements of the 'blab' at the top of the category. Almost as if they are teachers, determined to give you an 'F' grade if you have not covered the 'essay topic' in the detail and from the direction demanded..... the
re are obviously too many 'frustrated' teachers out there ... grrrrr. By all means point out that it is in the wrong place - contact the DooYoo Guide if you feel suitably incensed, but IMHO the opinion should be rated on merit. Once rated, they are rarely re-rated ...

The Ethos goes on to recommend:
"Do Rate honestly. Don't be afraid to rate opinions 'Not useful' and be sparing in your use of 'Very useful'.

"Remember always to rate opinions. Rating is crucial to the effectiveness of dooyoo."

Now far be it for me to say that the whole world should operate to the SidneyGee system but, whenever I read a Dooyoo opinion, I do expect certain things. I should say at this point that I am normally not disappointed, so by far the majority of my ratings are 'VU'. If 'VU' opinions appeared infrequently, then I would soon lose interest in the DooYoo site, as I would with a newspaper or magazine that 'failed to deliver'.

For a start, I expect to be 'informed', 'entertained' and preferably to be LEARNING something that I didn't already know when I read a promising-looking opinion (– always plenty of scope there !!). And, as indicated above, even if someone has a completely different opinion from me, I will still usually rate 'VU'. Should I find an opinion only SU or NU then, unless the reason is obvious, then I do try to comment on why I find the opinion 'wanting'.

As far as I am concerned the worst weakness that an opinion can have is where erroneous or plain wrong information is given out. I usually make a very robust comment when I feel that this has happened and rate at SU or NU, even if the general tide is towards 'VU'. But instead of booling on, as is my worst wont, I shall focus on alternative suggested classifications for ‘rating’:

NOT USEFUL : the opinions lacks sufficient dep
th and breadth to really do justice to any aspect of the topic, or contains sufficient wrong information to give a seriously distorted view of the subject;

SOMEWHAT USEFUL : an opinion needing more balance, depth and/or breadth or which includes some provably wrong information or errors that detract significantly from the quality of the writing;

USEFUL: a good opinion which is essentially accurate and states the case clearly but without adding any new information or offering no different angles on the topic;

VERY USEFUL : ideally an opinion essentially free from any factual errors that covers the subject in significant depth, making both the positive and negative aspects perfectly clear and, where appropriate, adding further dimension to the topic in question. However, as pointed out by thevenerablebede in the first comment on this opinion, there are rare occasions when an opinion introducing new factors into an area will merit a 'VU' even if it it lacks breadth and length.


So, that is how I try to read and rate. But Oh to be 100% objective !

Seriously, if you ever feel that I have rated your opinion unfairly, please email me and let me know. I am perfectly capable of losing my way in this life and making a mistake ....


Comments
Well, the Comments section is pretty well open to leaving all sorts of chat, that may or may not be derived from the opinion itself. I don't normally see much wrong with that and I enjoy the inter-change of ideas that can take place there - my favourite part of the DooYoo experience. Not ideal perhaps, but what's the odds? Could not see any viable and easy way of changing this part of the Dooyoo experience until those with the time & inclination to engage in Ciao mentioned the "Guest Book" aspect.

Sometimes however the debate can get quite heated and it can be easy for a writer or a reader to take 'offence'. Indeed, situati
ons can develop when comments are made that are seriously bordering on libel. Now, the policing of the site is now much less since the departure and non-replacement of Jo* and it can take some time for 'abusive comments' to be removed – and you have to request this yourself.

Again, if you feel that I have been unfair with a comment, please let me know. I might even apologise.

Footnote
OK I will admit that I wrote this rant/opinion a long time ago but I did not reckon that it was sufficiently 'robust' to post up. I was clearing out the Files on my Laptop over the weekend (pending replacement) and came across it.

However, my recent couple of 'navel-gazing-opinions' about other aspects of DooYoo seem to have stimulated a bit of debate and interest (including a comment from the Op-Meister-in Chief himself).

So how do you all feel about this one ?


UPDATE 9 May
Oh how things have changed on DooYoo.. As indicated above, the most serious problem seems to be the permanent loss of excellent well-established writers.

Scotgirl (for whatever reason? - I don't know) but both pje and ANDREWSJK have left for the same reason - because there are several well-established members (who just happen to be Guides) who have a habit of rating reviews against the tide of rating. For example, SU (or 'just U') against tens of VU's. They back it up with (shall we call them) either 'snidey comments', half-hearted attempts at justification, or by 'aping' the reviews they have read and getting their 'criticsim' in there.

That Charlie, <chuckle> has raised the issue on OpCom too, so it isn't just me who has taken this viewpoint.

I have experienced this behaviour from the same people and my own reaction has been very simple - just NOT to read/rate any more of their reviews. However, as indicated above, their activity has inhib
ited my posting of reviews, since I never want any of those creeps to have the 'last word' for any length of time on any of MY comments threads.


So why do they do this ? Several possible reasons. Trying to make their name stand out possibly ? Or, by taking an alternative view, trying to suggest that 'they' have greater insight than the review writer or those others who have rated fairly ? Or can it just be jealousy ? OK it may be occasionally a genuinely held belief based on careful analysis and weighing up the pros and cons. But it is usually the same people who do it, and far too often....

So, how to deal with this ? Well, a better guidance for rating, as suggested, and also a better 'ethos' requirement could be useful - oh, and certainly a requirement for the creep in question to offer far more justification for rating against the flow - and a firmly imposed 'Code of Practice' for the Guides.

I proposed such a thing when I was a Guide - but this was shot down by these very individuals who now seem to be driving good writers over to Ciao.

Without some action, then this will surely be just another nail in the coffin of DooYoo.

Just my latest 6 pennorth'

I wonder if I should grop over and give Ciao a try ...

© Sidneygee 2002

Summary: