Knol Reviews

"Thank you for your feedback! You have answered all questions about this knol. Tell us more by writing a review." - This is such a great lead into the review.

The first question makes for an easy start, answering more question is a step up but with a little more thought it's not so hard and then by the time you get to the end you are primed and ready to review. It was actually difficult not to do one, rather than the opposite. Brilliant upgrade.

Looking more into the question set, I'd like to comment on them overall, on one question in particular and on "next question". I've used the set maybe 10 times.

Overall: These questions don't fit certain kinds of Knols. For example, they really don't work for recipe Knols. One solution is to offer a third choice response on each question: Not Applicable. Another solution is to allow authors to select a question set from say five different choices.

Question 5: Q5 is written in the pejorative, as if a YES answer is criticism. I'm considering writing a Knol that compares or reviews a class of products designed for a particular use. And of course I'd link each product to its offering page. A better question would be "Is this Knol blatantly commercial, with clearly biased advocacy favoring a product or service?" Yes/No/Not Applicable

After answering a question, I'd like to see the next question without having to ask for it.

Thanks, guys and gals.

nobarking

The individual Review Scorecard questions are not really appropriate for the Review Knol that is created in the review process. Maybe turn-off this feature on a Knol created by the review process in the same way that you turn-off the Review feature for this special Knol.

Just a thought!

I suspect that the majority of visitors, and even Knol Authors, don't know that behind the Review link is an easy to complete Review Scorecard questionnaire.

So, how about an entry link where one of the questions is made visible in the Knol Project side bar. It should be in a box all by itself with the accompanying sliding scale and if possible placed up or near the 5 Star ratings.

I think that as many people as complete the star rating might be interested in contributing an answer to the proposed question. When they have answered this question the Knol Platform takes them to the full questionnaire and invites them to complete the remaining questions in the series. Seeing the ease at which this process could be completed, should encourage the majority of readers to do so.

A similar idea is to replace the “There are no reviews for this Knol” with one of the questions, which when clicked takes the reader to the Review Scorecard with the same invitation to complete the series.

As for the question to ask? Well obviously the one that generates the most responses. Each question could be trialled on different or the same Knol at different times to see which one generates the most responses. Another option is to set up a random rotation of the individual questions into the box mentioned previously.

This idea may be the bridge needed to encourage readers to move from just math ratings to the much needed human opinion review.

Peter -- I'm responding via a new comment so it's directly visible to all.

It's hard to fault you. Your logic is tops. I vote to put you in charge.

Let's face it: Knol generates income, and income has valid uses. Some small amount should be used to encourage the quality upgrade (taking out the trash, ending the plagarism and spam) that you and I fervently want.

A great model for quick reviews is www.pollingpoint.com. I'd like you to try it out, look at the rewards. I've done many polls and am very impressed with the layout, ease and speed. I think this model can be adapted to Knol. Insofar as rewards are concerned, sure, citizenship is its own reward. But a new digital camera is a great incentive, too.

Having been involved in promotions for many clients, I know that the cost of a program like this is not that high. A small portion of Knol's earnings could go into this. I'd like to hear from Knol Help on this. Wanna play? Gotta pay. Now is the time to do the marketing, while we are in the trough.

Let me know your opinion of pollingpoint.

Murry

OK, couldn't sleep. On top of QuikScore, Knol needs to offer full written reviews. How about 5 points for a Basic review and 10 points for Advanced review? Such a reward system would help build a meritocracy of reliable reviewers who support Knol as a habit. And it would pick up QuikScore reviews from transients and regulars alike. Reviewers could actually do all three, earning 16 points.

I'm uncomfortable with reviews becoming separate Knols. The logic of that escapes me.

Looking at this discussion in conjunction with Baskerville's articles on writing reviews, I had an inspiration. Here's the logic train, and it's based on Baskerville's Impressions Table.

First, maintain the current star rating

Second, offer QuikScore Review. If a registered reader takes a minute to QuikScore an article, they earn a point credited to their profile (we'll figure out rewards later). Here's a model for QuikScore:

QuikScore: Was this article worth your effort? If it's average to superb, take a minute to QuikScore it. You'll earn one point per review. Top reviewers will be featured in Knol's Honor Roll and may be eligible for annual prizes.

1. Arrive at a set of 10 rating/review questions that can be answered on a rating scale of 1 to 5 (as opposed to yes/no)
2. Program the questions so that each radio button selection will be counted
3. Answering the questions produces an overall score in the range of 10 - 50, with 50 being the top rating
4. Multiply the score by 2 because the final number would then align with our education system, which is based on 100 = A+ = 4.0 GPA = Excellent/Superb/Brilliant/Unsurpassed
5. The algorithm should have subroutines to detect and reject vandalism, such as authors with access to many computers, mobs enlisted against articles (think conservatives vs. liberals), etc.

OK, my dollars worth. Now let's form a subcommittee to build those questions.

nobarking going back to sleep

Would you believe that I got a rare Blue Screen of Death while writing this comment? I use XP and it's only the second time. However, the first time was last week.

OK, do what Peter suggests to improve the rating system. Maybe Knol's paid guys could form a small subcommittee with the mission of building the BEST rating system. I'd be happy to participate. Those questions being asked really need help. There are much better ways to frame the questions, and they should begin bottom up, leading to a conclusion that is consistent with Knol's mission. The first question, trust, should not be asked at all.

Another point: having high page views does not make an author a quality reviewer. Check out HowToDoThings, where the highest page-view authors have many published articles that no one rates. For example, on March 21, the top rated author submitted 12 articles. This author has written (or cut and pasted) over 400 articles in less than a year. Fewer than half of those are rated (5 star rating system). I would not want that author passing judgement on my kind of articles.

I note the changes to this Knol and the review questions and offer the following impressions as a Knol Author and Reviewer.

1 – I think that it is a positive step for the project that the Knol Team have now clarified the concept of the 'Review' system for a Knol. The decision as I understand it, is to make it a 'Review Scorecard' and separate it conceptually from the scientific peer review system.

2 - To ensure this clarity is consistent throughout the site, is it possible to now change the term in the Knol display to 'Review Scorecard' and not continue to leave the ambiguous term 'Review'

3 - Given the general acceptance that reviews should be and will be of limited number, is the statement "There are no reviews for this Knol" a fair one. Its a negative statement that I think implies a problem. How about converting this to a positive, suggestive one like "Impressed? Complete a Review Scorecard for the author now".

4 - Authors like Krishan, myself and others have been very concerned about the Knol review process and I think we may have unfairly asked the non-editing & non-domain expert Knol Team to solve something that is fundamentally 'in our court'. I agree with the Knol Team that, it is up to the Knol Authors and their associations to set the review requirements and standards for acceptance and ultimate publication in their spheres. I believe that the tools already provided are sufficient for that purpose. However, the author should be given control over which reviews they accept and have the right to reject the inappropriate ones that Krishan has identified.

5 - As to the new sliding scale in the Review Scorecard - I think it is a great improvement! It allows rating shades rather than the blunt Yes/No.

6 - As to the new questions - They are all good but I would like to comment on a few.

(a) Did you find this knol useful? I would like to add “for you or could it be for others you know?” I will want to review many Knols that while not useful directly to me could be very useful for others that I know or teach.

(b) Is this knol as good or better than most other web pages on its topic? I would like to change web pages for “top 10 search returns” on its topic. I think it adds greater focus and suggests a course of action in completing the Review Scorecard.

(c) Does this knol promote a product or service? This one is a real shock inclusion and one that I think should be re-worded. What is implied here? – Is promoting a product good or bad? Everyone of us that choose to have Google’s Adsense to the side and bottom of our Knols – Are we not promoting a product or service? Is the Review Scorecard insinuating that this promotion of a product is a good thing or bad? I think that the question implies that maybe we shouldn’t be promoting any product, yet your content policy states “COMMERCIAL ACTIVITY: You may use Knol to … promote your lawful products or services”. I understand the sentiment, that we don’t want excessive and inappropriate ‘pushing’ of commercial products particularly the low user-benefit ones, but to simply ask the question without qualifications is, in my mind, both cruel and unfair. How about "Has the promotion of products or services been appropriately handled"

Due to poor visibility and low returns for the reviewer, I do not think it is worth the efforts. A way must be found to give a Knol Review more visibility with its own knol status, Page views and star rankings?

I have now created a new knol and provided the link in my review as before and completed the queries

Lessons from Biotechnology to Nanotechnology