Post Reply Improving the reviewing of Crunchyroll
5086 cr points
Send Message: Send PM GB Post
20 / M
Offline
Posted 1/13/15 , edited 1/13/15
Here's my suggestion:

Crunchyroll as a site tracks your anime watching for an anime series. It also only allows members of the site to post reviews for a series. Why not have one work for the other and make a member have to watch a certain number of episodes of the series before they can post a review. That would keep the number of over-biased reviews down and let most of the reviews give honest interpretations of what the series offers. Biggest example I can give is for the series Madoka Magica. If you have seen the series, you will know that the series you watch in episode 1 is VASTLY different by episode 12. If you base a review off the first 2 or 3 episodes, the universe is just BARELY being scratched whereas someone who has finished the series will be able to give a much more honest opinion of the series overall with the knowledge of the themes that are presented.

What do you all think? Would you support this?
8193 cr points
Send Message: Send PM GB Post
32 / M / Netherlands
Offline
Posted 1/13/15
I actually totally agree with this idea. Think it would be great. But i also think it would make it to strict if i'd look at it objectively. What they could do however. And what would fall a bit under the same idea. Is do what steam does for game reviews. On steam when someone write about a game you can see how much time they invested into it. They could do that here as well. That if you write a review that you can see how many episodes you've actually watched. And then judge for yourself how serious you take the review you are reading.

But to be 100% honest. Both versions of the idea can be faked very easy. Just push play on episode 1. Mute the sounds and shut of your screen and sleep a night. Wake up and you watched about 20 episodes (if a season is long enough). Downside of autoplay i guess.
3171 cr points
Send Message: Send PM GB Post
23 / M / GA
Offline
Posted 1/13/15 , edited 1/13/15


I think it's more important to figure out a better way to make sure the more helpful reviews are more visible. To do that, there should be more incentive to rate whether or not a review was helpful. Just doing that, I think, would make a huge difference in the quality of reviews.

Also, sometimes I forget to log in. So, if that were the case, I would have to pretend to watch it all over again just to be able to review it! How would I do that? Just leave it running while I go to class? Nu uh! You've gotta get down to the root of the problem instead! Preventing people from reviewing it's just putting a tarp over a turd. Incentivize better reviews and encourage more rating of those reviews.

Oh! And definitely increase the character requirement!
8193 cr points
Send Message: Send PM GB Post
32 / M / Netherlands
Offline
Posted 1/13/15
You have a good point there. Though with incentives you either need people who will review the reviews to decide what is "good enough" to get an incentive. Or give it to anyone who writes a review but in that case i'm 99% sure you'll get so much junk reviews just for the incentive that you might as well close the whole section down. Think it's really hard to create something that will make it fair in use and also rewards those that write actual good helpful reviews and moves the bad reviews down so everyone can read the good reviews first. Hell i don't even know how to properly write it out lmao :).
3171 cr points
Send Message: Send PM GB Post
23 / M / GA
Offline
Posted 1/13/15 , edited 1/13/15
There are many, many things you could do to attempt to assure quality. Barriers to entry, force instance, have served to be effective in many respects! You could allow people to earn themselves a "trusted reviewer" tag which would simply give the reviewer higher visibility over others or, at the very least, higher credibility. If you think about it, this is actually already how most review systems work except that anyone could eventually earn the tag if, say, they receive a certain level of positive feedback on their work. On the same token, they could also lose it if they receive a certain level of negative feedback. This ensures no one would abuse their badge, so to speak and ensures quality.

Though, really, it all boils down to people's willingness to hit a little button that says "Was this review helpful: Yes / No." This is the inherit quality control mechanism we have here, however, it's really not used nearly as often as it should. If you want quality to rise to the top, people have to submit feedback. You can't just assume others will do it for you. This is really the keystone of the problem. If you do more to encourage and incentivize feedback, the rest sort of just falls into place.

How to accomplish that, well, that's where we gotta use our brains.

Posted 1/13/15

Blaqlite wrote:

There are many, many things you could do to attempt to assure quality. Barriers to entry, force instance, have served to be effective in many respects! You could allow people to earn themselves a "trusted reviewer" tag which would simply give the reviewer higher visibility over others or, at the very least, higher credibility. If you think about it, this is actually already how most review systems work except that anyone could eventually earn the tag if, say, they receive a certain level of positive feedback on their work. On the same token, they could also lose it if they receive a certain level of negative feedback. This ensures no one would abuse their badge, so to speak and ensures quality.

Though, really, it all boils down to people's willingness to hit a little button that says "Was this review helpful: Yes / No." This is the inherit quality control mechanism we have here, however, it's really not used nearly as often as it should. If you want quality to rise to the top, people have to submit feedback. You can't just assume others will do it for you. This is really the keystone of the problem. If you do more to encourage and incentivize feedback, the rest sort of just falls into place.

How to accomplish that, well, that's where we gotta use our brains.



people don't give thumbs ups if you give low rating to an anime whether it's credible or not. they only give thumbs up if you give the anime high ratings.
42864 cr points
Send Message: Send PM GB Post
47 / M / Memphis, TN
Offline
Posted 1/13/15
All of these suggestions so far have merit. I would point out that an honest reviewer, having reviewed a series before completing it (which I have occasionally--although rarely--done), would adjust that review accordingly. You can edit your CR reviews, after all.
3171 cr points
Send Message: Send PM GB Post
23 / M / GA
Offline
Posted 1/14/15 , edited 1/14/15

GayAsianBoy wrote:


people don't give thumbs ups if you give low rating to an anime whether it's credible or not. they only give thumbs up if you give the anime high ratings.




Hmmm, that does have some merit. The fan boys are strong. However, how many of those negative reviews are actually very credible? If a show is bad, you'll be able to tell from the reviews because it will almost universally be panned. If a show is bad, you're not going to see an overwhelming amount of positive reviews. But this does happen! If a show is... just alright, maybe a 3-4 star show, there's always going to be people who LOVE it and think it's the greatest thing in the world. Naturally, those reviews will be positively rated by people who also think it's the greatest thing in the world and the people who don't, well, they probably aren't even reading the reviews.

A potential solution to this is to make the review less of a "look at my factual opinion" and more of a discussion. Perhaps allowing comments? My only concern there would be the probably high chance of people getting flamed which would really just discourage people from reviewing at all and that's the opposite of what we want.

However, naturally, this is just human error. On the internet, we're constantly surrounding ourselves with things and other people that re-affirm our own beliefs. Even if a negative review is credible, you're right, if there are people who love it, that review is going to quickly find itself at the bottom of the stack. This is why many sites pay their reviewers so that they have people that, otherwise wouldn't really give two toots about the show and are immune to biases of fanboyism, still voicing their opinion.

Unfortunately, this is not really something I see happening on CR. However, this does lend good credence to my barrier to entry idea where we do have people that consider themselves "designated reviewers" that might even review a show that they otherwise wouldn't for the sole reason that their opinion is being seen and valued. That's a sort of payment.
41671 cr points
Send Message: Send PM GB Post
26
Offline
Posted 1/14/15 , edited 1/14/15
This is a good idea. However I see a flaw with it. Let's say you had to watch 6 episodes to write a review.

Seems like a good idea. However it doesn't matter what anime it is excellent or poor some people will like it.

People who watch 6 episodes of a truely bad anime will more then likely like the anime given they have dedicated that much time to the anime. I can't help but feel this would lead to a positive bias in reviews.

If you watch 4 episodes of a anime and truely dislike it I feel you should be able to voice your opinion on the anime.

However it doesn't be 6 episodes to review simply using that as a reference point as it gives you a good gauge on most anime by that time.

Its up to the others who have watched/consider watching the anime to choose whether the review is creditable or not. The page also lists total reviews (1-5 stars) so it should be noticeable right off the bat if a anime has 30 1 stars and 50 5 stars the series will be a hit or miss.
You must be logged in to post.