Apr 28, 2015

Why I Don’t Assign Ratings

Not that anyone is asking, but there is a distinctly missing feature of all of my many reviews: some kind of rating. I never give anything 8.5 out of 10, 3 out of 5 stars, or whatever system you typically see. I don’t do ratings, man.

Reviews are far too subjective for quantitative number assignment. Plus, it’s arbitrary and ambiguous. What does 5 out of 10 mean? That it is perfectly average? Is this on a bell curve? On all of the sites that tend to slap on the star ratings or whatever you inevitably seem to get the same user behavior: lots of folks just put 1 (because they hated it) or the max (because they loved it and it’s OMG SO GOOD). There never seems to be much middle ground and that’s because the decision is too much work. To get a ranking system going they’d have to compare it to past titles to see where it stands and really think deeply. Believe me, if you are someone who cares about that kind of consistency it is a lot of work to do right. And even then it is still incredibly arbitrary. (Many years ago when I had a GeoCities site, I had a few anime reviews there and I even did full broken out ratings and was constantly revising them as I watched more shows. It was an exponentially difficult task.)

Instead you get systems like Crunchyroll where everything has about a 4.2 average rating on a 5 star scale. On CR, less than 4 means it’s probably not good, 4.1 to 4.4 means it is average, and anything 4.5 or above is either really good (Shin Sekai Yori, currently at 4.7) or possibly just really popular (Attack on Titan, currently at 4.8). That’s… really dumb. Why bother with the stars when they rarely waver by more than 1 star anyway? Sure, you can learn the “true” scale, as I just detailed, but it is a very misleading score until you do.

I’m afraid I’m going to have to play the xkcd card because, well, Randall said best already:

It was no hasty decision for YouTube to switch to simple like/dislike ratings. Hell, some of you may not even remember when they had star ratings! But they were in a great position: they had the massive sample data to analyze, the huge incentive to improve their rating system, and the money and time to really dive into the statistics to make an informed decision. And yeah, they went with simple like/dislike. While one must consider YT’s specific circumstances and needs, even so such a decision should not be taken lightly.

My reviews require reading and considering it yourself. They require knowing a bit of what you personally like. How well do you think you align with my tastes? I try to strike a balance between giving a primer on what the show is as well as my opinion of its quality and enjoyment. I’m very biased but I try to be clear when something is merely my preference. It’s the standard disclaimer: YMMV.

All that said I do rate things in my Hummingbird library. However these are personal ratings of enjoyment. They let me remember what things I liked watching and roughly by how much. It’s not meant to be an indication of quality, merit, or anything else anymore: just how well I enjoyed it. I don’t ever include these with my reviews because it would be misleading.

Just for your curiosity, if any, here’s my handy table for picking a personal score. I like to think this keeps it at least somewhat consistent, but again only within my own library of scores, which is all I really care about anyway.

Rating By Quality By Love By Gut Feel
0 Atrocious Hated it How?
1 Awful Really disliked Existence offends me
2 Poor Disliked Any redeeming value
3 Bad Mildly disliked Actively disliked it
4 Mediocre Ehh... seen better Boring/uninteresting
5 Okay Meh, neutral Has its moments
6 Decent Not too bad Liked but not rewatch
7 Good Mildly liked Mix of great/shit
8 Great Liked it Consistently good
9 Excellent Really liked Mix of great/good
10 Exceptional Loved it Consistently awesome

I also like this guy’s more methodical approach, though I don’t really use it myself.

No comments:

Post a Comment