Oct 10, 2016

How I Assign Ratings

If you’re that mythical regular reader of mine then you might notice the title of this post is a bit intentionally contradictorily worded. About a year and a half ago I wrote a piece on exactly the opposite stance. So am I flip-flopping harder than a presidential candidate in October?

If you actually read the linked post from before I even conclude it by mentioning exactly the opposite anyway, that I have a scale I use for personal ratings. It was more an admittedly meandering post on the nature of ratings and sort of how/why I don’t feature them in my review posts even though I actually do rate everything I watch.

I’ve been thinking a fair bit about ratings and 10-scales and the like lately, and trying to fine-tune my process. So I thought I’d share it here for the record.

Before I get to the nitty-gritty of my personal 10-scale chart, allow me to list my assumptions.

Ratings are subjective. Anyone who says or believes otherwise is lying to appeal to objectivity or deluding themselves. There simply is no way to be objective with this sort of stuff. There isn’t. Period.

My ratings are explicitly my own. I don’t expect your rating to agree with mine nor do I think either of us should be offended when they inevitably don’t. That’s okay. That’s perfectly normal. They are subjective, after all. It’s an opinion. We can discuss why each of us gives it the rating we do in order to come to an understanding of sorts but it’s not a contest nor is it a debate to see who is “correct”, as that would imply there is a “correct” rating, which there isn’t. We may or may not change one or both of us’ minds afterwards but that’s not really the goal.

My ratings are based primarily on enjoyment not merit. I want my ratings to help me remember and sort shows I’ve seen previously by what I enjoyed. I may try to strike a balance, as I’ve said before, in my reviews to be fair but biased as the reviews are meant more to give people a primer and opinion on a series and while I usually will include how much I liked or didn’t like a show I try to also point out the type of person who would like or dislike it more than me. I’m not rating on merit or worth or quality. A really shitty anime that is so bad it’s good will get a high rating if I enjoyed watching its badness. Likewise, a well made, clever, and beautiful show might get a similar rating. That only means I enjoyed them about the same, not that one is better than the other.

My ratings are for the most part mutually exclusive. I want my ratings to follow a flowchart, so to speak. While they are subjective, I want them to be mostly deterministic. It’s subjective, because if you apply the flowchart to a different person they can come out with a different response. However, the same person will get the same answer/rating every time. Below, in my table of ratings, you’ll see what I’m getting at here. The ratings don’t need to change if I see a new show. I don’t need to adjust my 5/5 I gave a previous show because I just saw one I think is even better. Did I thoroughly enjoy both of them or not? I can still think one is clearly better but I don’t particularly care that my ratings say so. That’s more for a review/analysis/discussion to unearth. I’m not trying to produce a bell curve here. Rating a show with my method should require not seeing any other shows.

My ratings exist at a point in time. It’s important not to forget that time is a factor here. While I said earlier that the system is mostly deterministic, that’s including when. I’m allowed to change my mind. Tastes change. What I liked as a young teen in anime I may not so much now. That’s okay. I’m not going to downgrade my rating of Love Hina just because I haven’t seen it in over a decade and while I know I used to be super into that series as a teen I probably wouldn’t be nearly so now. I would only ever re-rate it if I re-watched it.

My ratings are aware of the recency bias. Don’t forget that this is a real brain thing. We all do it. It’s okay. Be aware of it and that’s enough. Did you just finish watching something and you’re still riding the high of that finale episode? Ready to give it a 9 or 10 because ho-lee-shit dude? I get it. I’ve been there, bro. It’s important to just acknowledge that you’re in that state and move on. Should you wait to rate? Maybe a little bit. A day or so. Wait too long and you’ll forget things, if you’re like me. Or just try to take a moment and think a bit more rationally, if you think you can (know thy self). Fortunately, my scale kind of expects this and naturally adjusts for you. The way it’s worded is specifically to intercept this sort of thing. But only somewhat, so you still need to be aware.

Okay… that about wraps that up. You still with me? Good. Hopefully the above will better explain the how and why my 10-scale is the way it is. I’m still not 100% satisfied with it, either. I still think it needs work on the low end especially, because I rarely find myself giving those ratings out. But, then, I’m also a bit careful with what I watch. I’ve seen enough to steer clear, usually, of the bad or things I know won’t care for. So I’m not too concerned that my ratings tend to average around 7 or so. Keep in mind too that this is a 1-10 scale so there is no perfect middle neutral score (don’t let 5 fool you; it is mathematically slightly below average and 6 is slightly above).

  1. Existence offends me
    A special place for shows that by their nature I wish didn’t exist… they’re not even bad enough to enjoy because they’re actually bad and either make me feel bad or mad or both or ugh…
    Examples: Hanaukyou Maid-tai (so far the only one I’ve given a 1 to)
  2. Any redeeming value at all
    While I may loathe it, I acknowledge there’s probably someone out there that likes this sort of thing and at least it doesn’t offend me on a conceptual level. Usually very poorly made, trashy subject matter, or just awful.
    Examples: Joshikousei (trashy), To Heart 2 (awful)
  3. Actively disliked it   
    Clearly not my cup of tea. A chore to watch. Probably a style or genre I just don’t much care for, or it’s just kind of a shitty show.
    Examples: Lime-iro Senkitan (oldie), Diabolik Lovers (newie)
  4. Boring/uninteresting   
    Overwhelmingly meh. It’s probably poorly executed or not my kind of show but it’s not a complete train wreck.
    Examples: Ojisan to Marshmallow (3-min), Da Capo (full length), Bungaku Shoujo (movie)
  5. Has its moments but otherwise forgettable   
    Barely okay or merely adequate. Usually given to 3-min-type series that are too short to do anything useful or shows that just do absolutely nothing interesting or new and while they may not be necessarily bad they’re just so bland and forgettable I regret bothering in the first place.
    Examples: Pan de Peace! (3-min), Anne Happy (full length)
  6. Liked but it’s full of problems   
    While I overall don’t regret watching, it probably had a lot of things that irked me, be they production issues, style, or general plot laziness, etc… this is a very frequent score as a lot of shows tend to ride this line of not wanting to rock the boat too much or be cheaply produced or have poor talent at the helm.
    Examples: Ai Yori Aoshi (oldie), Rokujouma no Shinryakusha!? (newie)
  7. Mostly good w/ minor issues   
    Definitely enjoyed but I can easily articulate problems I had with it and likely these are where scores will diverge as they are things people will have varying tolerances for in their own score. Probably my most frequent score, as a result.
    Examples: Kumo no You ni Kaze no You ni (safe), Elfen Lied (controversial)
  8. Consistently good   
    These are shows that were just always good. It never made too many glaring mistakes but it also never really reached any highs. Solid throughout and I thoroughly enjoyed it and could easily re-watch it in a few years with a friend who hasn’t seen it yet.
    Examples: Tenshi na Konamaiki (oldie), Kiznaiver (newie)
  9. Mix of great/good   
    Much like an 8 but this time it does occasionally scrape the ceiling of greatness. There’s a couple episodes or a handful of scenes or what have you that got me really pumped emotionally. These shows, despite being rated highly, might fall victim to being downgraded significantly on a second watch if those highs fail to resonate again without the surprise factor. The ones that stand the test of time are usually because they greatly appeal to me and/or are fantastic shows with a lot to appreciate beyond emotional highs.
    Examples: No Game No Life (probably hollow), Cowboy Bebop (easily stands the test of time)
  10. Consistently awesome   
    Nearly every episode was fantastic and left me wanting more. Reserved for shows that I adored and could re-watch many times, even alone (as usual). This is not a perfect rating for perfect shows. Often I can still discuss issues with it but for me the appeal was too strong. My tenouttatens go either to shows that are definitely flawed but pushed all of my specific buttons just right or are actually probably just ridiculously good ones that likely come with some degree of popular consensus.
    Examples: Kami nomi zo Shiru Sekai (muh buttons!), Akira (popular classic)

Now then. Interested in what I’ve rated stuff? My Kitsu profile is always the most up-to-date, usually to the minute. I sync things about every month or so back to my MAL profile because, well… let’s be honest: MAL has way more users and the data there is way more interesting to analyze as a result. It even has handy things like a compare feature (look under the Affinity to You section of someone’s profile if you’re also logged in) to see side-by-side how you and another user rated stuff, if you’re into that sort of thing. Just keep in mind the above… ratings are subjective so comparing them is often apples to oranges. That’s why average scores have only a very specific usefulness.

No comments:

Post a Comment