Anyway, the main issue I face is the problematic five-point scale. I don't like Likerts and their ilk, despite whatever the statistical literature may say, because I enjoy some discrimination in shades of meaning. I much prefer a ten-point scale for that very reason. Any rating system is open to bias, of course, as Respondent A may view the gaps among the choices widely whereas Respondent B may interpret them narrowly. Respondent C, alternatively, may not see the gaps as equidistant. I'm sure these all "average out," blah, blah, blah, in large samples. As my primary interest in rating scales is not comparison with others' opinions but of categorizing and justifying my own to myself, though, these issues aren't as pressing to me.
So use a ten-point scale, dude. Why argue with yourself about this? Well, check out the main rating sites, and you'll find this divergence of opinion on opinions play out marvelously. Some sites and programs, like iTunes, Goodreads, and TripAdvisor, allow users only a five-star or five-circle rating scale, but then average out all users' ratings to the second decimal point (Goodreads) or truncated stars or circles (iTunes and TripAdvisor). These methods effectively transform one-to-five scales into one-to-one-hundred or undefined scales. Not to mention that one hundred is far too much refinement for me; it reminds me all too well of grading history essays. (Shiver.) I therefore much prefer the rating systems on sites like IMDb, Metacritic, and Worlds Without End, which use ten-point scales (or, in the case of WWE, five stars with half-star increments). Others, like Moby Games, play with the system entirely and break down ratings into themes, such as "Gameplay" or "Graphics," and apply one-to-five scales to each theme. I assume they average these all up and grant them equal weight, a method with which I also take issue. Who determined arbitrarily that "Personal Slant" has equal weight to "Story/Presentation"? What about a game that intentionally lacks "Sound/Music"?
As a result, I want to spell out explicitly my ten-point scale system and explain how it'll interface with these various rivals to my rating-dom:
1. Avoid
2. Terrible
3. Bad
4. Uninteresting
5. OK
6. Decent
7. Great
8. Excellent
9. An Achievement
10. Favorite
 |
| My rating category inspiration, courtesy of http://iaretheshawna.blogspot.com/2012/10/review-upside-down-on-road-day.html |
1. Avoid (Battlefield Earth)
2. Terrible (Attack of the Killer Tomatoes!)
3. Bad (The Happening)
4. Uninteresting (Scooby-Doo)
5. OK (The Princess Bride)
6. Decent (The Departed)
7. Great (UHF)
8. Excellent (The Shining)
9. Cinematic Achievement (The Godfather)
10. Favorite (The Shawshank Redemption)
I consider this scale to be pretty evenly split and accommodating to a good level of shades of meaning. The bottom half, one through five, elicit negative to apathetic reactions, whereas the top half, six through ten, reflect positive ones. As I consider apathy to be bad, I lumped it into four and five, which may seem like the scale favors positive reactions over negative ones. That may be true, but I cite precedent: Rotten Tomatoes, which claims as "fresh" a cultural text with overall ratings at 60% or higher and "rotten" those at 59% or lower. I always liked how Rotten Tomatoes's scheme lined up with a lot of university grading scales, too, with passing and failing grades divided similarly.
It's also likely that the cultural texts I rate will, by selection bias, fall on the positive end of the scale more than the negative. Having a little distortion to the high end of the scale doesn't bother me a whole lot. It also helps smooth over some problematic comparisons drawn from my ratings. For instance, I often say that a given film is "great" when talking about it to friends, but when I compare it to other films, it could fall into an entirely different echelon.
Take UHF versus The Godfather. I love UHF. I'll go so far as to say it's a "great" film. It's campy as hell, and didn't win many glances (let alone awards) from the upper-crusty establishment for categories like sound editing. But I think it merits a 7 (Great). It was far more entertaining than The Departed, and framed my perspective in a far more enduring way. (Who doesn't love Spatula City?!) The Godfather, in comparison, is pretty universally regarded (myself included) as one of the greatest films of all time, a genuine human treasure. It's not one of my Favorites, which I reserve for films that both exemplify good filmmaking and change my life, but it's a cinematic achievement bar few. To distinguish it from UHF, a 9 (A Cinematic Achievement) will do. But if you have a five-point scale, how do you compare these two? UHF at 4 and The Godfather at 5? How can I distinguish The Shawshank Redemption, which I think blew both out of the water? Or The Shining, which to me is obviously better than UHF but not on par with The Godfather?
Furthermore, my threshold for "Favorites" is pretty high. I only put five films in that category and want them to stand out above the rest: Contact, Cloud Atlas, The Shawshank Redemption, Kinsey, and Shaun of the Dead. As motley as this group is (and in stark contrast with others' lists, I'm sure!), they all represent films that, to me, are sublime, multilayered works of art and remain with me every day. (I'm tempted to add American Beauty, which falls between 9 (A Cinematic Achievement) and 10 (Favorites), but let me not go down that road to a one-hundred-point scale.) Given that The Godfather, The Shining, and American Beauty all make my list of films that everyone should see before they die, though, consider Ranks 8 (Excellent), 9 (A Cinematic Achievement), and 10 (Favorites) to be apart from the rest. You can view others in this highest of film strata on my Best Films of All Time list on IMDb, which needless to say is an evolving group.
I didn't even mention trans-media comparisons. How does it look when I effectively put David Mitchell's Cloud Atlas at a 9 (A Literary Achievement, rated 4.5 on Worlds Without End) but the Wachowski's Cloud Atlas at a 10 (Favorites)? I even said in my Cloud Atlas (the book) review that I wouldn't compare the two, as books and films are incommensurable. Indeed, I still think they are. So deal a bit with that dissonance, I ask humbly.
Because films are my favorite form of cultural text, I started with them and fleshed out the scale most easily. The book scale would be a bit more difficult to break down at the moment. Nevertheless, for the rating and review sites I'm using, here is how my ten-point scale will break down:
Board Game Geek
Easy. BGG employs a ten-point scale, so mine fits well with theirs. I like how they provide recommended (if sloppily written?) explanations for each point, which largely align with mine:
1. "Defies description of a game. You won't catch me dead playing this. Clearly broken."
2. "Extremely annoying game, won't play this ever again."
3. "Likely won't play this again although could be convinced. Bad."
4. "Not so good, it doesn't get me bet could be talked into it on occasion."
5. "Average game, slightly boring, take it or leave it."
6. "Ok game, some fun or challenge at least, will play sporadically if in the right mood."
7. "Good game, usually willing to play."
8. "Very good game. I like to play. Probably I'll suggest it and will never turn down a game."
9. "Excellent game. Always want to play it."
10. "Outstanding. Always want to play and expect this will never change."
Some thoughts: I hate the word "average." It's so meaningless. Do games fall on a Bell curve? I don't think so. They tend to distort to the positive end, or else we'd never have either a) invented them or b) passed them along enough for others to render judgment so. And if you're a game shark like my partner, Leigh, though, you may put any game in the 8 or 9 category!
Goodreads
Slightly trickier. Goodreads locks reviewers into a five-star scale. For simplicity, I plan (although this may change as I rate more books) to round down when I feel a book lands between two star ranks. Books I rate as A Literary Achievement (9 out of 10) will round down to 4 out of 5, and so on. I don't really care for this system, for all the reasons that I stated above about the problems with five-point scales. But rounding down preserves Rank 5 for my Favorites, and I want to assign them special status. This system also keeps Ranks 8 and 9 together, meaning that any books to which I assign 4 or 5 stars on Goodreads will constitute my Best Books of All Time list, analogous to Ranks 8, 9, and 10 on IMDb. Furthermore, I already follow this practice in my iTunes ratings, too, given that iTunes also has a five-star system. At least for imaginative fiction, I can discriminate more accurately with Worlds Without End's scale (see more below). The only snafu comes with 1 (Avoid), or a .5 out of 5 stars on Worlds Without End. Given that, on Goodreads, zero stars only really means that a reviewer has yet to assign a rating, as opposed to a genuine rating of zero, I'll just round up.
IMDb
Easy. One to ten stars. One to ten rating system. See main chart above.
Metacritic
Ditto to IMDb.
Moby Games
This one is the most difficult of all, given the mandatory breakdown of ratings by theme. The cumulative results may not line up perfectly with a five-point scale, though, so that may accommodate some diversity. I've only rated one game so far--Assassin's Creed IV: Black Flag--which resulted in a 3 out of 5. Because that effectively pans out to 6 out of 10, "Decent" in my book, I'm comfortable with Moby's system so far. Time will tell if that changes.
TripAdvisor
Very similar to Goodreads, but as I'm not using a similar site with a ten-point scale, I'll just have to deal. But I'll justify any differences discursively in my Arcturus Stream review, claro.
Worlds Without End
Easy. One to five stars, with half-star increments, and therefore effectively a ten-point scale. This will allow me to differentiate Favorites (1984) from Literary Achievements (Cloud Atlas), at least in regard to imaginative fiction.
Needless to say, these scales are subject to refinement and adjustment over time. If I make any new changes, I'll state so right here.
