[PC] Normalize Player Ratings (Average = 0)

by **Woodruff** on Sun Sep 05, 2010 5:51 pm

temporos wrote:Concise description:
Migrate from a "1 to 5" star system to a "-2 to +2" star system.

Specifics/Details:
Right now: 1 is bad, 5 is awesome.
Migrate to: -2 is bad, +2 is awesome.
Migration should be retroactive.
Lack of a rating should be counted as a 0 or "average" rating.

How this will benefit the site and/or other comments:
Right now, the average player rating is about 4.8 stars: well above the 3 star "average."
If the scale is adjusted such that "average" is 0, anything below average is negative, and anything above average is positive, the average player rating will naturally stabilize around 0.
If a player does not wish to leave a rating for his opponents after a match, it counts as a 0 or "average" rating.
This proposed system will encourage a more intuitive and reliable measure of a player's attitude, gameplay, and sociability.
Players would be compelled to leave a rating only if they wish to leave an above or below average rating for another player.
Players aren't left jaded when someone leaves them a less-than-5-star rating (i.e., everybody wins).

What we really should migrate to is a pass/fail system. Up or down. Air Force officer recommendations used to be based on what was equivalent to the "5-star" system, and it failed precisely as this one has - with rating inflation. But some time ago, the Air Force moved to the up/down or pass/fail system and it is far more fair than otherwise, and you don't see the inflation you did previously. It was far less subjective.

by **stahrgazer** on Sun Sep 05, 2010 8:16 pm

MeDeFe wrote:What you're pointing out is exactly what the proposed system would counteract.
In other words: What you're pointing out IS a problem for the current system, it would NOT be a noticeable problem for the proposed system.

Yes it would still be a problem; players would still rate their "less than zero" friends as +2's (instead of +5's) and people they decide not to like as -2's (instead of 1 to 3) regardless of game skill or any problem in a particular game.

by **Metsfanmax** on Sun Sep 05, 2010 8:48 pm

stahrgazer wrote:
MeDeFe wrote:What you're pointing out is exactly what the proposed system would counteract.
In other words: What you're pointing out IS a problem for the current system, it would NOT be a noticeable problem for the proposed system.

Yes it would still be a problem; players would still rate their "less than zero" friends as +2's (instead of +5's) and people they decide not to like as -2's (instead of 1 to 3) regardless of game skill or any problem in a particular game.

It's trivial to point out that ratings are subjective. It has nothing to do with the value of this suggestion.

by **JoshyBoy** on Mon Sep 06, 2010 6:14 am

I fail to see the point of this suggestion, it seems like it's just people being pedantic.

by **Joodoo** on Mon Sep 06, 2010 8:07 am

If we're asking for something like +1/-1 to a person's rating then we're just going back and forth between the original feedback system and current ratings system, because the feedback format was (# of positive feedback - # of negative feedback).
A slight tweak to how this should work may be better.

by **Metsfanmax** on Mon Sep 06, 2010 8:24 am

JoshyBoy wrote:I fail to see the point of this suggestion, it seems like it's just people being pedantic.

You fail to see that the current rating system is completely meaningless, given that something like 90% of the average ratings fall between 4.5 and 5.0?

by **MeDeFe** on Mon Sep 06, 2010 8:25 am

JoshyBoy wrote:I fail to see the point of this suggestion, it seems like it's just people being pedantic.

The current system is not working because the ratings are hugely inflated. Implementing this suggestion would go a very long way towards solving that.

That's the point.

by **JoshyBoy** on Mon Sep 06, 2010 8:29 am

Metsfanmax wrote:
JoshyBoy wrote:I fail to see the point of this suggestion, it seems like it's just people being pedantic.

You fail to see that the current rating system is completely meaningless, given that something like 90% of the average ratings fall between 4.5 and 5.0?

No, I see that. I could have told you that. What I fail to see, is how the proposed changes fix it.

The old feedback system was great (and I really miss it

), apart from the one problem of moderating it.

by **JoshyBoy** on Mon Sep 06, 2010 8:30 am

MeDeFe wrote:
JoshyBoy wrote:I fail to see the point of this suggestion, it seems like it's just people being pedantic.

The current system is not working because the ratings are hugely inflated. Implementing this suggestion would go a very long way towards solving that.

That's the point.

Which specific suggestion are we talking about here?

by **pmchugh** on Mon Sep 06, 2010 9:03 am

The idea in the OP helps to fix two current problems with the ratings system but has 1 major flaw.

1. New players find it hard to get a decent rating as one bad rating drags their score down. Their score will take less of a hit from initial bad ratings as they are getting far more ratings at average in the beggining.

2. Players are rated too highly, this point has already been argued.

The problem is that people simply do no rate enough to make it viable. Think of someone like magleplunka or highlander attack, with thousands of games and such a low percentage of people rating it is almost certain their rating would be like 0 or +/-0.1

by **temporos** on Mon Sep 06, 2010 10:22 am

pmchugh wrote:Think of someone like magleplunka or highlander attack, with thousands of games and such a low percentage of people rating it is almost certain their rating would be like 0 or +/-0.1

Just because they have thousands of games under their belts doesn't mean they're above or below average. The whole point of the new system is that you deviate from 0 (or whatever you want "average" to be) only when you leave a significantly positive or negative impression on people. If you don't leave this impression for a significant number of games, then you really are average.

by **MeDeFe** on Mon Sep 06, 2010 10:48 am

JoshyBoy wrote:
MeDeFe wrote:
JoshyBoy wrote:I fail to see the point of this suggestion, it seems like it's just people being pedantic.

The current system is not working because the ratings are hugely inflated. Implementing this suggestion would go a very long way towards solving that.

That's the point.

Which specific suggestion are we talking about here?

The part of it where a player not bothering to actively rate someone would count as "average". Whether that be 0 on a scale from -2 to +2, or 3 stars on a scale from 1 to 5.

by **stahrgazer** on Mon Sep 06, 2010 10:48 am

Metsfanmax wrote:
stahrgazer wrote:
MeDeFe wrote:What you're pointing out is exactly what the proposed system would counteract.
In other words: What you're pointing out IS a problem for the current system, it would NOT be a noticeable problem for the proposed system.

Yes it would still be a problem; players would still rate their "less than zero" friends as +2's (instead of +5's) and people they decide not to like as -2's (instead of 1 to 3) regardless of game skill or any problem in a particular game.

It's trivial to point out that ratings are subjective. It has nothing to do with the value of this suggestion.

Your lack of logic astounds me. What's wrong with the current system - as is indicated even in one of the OPs posts, is that the ratings are "inflated" - and they only get inflated because people are using them subjectively. This new system doesn't fix that, and can't. So, if the ratings will be inflated due to subjectivity anyway, why make the change?

I can see the value of making "no ratings" equivalent to a 3, "average" to tone down the levels of inflation, but you'll still see the subjectivity-caused inflation.

by **MeDeFe** on Mon Sep 06, 2010 11:09 am

stahrgazer wrote:
Metsfanmax wrote:
stahrgazer wrote:
MeDeFe wrote:What you're pointing out is exactly what the proposed system would counteract.
In other words: What you're pointing out IS a problem for the current system, it would NOT be a noticeable problem for the proposed system.

Yes it would still be a problem; players would still rate their "less than zero" friends as +2's (instead of +5's) and people they decide not to like as -2's (instead of 1 to 3) regardless of game skill or any problem in a particular game.

It's trivial to point out that ratings are subjective. It has nothing to do with the value of this suggestion.

Your lack of logic astounds me. What's wrong with the current system - as is indicated even in one of the OPs posts, is that the ratings are "inflated" - and they only get inflated because people are using them subjectively. This new system doesn't fix that, and can't. So, if the ratings will be inflated due to subjectivity anyway, why make the change?

I can see the value of making "no ratings" equivalent to a 3, "average" to tone down the levels of inflation, but you'll still see the subjectivity-caused inflation.

And your logic is nonexistent.
There are about 20000 players on the site. People rating their friends higher than they might deserve amount to maybe 2% of the inflation. The remaining 98% are due to people not bothering/wanting/daring to rate those who don't leave an impression or who leave a bad impression. Reasons for people not rating have also been pointed out already. Compared to the big problem of people only leaving good ratings, the problem of people rating their friends better than they deserve is not noteworthy.

Because inflation cannot be ruled out 100% but only mostly ruled out you say that nothing should be done at all? You must be kidding.

by **stahrgazer** on Mon Sep 06, 2010 11:40 am

MeDeFe wrote:And your logic is nonexistent.
There are about 20000 players on the site. People rating their friends higher than they might deserve amount to maybe 2% of the inflation. The remaining 98% are due to people not bothering/wanting/daring to rate those who don't leave an impression or who leave a bad impression. Reasons for people not rating have also been pointed out already. Compared to the big problem of people only leaving good ratings, the problem of people rating their friends better than they deserve is not noteworthy.

Because inflation cannot be ruled out 100% but only mostly ruled out you say that nothing should be done at all? You must be kidding.

Currently "not rated" means zero added in, all across the board. That means the skewed 2% ratings appear high, like 4.8.

In your proposed change, "not rated" will mean zero added in, all across the board. The same 2% ratings will still appear. While a 1.25 won't look as high as a 4.8, the percentage of deviation will remain the same. you'll have very few zeros (we have very few 3's now), some -1.25's and some +1.25's, almost no 5's (we currently have almost no 5's) .

Do you understand the words that are coming out of my mouth? And through the keys? It will only LOOK like you've made a change, because the real problem isn't what the numbers look like, the real problem is that only 2% of the players will rate consistently, and those will STILL be either their friends or their foes.

Logically speaking, it won't make a bit of difference, even if it has the APPEARANCE of making a difference. You must work for some government somewhere, because only politicians and government workers count on "appearances" over actual results... and only politicians and government workers would think it's logical to do so.

by **MichelSableheart** on Mon Sep 06, 2010 4:03 pm

Actually Stahrgazer, I believe you may be underestimating the mentality shift that would result from making a non-rating count as average. There are currently roughly two groups of people giving out max scores. Those who rate their friends high, and those who rate everyone high all the time because it isn't nice to give out a 'bad' rating.

If most of the ratings you receive are average (which is the result of nonraters giving out average ratings), this second group has far less of an incentive to give out all max ratings, simply because they will see that an average rating is not bad. The result is less max ratings giving out without thinking, meaning a reduction of ratings inflation.

by **natty dread** on Mon Sep 06, 2010 9:33 pm

Either way: either this suggestion works, and the ratings scale will "correct" itself, or it doesn't and... where's the big harm or downside to this suggestion?

So it's worth a shot, eh?

by **Metsfanmax** on Mon Sep 06, 2010 11:23 pm

stahrgazer wrote:Currently "not rated" means zero added in, all across the board. That means the skewed 2% ratings appear high, like 4.8.

In your proposed change, "not rated" will mean zero added in, all across the board. The same 2% ratings will still appear. While a 1.25 won't look as high as a 4.8, the percentage of deviation will remain the same. you'll have very few zeros (we have very few 3's now), some -1.25's and some +1.25's, almost no 5's (we currently have almost no 5's) .

No, you obviously don't understand the suggestion as originally stated. Every time you don't actively leave a rating, in the suggested system, a zero rating is automatically left for you (as opposed to the current system, where no rating is given for you - I hope you understand that when someone leaves no rating, currently, it doesn't mean that a zero is added and the score is re-averaged; rather, it means that no score is added in). The 2% of ratings in question will have little effect.

Since people probably rate their opponents less than 50% of the time, this would be incredibly different from the current system, because the average rating would be much closer to 0 (the intended average score) than the average rating now is, in relation to a score of 3.

by **stahrgazer** on Sat Sep 11, 2010 8:29 pm

Metsfanmax wrote:
stahrgazer wrote:Currently "not rated" means zero added in, all across the board. That means the skewed 2% ratings appear high, like 4.8.

In your proposed change, "not rated" will mean zero added in, all across the board. The same 2% ratings will still appear. While a 1.25 won't look as high as a 4.8, the percentage of deviation will remain the same. you'll have very few zeros (we have very few 3's now), some -1.25's and some +1.25's, almost no 5's (we currently have almost no 5's) .

No, you obviously don't understand the suggestion as originally stated. Every time you don't actively leave a rating, in the suggested system, a zero rating is automatically left for you (as opposed to the current system, where no rating is given for you - I hope you understand that when someone leaves no rating, currently, it doesn't mean that a zero is added and the score is re-averaged; rather, it means that no score is added in). The 2% of ratings in question will have little effect.

Since people probably rate their opponents less than 50% of the time, this would be incredibly different from the current system, because the average rating would be much closer to 0 (the intended average score) than the average rating now is, in relation to a score of 3.

Obviously, you don't understand deviation percentages. It will only appear to be significantly different because you changed how the number values look. All that will happen with your system is that the decimal differences will have increased importance because you'll be averaging in a few thousand zeros. But nothing really changes unless people's tendency to rate appropriately changes - and the reason they don't is because not everyone agrees on "what is appropriate?"

by **Metsfanmax** on Sat Sep 11, 2010 8:46 pm

stahrgazer wrote:
Metsfanmax wrote:
stahrgazer wrote:Currently "not rated" means zero added in, all across the board. That means the skewed 2% ratings appear high, like 4.8.

In your proposed change, "not rated" will mean zero added in, all across the board. The same 2% ratings will still appear. While a 1.25 won't look as high as a 4.8, the percentage of deviation will remain the same. you'll have very few zeros (we have very few 3's now), some -1.25's and some +1.25's, almost no 5's (we currently have almost no 5's) .

No, you obviously don't understand the suggestion as originally stated. Every time you don't actively leave a rating, in the suggested system, a zero rating is automatically left for you (as opposed to the current system, where no rating is given for you - I hope you understand that when someone leaves no rating, currently, it doesn't mean that a zero is added and the score is re-averaged; rather, it means that no score is added in). The 2% of ratings in question will have little effect.

Since people probably rate their opponents less than 50% of the time, this would be incredibly different from the current system, because the average rating would be much closer to 0 (the intended average score) than the average rating now is, in relation to a score of 3.

Obviously, you don't understand deviation percentages. It will only appear to be significantly different because you changed how the number values look. All that will happen with your system is that the decimal differences will have increased importance because you'll be averaging in a few thousand zeros. But nothing really changes unless people's tendency to rate appropriately changes - and the reason they don't is because not everyone agrees on "what is appropriate?"

No, it is a substantial change. Let's say I have played 1000 players and 25% of them rated me. 80% gave me 5 stars for Gameplay, 10% gave me 4 stars, and 10% gave me 1 star. Under the current system, my rating would be 4.5 out of 5. Under the new system (where 80% of the raters give me 2, 10% give me 1, and 10% and give me -2), my rating would be +0.375. You don't think those are different?

by **temporos** on Sun Sep 12, 2010 10:42 am

stahrgazer wrote:Obviously, you don't understand deviation percentages. It will only appear to be significantly different because you changed how the number values look. All that will happen with your system is that the decimal differences will have increased importance because you'll be averaging in a few thousand zeros. But nothing really changes unless people's tendency to rate appropriately changes - and the reason they don't is because not everyone agrees on "what is appropriate?"

How is this a bad thing? The decimal differences are supposed to be very important. Almost everyone would be within .1 or .2 of "average," which is what is supposed to happen. In the current system, there is far more likely to be a scenario in which the ratings become polarized (either very high or very low). This is not statistically accurate, and it destroys the whole point of rating someone. Under the proposed system, the CC population would be represented by a "bell curve," with the majority of players within one or two standard deviations from the "average." There would be very, very few with extremely high or low ratings. That's how statistics works... unless you're purposefully trying to skew the data... :-s

by **Darwins_Bane** on Mon Sep 13, 2010 12:23 am

i like this idea. it should get submitted.

by **TheForgivenOne** on Mon Sep 13, 2010 3:07 am

I'm gonna sticky this.

by **Victor Sullivan** on Mon Sep 13, 2010 5:45 am

Darwins_Bane wrote:i like this idea. it should get submitted.

Agreed.

by **Woltato** on Tue Sep 14, 2010 8:04 pm

Bit of a radical suggestion but why not do away with current ratings system altogether and have ratings based on the number of friends and foes you've got. IE the more players who've foed you the lower your rating, the more who've friended you the higher your rating.

Not sure exactly how to calculate it but think a system based on this would give a much more accurate indication of what people are like to play with and we wouldn't have to bother with the hassle of leaving ratings after every game.