Without going into technical details, we use a modification of the Elo rating system used in chess. Everything is given a rating value, and an uncertainty value. The amount that your rating changes depends on the relative rating value, and the uncertainty values.

When the uncertainty value is low enough, the rating is displayed. There is no hard and fast rule about the number of attempts / correct attempts. e.g. if 20 level 1's fail to solve a level 5 problem, we would not have much certainty about the rating. Same applies if 20 level 5's solve a level 1 problem.

As a significant proportion of submitted problems have issues with them, we have decided to take a conservative approach to ratings. As the quality of submissions improve (say due to having reputable problem creators, active reporting and responding, or member moderation), we can afford to be more lax. Until them, I rather err on the side of caution.
–
Calvin Lin
Staff
·
3 years ago

Log in to reply

@Calvin Lin
–
Hmm. So can you yourself official-ize ratings at your will? That was another big question I had.
–
Finn Hulse
·
3 years ago

Log in to reply

@Finn Hulse
–
The answer is obviously yes. Do we do so? Not often. Instead, we may just set an initial guess-timate, and let the system take over. This is similar to when a member sets the initial level of the problem.

We will not entertain requests of "please official-ize my problem ratings,"
–
Calvin Lin
Staff
·
3 years ago

I'm also wondering the above points, although I don't seem to have been victim to any of those problems Finn mentioned.
–
Daniel Liu
·
3 years ago

Log in to reply

@Daniel Liu
–
Really? Here's an example of what I mean. If one of my problems doesn't take root immediately, it will be lost forever. :O
–
Finn Hulse
·
3 years ago

Log in to reply

@Finn Hulse
–
That problem wasn't very good anyways. Not creative at all; in fact, such a quadrilateral can't exist. It deserved to not be known, I guess.
–
Daniel Liu
·
3 years ago

Log in to reply

@Daniel Liu
–
Haha, yeah. Another example of very GOOD problems would be your Inequality 1, 2, and 3 problem set. Remember that?
–
Finn Hulse
·
3 years ago

Log in to reply

@Finn Hulse
–
Oh yea, I think I better add some more problems to that before sharing.
–
Daniel Liu
·
3 years ago

@Finn Hulse
–
@Finn Your examples do not support your argument. If anything, they demonstrate that things are working as they should.

The first (geometry) problem is wrong, and had it's rating removed, hence the "This problem does not have a correct answer. Do you know why?". For the inequality problems, note that the set has not been shared. In fact, these problems each been viewed by \(< 10\) people, and attempted by \( \leq 1 \) person each. As such, I would be highly dubious of the system if any of these problems have an "official rating".

Edit: I have deleted Daniel's question.
–
Calvin Lin
Staff
·
3 years ago

@Finn Hulse
–
I keep on getting the same answer over and over again, and your problem says that it is wrong. I submitted a dispute. I just can't see what I am not getting... plus, there is no solution submitted so far. Mind submitting a solution so I can see how my answer is wrong?

EDIT: I checked it on Wolfram Alpha now, and it also confirms my answer.
–
Daniel Liu
·
3 years ago

Log in to reply

Comment deleted
May 14, 2014

Log in to reply

@Finn Hulse
–
Are you sure? I used wolfram alpha to bash it after I gave up, and it confirms my answer...

Can you post a solution? Thanks.
–
Daniel Liu
·
3 years ago

Log in to reply

@Daniel Liu
–
@Calvin Lin Doesn't Newton's Sums work? I keep on getting the answer of \(\text{[REDACTED]}\) when I use it; what am I missing?
–
Daniel Liu
·
3 years ago

Log in to reply

@Daniel Liu
–
@Daniel Liu I did not work through this problem, and merely trusted that Finn dealt with the reports accordingly. I have just reviewed it, and corrected the problem accordingly. Sorry for the inconvenience.
–
Calvin Lin
Staff
·
3 years ago

## Comments

Sort by:

TopNewestWithout going into technical details, we use a modification of the Elo rating system used in chess. Everything is given a rating value, and an uncertainty value. The amount that your rating changes depends on the relative rating value, and the uncertainty values.

When the uncertainty value is low enough, the rating is displayed. There is no hard and fast rule about the number of attempts / correct attempts. e.g. if 20 level 1's fail to solve a level 5 problem, we would not have much certainty about the rating. Same applies if 20 level 5's solve a level 1 problem.

As a significant proportion of submitted problems have issues with them, we have decided to take a conservative approach to ratings. As the quality of submissions improve (say due to having reputable problem creators, active reporting and responding, or member moderation), we can afford to be more lax. Until them, I rather err on the side of caution. – Calvin Lin Staff · 3 years ago

Log in to reply

– Finn Hulse · 3 years ago

Hmm. So can you yourself official-ize ratings at your will? That was another big question I had.Log in to reply

We will not entertain requests of "please official-ize my problem ratings," – Calvin Lin Staff · 3 years ago

Log in to reply

Plugging Roots into Functions? My personal favorite and only Level 5 problem I have. :D – Finn Hulse · 3 years ago

I see. Also, have you seenLog in to reply

– David Lee · 3 years ago

Sorry, it went to level 3.Log in to reply

– Finn Hulse · 3 years ago

I know... *tear ;|Log in to reply

– Yuxuan Seah · 2 years, 11 months ago

Haha... XDLog in to reply

Log in to reply

– Finn Hulse · 3 years ago

Mmm. Good suggestion. :DLog in to reply

I'm also wondering the above points, although I don't seem to have been victim to any of those problems Finn mentioned. – Daniel Liu · 3 years ago

Log in to reply

– Finn Hulse · 3 years ago

Really? Here's an example of what I mean. If one of my problems doesn't take root immediately, it will be lost forever. :OLog in to reply

– Daniel Liu · 3 years ago

That problem wasn't very good anyways. Not creative at all; in fact, such a quadrilateral can't exist. It deserved to not be known, I guess.Log in to reply

– Finn Hulse · 3 years ago

Haha, yeah. Another example of very GOOD problems would be your Inequality 1, 2, and 3 problem set. Remember that?Log in to reply

– Daniel Liu · 3 years ago

Oh yea, I think I better add some more problems to that before sharing.Log in to reply

– Finn Hulse · 3 years ago

Yeah. :DLog in to reply

– Pankaj Joshi · 3 years ago

This problem's with me too!Log in to reply

The first (geometry) problem is wrong, and had it's rating removed, hence the "This problem does not have a correct answer. Do you know why?". For the inequality problems, note that the set has not been shared. In fact, these problems each been viewed by \(< 10\) people, and attempted by \( \leq 1 \) person each. As such, I would be highly dubious of the system if any of these problems have an "official rating".

Edit: I have deleted Daniel's question. – Calvin Lin Staff · 3 years ago

Log in to reply

– Finn Hulse · 3 years ago

That's all well and good. :DLog in to reply

– Pankaj Joshi · 3 years ago

That's with me too!Log in to reply

– Pankaj Joshi · 3 years ago

That's with me tooLog in to reply

@Suyeon Khim @Calvin Lin @David Mattingly @Arron Kau And all other staff members. :D – Finn Hulse · 3 years ago

Log in to reply

Oh BTW @Daniel Liu I'll give you a high-five over the internet if you can solve Plugging Roots into Functions it's my first truly Level 5 problem. Good luck! :D – Finn Hulse · 3 years ago

Log in to reply

EDIT: I checked it on Wolfram Alpha now, and it also confirms my answer. – Daniel Liu · 3 years ago

Log in to reply

Log in to reply

Can you post a solution? Thanks. – Daniel Liu · 3 years ago

Log in to reply

@Calvin Lin Doesn't Newton's Sums work? I keep on getting the answer of \(\text{[REDACTED]}\) when I use it; what am I missing? – Daniel Liu · 3 years ago

Log in to reply

@Daniel Liu I did not work through this problem, and merely trusted that Finn dealt with the reports accordingly. I have just reviewed it, and corrected the problem accordingly. Sorry for the inconvenience. – Calvin Lin Staff · 3 years ago

Log in to reply

– Finn Hulse · 3 years ago

Yeah sorry. D:Log in to reply

– Finn Hulse · 3 years ago

Woah woah woah bro take the answer off!Log in to reply