Waste less time on Facebook — follow Brilliant.

Get feedback when you report a problem!

We have updated the "Report problem" procedure, so that you can interact with the problem creator and the community. This will help you understand if the problem is indeed wrong, or if there is a flaw in your thinking. You will also be notified of what changes, if any, have been made.

To report a problem, take the following sequence of steps:

  1. Verify your email address, so that you can be updated about any proceedings.
  2. Select "Report Problem" from the "dot dot dot" menu in the lower right corner.
  3. Acknowledge that you want to continue
  4. Submit your clarification/dispute/report
  5. Let the problem creator / community respond. You can upvote disputes, and vote on comments.
  6. To get back to your dispute, click on "View disputes" in the "dot dot dot" menu. This will work even if the problem has been deleted, and is a way for us to inform you that "your dispute is valid and the creator has withdrawn the question". Note that you can only submit 1 dispute per problem, so edit or comment as necessary, if you have further doubts.

As this is a new feature, we would like to hear what you think about it. Try this non-sensical problem if you want to play around with the dispute format. There will certainly be further improvements over the next few weeks. If you have any suggestions for improvement, please comment!

For problem creators and interested general public who want to learn more about reporting and flagging, please join the discussion Why is my problem flagged

Note by Calvin Lin
2 years, 4 months ago

No vote yet
1 vote


Sort by:

Top Newest

First off, just thought I should mention that there is a typo in step 3 above the "Cancel" and "Continue" boxes - I think that it should read ".... to file a dispute or view existing ones, .....".

Overall, this looks like a great upgrade, but I do have a couple of concerns.

(i) I have reported questions because of a lack of clarity without having entered an answer. If I enter the dispute process now to detail a clarity concern and the issue is eventually acknowledged and resolved, I will have forfeited my ability to answer the problem even though I've assisted in improving it and might still want to get credit for solving it.

(ii) Suppose I enter an answer which is considered incorrect, then dispute the posted answer. If the posted answer is found to be incorrect and my original answer is correct, will I get credit for my correct answer, (as has been the case up until now), or will the forfeiture of my right to answer by entering the dispute process end up preventing me from getting the credit for my correct answer? And if both the posted answer and my original answer both end up being incorrect, then what would happen?

Regards. Brian Charlesworth · 2 years, 4 months ago

Log in to reply

@Brian Charlesworth Thanks! We missed that typo completely :) Let me get that fixed.

(i) What we noticed was that - People were entering their various answers before submitting a report OR
- After reporting a problem, they rarely return to it even after it has been corrected.
For the majority, this system is an improvement because it lets them be aware of what has happened.

Of course, it would be great if many people would like to report questions to help clarify a certain concern, as that improves the overall quality of the site. When we get feedback that people want to do this, we will look at allowing for ways to submit a "minor report".

(ii) Yes, when (if) the problem gets updated to the correct answer "B", all of those who answered "B" will be marked correct. The forfeiture of your right to enter more answers does not invalidate the attempts that you have already made.
If both the posted answer, and your original answer are wrong, then you will be informed that the answer has been updated to the correct answer "C", preferably with an explanation. Calvin Lin Staff · 2 years, 4 months ago

Log in to reply

@Calvin Lin Thanks for the quick response.

(i) Yes, I can see how the new system will be an improvement for the majority. I think then that my approach to concerns with clarity will be to enter an answer, (double-checked), based on the most likely interpretation. If it is considered incorrect then I will enter the dispute process and state my reasoning and/or concerns, fully aware of the potential forfeiture. (Knowing that I was justified in reporting the problem will probably be almost as satisfying as actually solving it.)

(ii) Great. That sounds reasonable.

As with any upgrade I'm sure you're anticipating having to make a few tweaks here and there, but at this stage it looks promising. :) Brian Charlesworth · 2 years, 4 months ago

Log in to reply

@Brian Charlesworth Right, we prefer to release a feature and see how the community reacts to it / actually uses it, before making further decisions about how to tweak it to perfection. I believe that we are most of the way there, but the hardest part is always anticipating how others will use the feature.

After all, we are building something a brand new experience here ;) It's not like we can go to Yahoo Answers and directly copy their format. There will always be some experimentation and tweaking, as we get features to achieve their goals. Calvin Lin Staff · 2 years, 4 months ago

Log in to reply

@Calvin Lin Yahoo Answers royally messed up their roll-out of changes early last year, so much so that I jumped off and boarded Good Ship Brilliant. You're 'growing your brand' the right way here; may the good work continue. :) Brian Charlesworth · 2 years, 4 months ago

Log in to reply

@Calvin Lin What about this bug

Calvin Edit: Moving this (slightly off-topic) conversation here. Krishna Sharma · 2 years, 4 months ago

Log in to reply

@Calvin Lin Sorry for being such a stickler, but I just thought I should mention that the typo I pointed out is.... still a typo. :) Brian Charlesworth · 2 years, 4 months ago

Log in to reply

I have occasionally noticed problems with a difficulty level not matching the true level of the problem--for instance a calculus level 5 question that is calc 1 related rates that could appear on a standard assignment. Is there a process for remedying/reporting this? I hate feeling as if the site content is being diluted. M M · 2 years, 4 months ago

Log in to reply

@M M Certainly. I would appreciate if you file a report, and I will take a look at it.

Sometimes, the creator could set the wrong level. I am pretty confident in the ability of our rating system to bring it in line over time. For new problems, they are still highly susceptible to initial conditions and perturbations. We strike a fine balance between withholding ratings till we are somewhat confident, vs releasing ratings early for the benefit of the creators / community. Calvin Lin Staff · 2 years, 4 months ago

Log in to reply

I just waded through the "Reframing Problems" and the answers are unintelligible to anyone that doesn't know the notation scheme they use; They never tell us how to get "k." And giving me 2 random numbers inside a bracket doesn't help me, AT ALL understand WTF they are talking about.. It's like I jumped from an Algebra review course to DiffyQ.

I have to say I understand LESS now that I have tried to parse out their examples than I did before... Michael Grafton · 1 month, 2 weeks ago

Log in to reply

@Michael Grafton Thanks for the feedback. Ace the AMC is an advanced exploration, which assumes that you are familiar with several of the basics. As such, it skips over basic explanations in favor of getting to the deeper theory.

From your comment of "giving me 2 random numbers inside a bracket doesn't help me, AT ALL understand WTF they are talking about", it sounds to me that you are unfamiliar with combination notation of \( { a \choose b } \). If that's the case, I would recommend that you check out the Combinations chapter to build up the basics of counting.

I will add a note to make it clearer when someone should review other material before proceeding, especially if they are unfamiliar with concepts/notation/definitions mentioned in the solution. Calvin Lin Staff · 1 month, 2 weeks ago

Log in to reply

@Calvin Lin I have been plugging along and this chapter, "Reframing Problems" is the first one to have notational issues. Perhaps my impending geezerhood is the problem? I last took Calculus around 30 years ago. Michael Grafton · 1 month, 2 weeks ago

Log in to reply

@Michael Grafton For the "advanced" material, we made the unwritten assumption that the simplier concepts would not be explained in full detail, and instead chose to focus on developing out the through processes for people that are familiar with it already.

We can do a much better job with making it clearer where / how people can access more elementary material when they get stuck, so that they do not feel lost and helpless.

If you see this again, I would appreciate if you go to the menu and select "Report problem", which would make it easier for us to respond. Calvin Lin Staff · 1 month, 2 weeks ago

Log in to reply

I'm not sure where to ask this, so here, but is there any way to delete a set you've accidentally made? Omkar Kulkarni · 2 years, 3 months ago

Log in to reply

@Omkar Kulkarni No, we do not have the functionality to delete a set as yet. You can remove all items in the set and rename it, if you don't want it anymore. Calvin Lin Staff · 2 years, 3 months ago

Log in to reply

A quick comment: In Which is correct? and The Mississippi Problem, if I look at the disputes, there are more than what can fit on a single screen. However, there is no vertical scroll bar, so I can't see all of the disputes. This happens in both Firefox and Chrome. Jon Haussmann · 2 years, 4 months ago

Log in to reply

@Jon Haussmann Yes, we are aware that scrolling on disputes (and sometimes solutions) doesn't always work as it should. We are working on fixing this bug.

One way to get around this, is to hit reply, which seems to force the screen to load more. Sorry for the inconvenience. Calvin Lin Staff · 2 years, 4 months ago

Log in to reply

I have to re-iterate one of Brian's concerns: To submit a report, a user should not have to forfeit his/her ability to answer the problem. Under the previous system, the following sequence was possible: I see a flawed problem, and I submit a report \(\rightarrow\) The proposer corrects the problem \(\rightarrow\) I can now answer the corrected problem, and this is how it should work. If I have to give up my right to submit an answer, it creates a strong disincentive for me to not submit a report. Jon Haussmann · 2 years, 4 months ago

Log in to reply

@Jon Haussmann Yes, I understand. I will be working on allowing for reporting of minor issues which are easily clarified. However, this will add much complexity to the entire reporting interaction.
It will take us some time to figure out a simple approach. In the meantime, please bear with it. Calvin Lin Staff · 2 years, 4 months ago

Log in to reply

Thank you very much, my problems had typos recently. John Errol Obia · 2 years, 4 months ago

Log in to reply

Good Ayanlaja Adebola · 2 years, 4 months ago

Log in to reply

When can we start posting problems on Brilliant from Android®??@Calvin Lin

Calvin Edit: Comment is off-topic, so move this discussion elsewhere if you want to continue. There is a new andorid release planned over the next 48 hours. Vikram Venkat · 2 years, 4 months ago

Log in to reply


Problem Loading...

Note Loading...

Set Loading...