What is the best way to do an experiment to determine the percentage of subscribers to Brilliant? Since there is no way to determine exactly how many subscribers there are, or even the exact number of Brilliant users, we’ll have to do one of the following:
Look for every user and ask whether they are a subscriber.
Take a sample size of users you can find in the website.
Go to your profile page, change the url to a random string of letters and/or numbers.
(Note: There is a way to take educated guesses. You can look at the format and change appropriate parts.)
Each of the options has a downside:
Option 1: It’s very hard to locate all users on Brilliant. Most people don’t even visit 500 websites in a day!
Option 2: It’s very likely that the sample size is biased. It’s possible that people subscribed because they are using Brilliant a lot, therefore it’s easier to locate them. Others may not regularly use the site.
Option 3: There are many possible combinations to guess.
So, the challenge is to find an efficient way to find the percent of Brilliant users who are subscribers.
Let’s say that we want the method to be:
Manageable in 24 hours
Able to yield a result within 1% of the actual answer 99% of time