Hey Ben, this looks like a neat project/app! I'm always curious with apps like this about how financially viable it is. Seems to be a fair solution to be able to generate credits yourself by ranking and then buying more credits to get ranked, do you have any numbers or insights you are willing to share? (or have shared somewhere else before?)

Hi, thanks!

That's a great question. It is not making anybody rich, but I cover my costs and make a few hundred extra a month. I think on the order of $500+ MRR.

Besides buying credits there is also a pro membership where people choose which demographics rank them (age, location, gender), can have more tests, larger test sizes and always be at the front of the queue for ranking.

This is cool. It could be generalized into arbitrary ranking (not just photos).

Questions: how do you plan to deal with people that will randomly rank other's pictures to get credits quickly (or even build a script to do so)?

Hi! Sorry for the late response, went to bed and forgot about this post. Thanks :)

We check to make certain people aren't ranking formulaically, and if they are we invalidate their votes and ban them automatically. We also check how close people's rankings are to the consensus retroactively and do some fancy anti cheating patterns there as well.

Very interesting! Have you also thought about using an invitation system (like the one on lobste.rs - https://lobste.rs/about#invitations)?

Would you mind to tell a bit more about the exact logic and algorithms you use for those checks? I'm working on project in a totally different space (offers) but it shares the same idea where users vote on other users submissions and get credits for doing so (to incentivize participation and keep it fair). I haven't really worked out how to prevent abuse, so I'd be curious to learn from others. If you can share things (even just resources) I'd be super happy. Feel free to shoot me a mail to ca-rankpicstuff@willscher.com in case you don't want to make things public.