Tribunal... Disappointment - Redtracker

Lurkios

So I did a little experiment.

I wait for a day that all the cases I judge are "punish" cases. The next days e-mail grants me 50lp for my correct judgements. (100%)

The next day I skip all punish cases and pardon only cases which, to me, CLEARLY deserve to be pardoned (yes, this took nearly an hour). The next days e-mail grants me 20lp. (40%)

Now obviously this is an excessively small trial group, but really? 40%? Just by mashing the report button because I didn't like my lanes face I have a 60% chance of getting them banned (or whatever else RIOT comes up with) for NOTHING? Is 5lp really worth all that?

1 year, 9 months ago

Source

Pendragon

Director of Player Experience

Two comments

First - Sometimes it takes a few days for cases to be closed, so you may receive another Email with the rest of the case rewards if you were more accurate.

Second - we regularly audit the Tribunal, and set threshholds for required accuracy based on the actual percentage of cases that should be "punished/pardoned" through intelligent sampling.

Players who deviate too far from the actual punish/pardon ratios will get their rights revoked.

1 year, 9 months ago

Source

Pendragon

Director of Player Experience

So would this testing be more accurate if say he tried it for a month, then waited a few weeks to "clear his queue"? Or would we not really be able to test the effectiveness ourselves?

His testing will never be accurate the way he conducted it. (unless his accuracy was random).

He needs a much larger sample size for a pattern to develop.

http://en.wikipedia.org/wiki/Law_of_large_numbers

1 year, 9 months ago

Source

Pendragon

Director of Player Experience

and what happens to the people that got banned because of them?

It requires too many votes for that (low) number of abusers to have an impact.

1 year, 9 months ago

Source

Pendragon

Director of Player Experience

Remember too -- As we assess the accuracy of the Tribunal, we are slowly opening it up to more and more "questionable" cases.

When it was launched - it was basically 100% cases were guilty (statistical probability). Now it's less - but it's still high enough for all-guilty sample patterns to emerge regularly.

1 year, 9 months ago

Source