Reconsidering the ruling on Khorne?

[Digger Goreman]

1. I have no idea if the tiers were developed with the idea that tournaments would be swiss in mind. Do you know that they weren't? If so then it's not a source of error at all.

I was under the impression that CRP was developed with reference to Open Leagues, no other format was considered relevant? Or was that just JJ's view?

Since BB started as a board game, I always assumed so.... It is what I referenced in my manifesto when I wrote about tournaments and online being artificial environments.... Yes, each format has something to offer, but the strength of one is problematic when applied to the other.... The lessons are similar to what happens when using animal subjects in human medicine.... Besides all the variables you must control, or account for, there will be a varying degree or error whether using fish, various rodentia, or even humans....

Someone, elsewhere, suggested the reality of what we have now: different rule sets for the different formats.... Cyanide is doing their thing... NAF concentrates on the tournament scene (with the massive tourney variables alluded to), FUMBBL tries to follow, but faces coding challenges on some skills, and TT varies in house rules, options and selected formats.... Clock, or not, IP, set schedules vs challenges, cherry picking, what members habitually play, or refuse to play... the variables are mind boggling....

I label no one... but tis a fools errand....

[Gaixo]

I label no one...

That's a good one.

[Wulfyn]

1. The Swiss format - matching strong against strong and weak against weak will pull the stats for both towards the middle (50%).

Good thinking, but this is not quite necessarily true. Let's take Lizardmen and Wood Elves as an example. Both teams are good and are likely to win against lesser teams, coach skill being equal. That increases the chance that Lizards and Wood Elves will face each other later on. However we know this matchup to be pretty favourable for Wood Elves who can Wardancer leap in to kill Skink ball carriers. Therefore it could further increase the Wood Elf win rate.

[Shteve0]

However we know this matchup to be

GAAAAH!

[Wulfyn]

However we know this matchup to be

GAAAAH!

Haha

[Milo]

Really interesting discussion here, although it seems to have diverged dramatically from the Khorne roster it was originally about to just a discussion of statistics and CRP+. (Which, for the record, I like some of and not other parts, but I'd be happy to discuss that directly with Martin if he wants, rather than going into it here.)

I do want to make a couple of points. First of all, I have to give a lot of credit to Martin for taking the -- at times harshly worded -- criticism of his CRP+ constructively. I know well what it can be like to become protective of the rules you've designed and how hard it can be to maintain enough neutrality towards them to listen to and incorporate dissenting opinions.

Secondly, the CRP is largely stable and widely satisfying, certainly the closest ruleset we have ever had to supporting all three environments and any degree of team longevity. For that reason, I understand the reluctance to make any changes without thorough testing. That said, I think it's worth acknowledging that Martin has done far more thorough research and testing than Jervis originally did with many of his rules changes. In that regard, I think people are holding him and his rules to an unfairly high bar. You're asking someone to support his ideas with in-depth statistical review and tens of thousands of datapoints to justify slight tweaks to a ruleset that was originally written by a guy in his bathtub.

[dode74]

Martin doesn't have the remit JJ does. There is no comparison.

[Milo]

Martin doesn't have the remit JJ does. There is no comparison.

I would argue that at the moment, there is no one with the remit for rules related to Blood Bowl. GW has abandoned it, the BBRC was dissolved. Maybe Cyanide, although I doubt their changes would meet widespread approval.

That is a vacuum of responsibility that I think anyone could step into. Does it grant any special privilege to Martin's set of house rules over anyone else's? No, but at least he has devoted the time and testing to show that his rules are not being thought up whilst relaxing in the bathtub.

[dode74]

What testing? Some games have been played but there had been precisely no analysis of the results.

And I won't speculate as to where he thought them up

[Milo]

I think most blood bowl rules are tested anecdotally at first from local league/tournament results. (Some are not tested at all, actually.) Probability analysis can also be done to determine the impact of some changes.

With online leagues, it would be possible to get a much larger dataset of both coach feedback and win/loss results. I'm not sure it's really possible to do more than that.

Let me ask, though, since you are one of the major dissenting voices on this thread:

1) Whom would you consider to have the necessary "remit" to suggest rules changes? (Note that I say suggest, not impose nor enforce.)
2) What degree of testing would you consider necessary to resolve the "burden of proof" you are suggesting his CRP+ lacks?

[dode74]

The Vault process was more than anecdotal. Furthermore, "we've always done it that way" is an invitation to make the same mistakes.

1. Anyone can suggest anything. I've no problem with suggestions when they are presented as such. Having read the thread you'll have noted that one of my major issues is presentational: the name CRP+ evokes improvement and officiality, as does the wording of the invocation of the BBRC (although I've not looked at his site in a few weeks to see if that had changed).

2. It would have to be shown to meet its stated aims, the majority of which are measurable. I've been fairly clear on this earlier in the thread.

[Milo]

Refresh me on the vault process, please; I think I was out of the loop at that point of the BBRC.

[dode74]

Probably better if that comes from someone closely involved rather than second hand from me.

[plasmoid]

Hi Milo and Dode,

Milo - I'd be perfectly happy to hear/discuss your thoughts on the CRP+ rules. Just send me a PM

Dode

CRP+ evokes improvement and officiality, as does the wording of the invocation of the BBRC (although I've not looked at his site in a few weeks to see if that had changed).

I did actually rework the site a few weeks ago like I said I would.
I don't have a list of what I did, but I tried to follow your suggestions. Some if it has made reading the site more "front-heavy", rather than casual explanations first and then more thorough explanations further in. So, not as good a read IMO, but maybe it will get some of the accusations off my back.
Off the top of my head:
*I rewrote the introduction.
*I moved up the stuff about redefining the tiers.
*I moved up the stuff about the changes not being driven by data points.

I still think your beef with the name is unfair, but I get that we won't agree on it.
Cheers
Martin

[plasmoid]

Oh - Milo and Dode, there's also this:

The Vault process was more than anecdotal. Furthermore, "we've always done it that way" is an invitation to make the same mistakes.

I think the LRB and Vault process has a rather impressive improvement record actually. And it got things done.
It seems to me that that the alternative has a silver lining, but in reality all it does is preserve the status quo. Which is perfect, if that's what you want. (The general you, just to avoid any misunderstandings).

I did the best research I could, and the rules are getting played. And I get totally anecdotal feedback on them. That's what I have.

I also know where data collection will get me. Given the scale of playing/testing, for the next 10+ years all the CI's will be so wide, that it will be impossible to tell whether the changes worked or not. Couple that with the null hypothesis all this will do is get me told Again and Again that I can't prove that they worked. I already knew that.

Milo said:

I think people are holding him and his rules to an unfairly high bar

I'm glad you say that. It certainly is completely unprecedented when it comes to developing BB rules.

Dode said:

The Vault process was more than anecdotal.

It was totally anecdotal. If only the old specialist games forum still existed....
The only serious match-level data Collection done was done by me, and those resulted in a few roster changes.
Everything else was discussion driven.

Sure, Tom recieved a lot of emails that never made it to the forums. Perhaps some of them even contained match level data (League results tables). But if they were never presented in any public discussion. Nor were they presented to me when I collected the match level data for the BBRC.

Cheers
Martin