Thinking ahead: NTBB 2013

[Darkson]

The maths for clawpomb is spot on, yet how many people complain about that.

[Shteve0]

Thing is Mike I don't know how many people actually have more faith in all this maths than the 'gut feeling suck it and see'.

Thing is koadah, that the central assumption to this whole exercise is that some teams are more powerful than others. Not "feel more powerful", not "look more powerful", but actually outperform the other rosters in their tier more than was the design intention. Holding on to that assertion and then dismissing actual performance data, even selectively picking what feedback 'feels right' and what doesn't, is a nonsense that assumes that one person's gut feeling is better than the combined gut feelings of the design team. If the argument here was that the NTBB was designed because they didn't like one or other central tenet of the LRB design, that's one thing. Accepting that they're fine and then tweaking the rules on the basis that you have a better ability - through your own gaming experiences and those of your mates - to judge what is good and what is bad is another thing entirely. Surely the one advantage we can claim to have over the designers in this respect (apart from having better hair) is that of the data we have available?

But, you're right, some people will continue to believe that positive thinking, talking to your dice, wishing on a star and superior genetics are the real contributors to whether you can win with a roster. Perhaps they're the same people who have their vacuum exorcised when the dog barks at it, click internet banking links from their emails and think that trickle down economics is an effective model for engendering social equality.

[VoodooMike]

I agree. Bad pricing of the various players (Orcs cheaper than Humans, for example), wrong relative costs of the skills (e.g. TV 1000 Amazons vs TV 1000 Dwarfs) and bad calculation of skill stacks (e.g. clawpomb) are other flaws.

I'm not sure I agree with those "other problems" as being obvious places to address supposed imbalance. In the long run it doesn't much matter what players cost, unless we're talking about the perpetual play environment (which NTBB is supposedly not).

Thing is Mike I don't know how many people actually have more faith in all this maths than the 'gut feeling suck it and see'.

And some people still think the earth is flat... doesn't matter what people have faith in, the truth is the truth.

The maths for clawpomb is spot on, yet how many people complain about that.

Doesn't always matter what people COMPLAIN about, it matters what the numbers are saying. If people opt to ignore the numbers then that's their prerogative... but it is opting to be ignorant. People's complaints are the anecdotal evidence that gives us an idea of where to look, using numbers. If the numbers don't bear out the complaints then the complaints are unfounded, plain and simple.

[koadah]

Thing is Mike I don't know how many people actually have more faith in all this maths than the 'gut feeling suck it and see'.

Thing is koadah, that the central assumption to this whole exercise is that some teams are more powerful than others. Not "feel more powerful", not "look more powerful", but actually outperform the other rosters in their tier more than was the design intention. Holding on to that assertion and then dismissing actual performance data, even selectively picking what feedback 'feels right' and what doesn't, is a nonsense that assumes that one person's gut feeling is better than the combined gut feelings of the design team. If the argument here was that the NTBB was designed because they didn't like one or other central tenet of the LRB design, that's one thing. Accepting that they're fine and then tweaking the rules on the basis that you have a better ability - through your own gaming experiences and those of your mates - to judge what is good and what is bad is another thing entirely. Surely the one advantage we can claim to have over the designers in this respect (apart from having better hair) is that of the data we have available?

But, you're right, some people will continue to believe that positive thinking, talking to your dice, wishing on a star and superior genetics are the real contributors to whether you can win with a roster. Perhaps they're the same people who have their vacuum exorcised when the dog barks at it, click internet banking links from their emails and think that trickle down economics is an effective model for engendering social equality.

Sorry mate I was talking to VoodooMike.

If you are the new VM please tell me how the maths are going to tell you what to change. Please tell me how much effort you think it will take to come up with something better than the usual suck it & see.
If you pull the Fumbbl/Cyanide replay files you could probably pull all skills and have a better idea of their team composition. Go ahead, do it. How big are your sample sizes going to be for each match up?

How is your maths going to tell us how well the wrestle zons are going to perform?

If you guys are going to put in the effort then great. I'll have a look when you're done.
Maybe VoodooMike could do it. Personally I doubt that anyone else on here could. And I doubt that Mike could be arsed.
Go ahead prove me wrong.

In the mean time I'd be happy to stick with CRP+ (not NTBB).
Though in reality I haven't had enough trouble with the CRP to even warrant implementing CRP+

[koadah]

[

Thing is Mike I don't know how many people actually have more faith in all this maths than the 'gut feeling suck it and see'.

And some people still think the earth is flat... doesn't matter what people have faith in, the truth is the truth.

Truth? you can't handle... oh wait.

OK, you do it and we'll look.

Edit: It is not even as if effectiveness is the main issue. As we saw with the Khorne Daemons people need to like the roster first. How does the maths deal with that? Are you going to program in a fluff coefficient?

[Shteve0]

Sorry mate I was talking to VoodooMike.

Nice to meet you too.

FWIW, our league is one of the three playtesting NTBB, and we are now reverting to CRP for the forthcoming season. I don't think anyone in our league believes CRP to be perfect, but we originally implemented NTBB on the belief that it was intended to address statistical imbalances in the game and nerf "broken" strategies. I don't like to be critical of people's work, but it seems that certain statistical evidence was dismissed in favour of the stats not matching what the author(s) felt was correct, and other teams were nerfed or buffed based on personal preference, which we don't agree with.

The thrust is that we are quite capable of arguing over a bunch of beers and setting the rules to rights with ideas about what would be more or less powerful in our private theorybowl metagame; we don't need NTBB to do that for us. What we thought we were getting is a set of rules that has carefully selected certain acknowledged (as in, statistically certifiable) outlying performances of certain rosters/skill combos/strategies and sought to carefully restrain them through gentle tweaks and testing.

As for the amazons, the point is not about using the data to test the proposed change, it's using the data to determine why a change is necessary and by how much. For example, you might start with "Amazons perform at 57% win ratio over their first 30 games; we want that closer to 53%, so will be making a fairly firm nerf to the roster" and then going from there. As it is, they missed the first round of NTBB nerfs and then got hit on the basis of something not disimilar to "I heard Amazons won a few tourneys, so I'm going to give them a nerf". Totally different approach, and not one that sits at all comfortably in the "NTBB" shell.

[Darkson]

I personally think that given plasmoid's goal (without going into the rights or wrongs of it), it needs more than just "a few tweaks" - it needs a major rewrite from the ground up.

[VoodooMike]

If you are the new VM please tell me how the maths are going to tell you what to change. Please tell me how much effort you think it will take to come up with something better than the usual suck it & see.
If you pull the Fumbbl/Cyanide replay files you could probably pull all skills and have a better idea of their team composition. Go ahead, do it. How big are your sample sizes going to be for each match up?

I'm the old VM, and I'll tell you that the method you're talking about is not the method I'd prefer to use, as I've said. It's fairly unlikely you're going to be able to determine exact effects of things like a single skill, for the reasons I've already mentioned (excessive error due to additional factors that nobody can agree on a definition for in order to control for them... which makes small, individual effects difficult to pinpoint). That's not the very basic point that Shteve0 was trying to make, though, and his point is very valid.

My focus has always been perpetual play... some folks like to claim that's a totally different animal from league play, and that's simply not true: at a match level, which is where all the games are played, its the same game regardless of how you decide how to match teams together. The main difference between league play and perpetual play is that the latter allows more time for problems to show up, and the assumption that treating league play differently makes is that the lifespan of play will simply be too short for the game to show any age-related difficulties... which is just a matter of laziness. You fix perpetual play, you've fixed league play (assuming a fix is needed, as I've said).

Effort? All of it takes effort.. I'd say it'd take less long-term effort to work on models to come up with answers than it would to keep throwing crap at the rules and forcing people to playtest it until you find something that maybe works... comes down to who puts the effort in, the designer (redesigner) or the players who agree to constantly playtest in order to get some sort of data. Of course, no data is being harmed (or used) in the pursuit of NTBB... its all based on "so yeah, people seem to like it" which can just as easily be attributed to the hawthorne effect as it being a genuinely beneficial change. How long did people sing the praises of the AI mods for LE... only to find they actually didn't change anything? People feel effects based on expectation even if none are present... which is why the world favours numbers over anecdote.

I think I should reiterate a point I made on FUMBBL a long while back - you can start doing most statistical analyses once you hit a sample size of about 30. The problem we run into is that people start demanding all sorts of random things like... changes across TV, or effects at large TV differences, or... and there's no way to get the esoteric data they ask for... which makes them decide there's no point in looking at ANY data, since their question about Orcs vs. Skavens at a TV difference of 6000 on tuesdays couldn't be answered.

I hope you followed all that. It got a bit snakey.

How is your maths going to tell us how well the wrestle zons are going to perform?

The math won't tell you how things will play out - the stats tell you if changes are needed in the first place (assuming we're talking about the meandering tier concept) and should suggest what sort of change is needed if you look at the change in win% across TV levels. A team that starts strong and then drops off as TV increases, likely needs to have better skill access but lose either a strong skill (especially on linemen if applicable) or a point in AG or ST. Initial stats and skill access will be what determines initial roster success, while skill access will determine the change in success over time. The more relevant question to NTBB is - how are you deciding what to change if you're NOT basing it on the actual data? If its going by feel, and success of the changes is based on "do people like it?" then its not narrowing the tiers, its simply "making the game my way" with a glossy name meant to suggest otherwise.

If you guys are going to put in the effort then great. I'll have a look when you're done.
Maybe VoodooMike could do it. Personally I doubt that anyone else on here could. And I doubt that Mike could be arsed.
Go ahead prove me wrong.

How would you judge anything I put forward? You'd "feel it out" which is exactly the worthless method for determining anything that we've been discussing thus far. At the end of the day it's pretty hard to fix something when nobody can agree on what's broken.

Edit: It is not even as if effectiveness is the main issue. As we saw with the Khorne Daemons people need to like the roster first. How does the maths deal with that? Are you going to program in a fluff coefficient?

It's not hard to stay true to the fluff without having to sacrifice on the numbers. People will automatically hate any changes, and then their opinions will stabilize if their change-hating doesn't run change away. You'll notice that the khorne roster now has a split on people who dislike it because its crap, and people who like it because its k-rad demons... its not universal dislike.

[koadah]

The math won't tell you how things will play out - the stats tell you if changes are needed in the first place (assuming we're talking about the meandering tier concept) and should suggest what sort of change is needed if you look at the change in win% across TV levels. A team that starts strong and then drops off as TV increases, likely needs to have better skill access but lose either a strong skill (especially on linemen if applicable) or a point in AG or ST. Initial stats and skill access will be what determines initial roster success, while skill access will determine the change in success over time. The more relevant question to NTBB is - how are you deciding what to change if you're NOT basing it on the actual data? If its going by feel, and success of the changes is based on "do people like it?" then its not narrowing the tiers, its simply "making the game my way" with a glossy name meant to suggest otherwise.

It seems to me that this is what Plasmoid did with the zons. Once the change is made test it to see if it really worked.

How many leagues do we really have testing this? How many zon teams? How many matches? My suspicion is that the margin of error is wide enough to push a planet through.

If you guys are going to put in the effort then great. I'll have a look when you're done.
Maybe VoodooMike could do it. Personally I doubt that anyone else on here could. And I doubt that Mike could be arsed.
Go ahead prove me wrong.

How would you judge anything I put forward? You'd "feel it out" which is exactly the worthless method for determining anything that we've been discussing thus far. At the end of the day it's pretty hard to fix something when nobody can agree on what's broken.

How would I judge it? Same old dinosaur method of wanting see results from a lot of games. Suck it and see. If I could be arsed.
But yes you are right I'd probably use gut feel and eye balling to help decide whether or not I could be arsed.

Edit: It is not even as if effectiveness is the main issue. As we saw with the Khorne Daemons people need to like the roster first. How does the maths deal with that? Are you going to program in a fluff coefficient?

It's not hard to stay true to the fluff without having to sacrifice on the numbers. People will automatically hate any changes, and then their opinions will stabilize if their change-hating doesn't run change away. You'll notice that the khorne roster now has a split on people who dislike it because its crap, and people who like it because its k-rad demons... its not universal dislike.

It's not just about hating changes. It's hating changes we don't like. Or don't think we'll like.

My focus has always been perpetual play... some folks like to claim that's a totally different animal from league play, and that's simply not true: at a match level, which is where all the games are played, its the same game regardless of how you decide how to match teams together. The main difference between league play and perpetual play is that the latter allows more time for problems to show up, and the assumption that treating league play differently makes is that the lifespan of play will simply be too short for the game to show any age-related difficulties... which is just a matter of laziness. You fix perpetual play, you've fixed league play (assuming a fix is needed, as I've said).

Yes it is different.

Plasmoid specifically mentions " tournaments and short leagues". Tournaments are what 3,6 games? Short Leagues? 9, 18, 30?
"You fix perpetual play, you've fixed league play" Sure but fixing perpetual play is harder. If you have already decided that that you are only interested in short leagues you can 'fix' those and leave the long leagues to someone else.

[VoodooMike]

It seems to me that this is what Plasmoid did with the zons.

Except that he admittedly did not - he didn't use the data we're talking about to determine what changes should be made.. or even what type of changes should be made.

How many leagues do we really have testing this? How many zon teams? How many matches? My suspicion is that the margin of error is wide enough to push a planet through.

I don't see that it matters.. again, if you're patching by feel then you'll probably make some things better, some things worse... just like every "official" edition of blood bowl has over the years, and for the same reasons. What's the point in just moving the problems around? Likewise, if the changes are only meant to be for super short-term play, you'll basically NEVER have enough data of the sort that is supposedly relevant, to determine the effects of the changes. It's just more of the same process that created the thing that NTBB suggests its going to fix.

How would I judge it? Same old dinosaur method of wanting see results from a lot of games. Suck it and see. If I could be arsed.
But yes you are right I'd probably use gut feel and eye balling to help decide whether or not I could be arsed.

And it'd only take a decade to get someone to implement the changes somewhere that anyone could test them to the degree that sufficient data would be available, unless, for example, I decided to create a complete BB game setup like FUMBBL... and that's the part where I can't be arsed. Build an entire game I can't profit from due to it belonging to GW just to prove a numeric point to people who are essentially the two guys on the balcony from The Muppet Show... the cost/benefit is pretty crappy.

It's not just about hating changes. It's hating changes we don't like. Or don't think we'll like.

Same thing. Everyone worries they won't like new stuff, which is why they hate change. Better the devil they know than the devil they don't, etc. Most people badmouth Jervis every which way but sunday, but refuse to consider things he doesn't sign off on. It's so candy-assed.

Yes it is different.

No its not different... when you focus on a subset you're just letting statistical error cover up your failures, like sweeping dirt under a rug. The problems are there, you're just using smoke machines to make them seemingly go away.

"You fix perpetual play, you've fixed league play" Sure but fixing perpetual play is harder. If you have already decided that that you are only interested in short leagues you can 'fix' those and leave the long leagues to someone else.

Oh no, you mean we'd have to use our heads instead of our guts? You fix perpetual play and you've fixed the entire game... you focus on a small subset and you might make things seem a bit better in the short term, but likely you've done nothing but make the game more muddled and unbalanced in the long term... which is likely what has happened across many versions of Blood Bowl. To say "it wasn't designed for perpetual play" is exactly the same as saying "the game wasn't very well designed", because we're saying that it falls apart if you play too long. It isn't a "design decision" it is a "design flaw". Focusing on short term play is a decision again, to perpetuate a design flaw... which is assuming you even use appropriate data to make your decisions, which we know NTBB has not (by plasmoid's own admission).

[MattDakka]

You fix perpetual play and you've fixed the entire game... you focus on a small subset and you might make things seem a bit better in the short term, but likely you've done nothing but make the game more muddled and unbalanced in the long term... which is likely what has happened across many versions of Blood Bowl. To say "it wasn't designed for perpetual play" is exactly the same as saying "the game wasn't very well designed", because we're saying that it falls apart if you play too long. It isn't a "design decision" it is a "design flaw". Focusing on short term play is a decision again, to perpetuate a design flaw... which is assuming you even use appropriate data to make your decisions, which we know NTBB has not (by plasmoid's own admission).

+1. Well put. TV bad calculation is another proof of design flaw.
If TV were very accurate over time, perpetual MM/Box leagues would be more balanced, and we wouldn't see the minmax/sweetspotting phenomena.

[koadah]

It seems to me that this is what Plasmoid did with the zons.

Except that he admittedly did not - he didn't use the data we're talking about to determine what changes should be made.. or even what type of changes should be made.

And he just happened to come up with the kind of changes that you think he should have? \o/ for coincidence. Or was it gut feel.

How many leagues do we really have testing this? How many zon teams? How many matches? My suspicion is that the margin of error is wide enough to push a planet through.

I don't see that it matters.. again, if you're patching by feel then you'll probably make some things better, some things worse... just like every "official" edition of blood bowl has over the years, and for the same reasons. What's the point in just moving the problems around? Likewise, if the changes are only meant to be for super short-term play, you'll basically NEVER have enough data of the sort that is supposedly relevant, to determine the effects of the changes. It's just more of the same process that created the thing that NTBB suggests its going to fix.

If you play enough games you'll have enough data. If you don't you won't. If I understand you and Dode correctly the more games the smaller your margin of error. The larger your margin of error the less confidence and the better case for the manual override.

How would I judge it? Same old dinosaur method of wanting see results from a lot of games. Suck it and see. If I could be arsed.
But yes you are right I'd probably use gut feel and eye balling to help decide whether or not I could be arsed.

And it'd only take a decade to get someone to implement the changes somewhere that anyone could test them to the degree that sufficient data would be available, unless, for example, I decided to create a complete BB game setup like FUMBBL... and that's the part where I can't be arsed. Build an entire game I can't profit from due to it belonging to GW just to prove a numeric point to people who are essentially the two guys on the balcony from The Muppet Show... the cost/benefit is pretty crappy.

I believe that Fumbbl already covers the rule changes except the bank. So we're waiting on custom rosters for the rest.
eta? I dunno. But what i'm asking is where are your rules? What data do you have to back them up?

It's not just about hating changes. It's hating changes we don't like. Or don't think we'll like.

Same thing. Everyone worries they won't like new stuff, which is why they hate change. Better the devil they know than the devil they don't, etc. Most people badmouth Jervis every which way but sunday, but refuse to consider things he doesn't sign off on. It's so candy-assed.

I was all for going to LRB5 from LRB4 and I'm all for trying CRP+ now. I do like my blodge zon blitzers though. I must be one of these evil min/maxers that we keep hearing about.

It seems to me that Amazons and wood elves are pretty under used considering that they are winningest teams around. Maybe definitions of 'good' vary.

Yes it is different.

No its not different... when you focus on a subset you're just letting statistical error cover up your failures, like sweeping dirt under a rug. The problems are there, you're just using smoke machines to make them seemingly go away.

Oh yes it is and oh no you're not.

If anything 1000 games at 1-30 games is of more use than 1000 games spread over the whole range if 1-30 games is all you are interested.

I'm really the opposite. I don't care about the 1-20 games. It's the 20/30 games + that would interest me.
So it can be as unbalanced as you like if it evens up later on. And, um... you tack on some artificial constraints to try to stop it unbalancing again later on.

"You fix perpetual play, you've fixed league play" Sure but fixing perpetual play is harder. If you have already decided that that you are only interested in short leagues you can 'fix' those and leave the long leagues to someone else.

Oh no, you mean we'd have to use our heads instead of our guts? You fix perpetual play and you've fixed the entire game... you focus on a small subset and you might make things seem a bit better in the short term, but likely you've done nothing but make the game more muddled and unbalanced in the long term... which is likely what has happened across many versions of Blood Bowl. To say "it wasn't designed for perpetual play" is exactly the same as saying "the game wasn't very well designed", because we're saying that it falls apart if you play too long. It isn't a "design decision" it is a "design flaw". Focusing on short term play is a decision again, to perpetuate a design flaw... which is assuming you even use appropriate data to make your decisions, which we know NTBB has not (by plasmoid's own admission).

Go ahead. Fix it all. But try not to make it too bland.
It would be fine for Plasmoid to say it's not designed for perpetual play. People who want perpetual play can choose to use his rules or not.
What do you guys have that is better than CRP? Where is your evidence?
What do you have that is better than CRP+? Where is your evidence?
What do you have that is better than NTBB? etc etc

[harroguk]

Who cares, if you dont like it, dont play it. but let the people who do want to play it get on with it.

[koadah]

You fix perpetual play and you've fixed the entire game... you focus on a small subset and you might make things seem a bit better in the short term, but likely you've done nothing but make the game more muddled and unbalanced in the long term... which is likely what has happened across many versions of Blood Bowl. To say "it wasn't designed for perpetual play" is exactly the same as saying "the game wasn't very well designed", because we're saying that it falls apart if you play too long. It isn't a "design decision" it is a "design flaw". Focusing on short term play is a decision again, to perpetuate a design flaw... which is assuming you even use appropriate data to make your decisions, which we know NTBB has not (by plasmoid's own admission).

+1. Well put. TV bad calculation is another proof of design flaw.
If TV were very accurate over time, perpetual MM/Box leagues would be more balanced, and we wouldn't see the minmax/sweetspotting phenomena.

They didn't design for those TV matched leagues at all.

It's up to those leagues to be brave and design their own TV if they dare.

I would guess that they kept it simple for easy calculation on TT. If the computer is doing it it can be as hairy as you like.

[Shteve0]

Koadah, you're missing the point: the amazon roster performance stats don't support this "fix"; and the fix is not a fix, it's a rewrite. Plasmoid has admitted to intentionally changing their playstyle, and for my money has got it badly wrong (but that's a different argument based on how it impacts their progression). Meanwhile, the stats he chose as a supporting argument for the tweaks indicate that the Necro and Pro Elves enjoy excellent win% in the range he's looking to normalise, but skipped the nerfs because he felt they weren't as good as the data suggests (!) All of this is fine, but don't try and tell me that the rosters are more balanced than CRP. There's no indication that the theorygoal here (to pull in the statistical outliers) has been addressed, other than some fairly massive boosts to the stunty teams.

The thrust of it is that I can only come up with one definition for "Narrow Tier Blood Bowl", and that essentially takes the tiers - a loose, open ended definition of team level based on overlapping bands of win percentages - and making those bands, you know, narrower. Under the current definitions, a team at 45% win ratio is potentially in any of T2, T1.5 and T1. Some teams in T2 report consitently better performances (ie a higher win%) than those in T1. By taking a team in mid T1 - skaven, for example - and giving them a nerf based on the fact that you and your mates once played in a league with an annoying skaven player, all you're doing is moving them within the tier bands, you're not actually doing anything to address the tier structure at all.

Essentially if you were to take the NT concept literally and actually look to address the tier bands in a meaningful manner (in order to, for example, ensure that players who took a team to a tournament knew precisely the power level of the team they were fielding), you'd want to address the banding overlaps, and furthermore I'd strongly advocate leaving a little daylight in between. For example, this might look like T1 at 54-58%, T1.5 at 48-52%, T2 at 42-46% and T3 at 34-38%. That's what a narrow(er) tier system looks like, and yes it's not going to be easy to get right and will take gut calls, but there you have your target win%s, TV matching range for your data collection (tourneys, so looking at resurrection data in the most played formats) and can tweak from there. The tweaking will be largely gut based, yes, but you will have the amount you want to nerf by (eg looking for a 2.7% win% reduction) to guide you. And no, data won't be available for how this works immediately, but at least you have the justification for why you want to do it (and how much by) in black and white.