Saturday, December 14, 2019

Western States Has A Problem – Part III: A Non-Weighted, Stratified Lottery Proposal

Previous posts in this series:

The Proposal:


1. Eliminate any weighted lottery system. Abolish compounding tickets: one ticket per entrant, per year.

2. Create a stratified, multiple sub-lottery system comprised of four groups: Never-Starters, Veterans, Super-Masters, Everyone Else. Lottery odds are based on number of times a runner has run the event, not number of tickets.

*****
Background. This proposed lottery alternative is based on two pre-existing systems: the original (basic) Western States lottery, and components of the stratified lottery system current employed by The Hardrock Hundred Endurance Run (HR100).

The former - a simple, "one year, one ticket" basic lottery - was the original lottery strategy of Western States from 1981 until 2011, when then first weighted system, "n+1", was instituted. This system, while blind to both experience, previous runs, and previous lottery attempts, also eliminates any incentives for entering prematurely, or continuing to enter in order to “keep my tickets”.

The latter is a lottery system developed by The Hardrock 100: a stratified lottery system consisting of three separate lotteries, one each for:
  • First-Timers
  • Veterans (5 or more finishes)
  • Everyone Else
In this system, three separate lotteries (and the same number of waitlists) are held, each with a set number of entries. Prospective entrants fit into one of three groups, but transition over time, depending on entry and race experience. A stratified system allows the race to pre-determine the desired race composition, balancing new runners and veterans.

The Details. Here is a detailed explanation of the proposed alternative:

1. Eliminate any Weighted Lottery System

De-incentivize premature entry by abolishing the compounding ticket system, entirely. This includes both the current "2n-1", as well as the preceding "n+1" system.

The only way to curb the trend of premature entry is to eliminate a geometric entry system that rewards it. With entrant numbers skyrocketing and all entry odds plummeting, runners know fully well this reality: "If I want to ever run Western States I have to enter now". 


Below are some interesting data:




This is the overall growth of lottery entrant per year, dating back to 2000. Two important milestones are worth noting:
  1. the n+1 lottery began in 2011 
  2. the geometric system began 2012
A less important, but also interesting milestone: 50-mile qualifiers were eliminated, with only a limited number of 100K qualifiers allowed, starting in ~2013. 

Next, take a look at this graph:
This represents the growth of one’s ticket cache, for each consecutive year of entry.

It’s impossible to prove that a geometric system can definitively cause the geometric growth currently observed in the lottery.

However, basic economics tells us: a geometric incentive will create a geometric demand.

Returning to the Western States entry data: when superimposing the historic changes to the lottery, the correlation between those changes and the resultant entry numbers becomes more compelling:


This hybrid chart superimposes the history of the lottery: what changes were made, when, and the correlated changes in applicants. 


Note three fairly distinctive slopes in the history of the Western States lottery: the basic lottery era, the “n+1” years, and the geometric system. (Also note the small plateau that occcured in 2014 after the tightening of qualifier races).


Looking closer at each period:

From 2000 to 2010, a basic lottery was held that also included the Two-Time-Loser clause: if you entered twice before and we're not picked, you would automatically enter.

Growth was relatively steady until the end of the decade. Growth was modest early on (+50-100 added applicants per year).

During this time of 500 or less applicants, the "Two-Time Loser" functioned well. But once interested piqued in the late 2000s, this initial incentive was quickly overwhelmed.

(Note: a reader might think that the Two-Time-Loser was a major incentive. It wasn't. If anything, it was a dis-incentive to prematurely apply because it guaranteed entry. Why rush to something you're guaranteed to get? Additionally: at this time with the basic lottery, odds were still holding at 50% for all applicants).

As total applicants began to triple and even quadruple the maximum allowable field size, it became clear that in a basic, indiscriminate lottery, eventual selection could take inordinately long. Moreover, a guaranteed entry after two "losses" became impossible. Thus, the Two-Time Loser clause had to be abandoned and the “n+1” system was conceived and implemented for the 2011 race:




Correlating with the implementation of “n+1”, we see a growth increase of upwards of 400 applicants per year from 2011 to 2013.

Somewhere along the line, the Race got even more concerned and felt that additional protections be put into place to allow for a more expedient and “guaranteed” entry for committed runners. The 2n-1 system was implemented in 2013 (of note: tickets were retroactively compounded for 2nd and 3rd-year applicants from the previous n+1 system):




From 2014 through the 2020 lottery, growth began in this period at 400 applicants per year and has progressively increased to a current annual growth rate of 800-1000 applicants per year.  The race now increases in applicants per year, more than double the total applicant numbers (~500) from 20 years ago. 


While correlation does not equal causation, the changes made to the lottery do, in fact, act like independent (input) variables with a dependent variable (applicants) result: an adjustment “X” yields in a change of “Y”. 

But indeed, it is just a correlation that is impossible to prove. 

Additionally, one could argue that growth in Western States also correlates to overall growth ultrarunning. 

Here is that data comparing Western States applicants and the total number of hundred-mile finishers in North America. 




A cursory look would say that the Western States applicants fairly neatly correlate with overall growth in 100-mile finishes.

They do, early on: before the weighted lotteries began.

But looking closely, at 2015 and beyond - when the geometric lottery system really took hold at Western States:




And what you’ll see is that, despite an ever-increasing number of new hundred-miler races in North America (not all of which are Western States qualifiers -- thus this group can grow, independently), Western States lottery applicants have grown at nearly double the rate of the rest of the 100-milers on the continent.

Something else is driving that additional growth.

Has Western States gotten that much cooler since 2015? 

One could argue that the modern “documentary era” - from films such as Unbreakable (2011) - can spike interest in Western States more than the rest of the sport. However, two counter-points:
  • Other (and equally compelling) documentaries for other races abound
  • Western States has always had (and initially cornered the market in) cool documentaries
It is unlikely that media exposure explains the growth disparity. 

Have any other changes to the race occurred in the last five-plus years to account for the growth?

The biggest change: an incentivized lottery system. 

One compelling experiment: abolish the weighted system, and track the next five years of applicant data. I predict applicants would drop precipitously - by jettisoning the premature entries - and once again align with the overall growth trend in 100-mile finishes.

Returning to the geometric data and the frightening right side of the chart:




If this growth continues at the same rate - far exceeding the growth of all other hundred-milers - we may see this year’s record number of applicants double in four years. 

It’s worth considering if the very system devised to handle growth, is in fact, the overt cause. 

I hope I’m wrong. But if I’m not: 
*****

Eliminating a weighted system would potentially decrease the rapid growth in lottery applicants by all-but-eliminating two substantial cohorts of "premature runners":

A. First-Timers “Getting in Line”.  After running his/her first qualifying race, current prospective first-time Western States runners face a dilemma:



enter the lottery now, regardless of readiness or willingness to run this year
-or-
wait, and face worsening first-year ticket odds with each passing year

That is the reality, as single-ticket odds have plummeted in the past five years: on average, shrinking in half with each passing year.

In fact, for today’s new entrants, the geometric lottery has become a version of The Red Queen’s Race: “It takes all the running you can do, to keep in the same place!”

The second group of "premature" entrants:

B. “Fatigued but Compulsive” Multi-Year Applicants. Even if a prospective runner is fully prepared and hungry to race when entering the lottery their first year, life happens.

As you can see in the data, there is always a drop off from year one to year two of entrants (that cannot be solely accounted for by being selected). But in those early years, the ticket growth is rather linear:

Year 1: 1 ticket
Year 2: 2 tickets

For runners who have applied in three consecutive years, the incentive to continue applying becomes more palpable:

Year 3: 4 tickets
Year 4: 8 tickets

But preparation and desire to run is neither linear, nor geometric. Sometimes it goes down: for various reasons, after several years of commitment, an entrant might lose either the physical or mental ability or willingness to enter. 

Yet the geometric ticket doubling continues, gaining real steam:

Year 5: 16 tickets

But life goes on, too: kids, family, work, injury illness. The new once-in-a-lifetime Lottery Bye provides a one-time year of respite from the grind of qualifying.

But once used up, multi-year applicants get stuck in a position of "accrue it or lose it": either continue to apply, year after year, or "crap out" and lose all of your "ticket earnings".

This fear of ticket bankruptcy creates another cohort of "premature entry": veteran lottery losers who are compelled to qualify, apply and accrue (often in very large, high-end geometric volume) during years in which they may not be willing or able to run. 

Some examples include top Masters runner (and friend) Bob Hearn:


WS veteran and timed-run national team member, Bob Hearn, upon entering the 2020 lottery: “I do not want to run it next year! But the lottery forces me to apply anyway”
I don’t fault Bob. He’s an accomplished runner with a lot of other goals and priorities. But his hand is forced: he either keeps applying (with hopes he “won’t get picked” this year, but maybe next), or give up and lose everything.

Preparation and desire are not linear. Besides injuries, illness, and the rest of Life, sometimes "qualification fatigue" -- the drain of having to plan for, gain entry to, and finish a Western States qualifier -- is too much and folks want a break:


A common refrain from first-timers, compelled to enter before they truly want to run: “Please don’t pick me”, followed a few years later by “qualifier-fatigue”
“Stopped caring”, but has to keep applying, anyway.

But should they give up, and give it all away? 

With no recourse other than to continue to apply until they either “win or bust”, “qualifer-fatigued” runners are compelled to trudge onward, even if his/her will or readiness to run has waned. It’s better to continue than to unceremoniously lose all the tickets you’ve worked to “earn”. 

Or, a runner incurs multiple illness or injury. Or multiple children. 

Or, the finite list of qualifiers – and their restrictive lotteries – shuts them out. 

After that Bye is used up, runners must let it ride each year and hope for the best. Or give up and lose it all. 

What becomes of the runners who finally “crap out”? This is another elusive data set. Who is talking to them?

Coaches like me are. The nearly decade long “win or bust” of the Western States lottery has soured a lot of runners on Western States. Having never experienced the race, they’ve given up. At best, they’ve simply let it go. At worst, they’re jaded and bitter. 

Precisely the opposite of what the race was intending to prevent.

(It’s worth noting that HR100 has unlimited lottery byes. This allows muti-time losers to take respite in any given year without losing tickets. But more importantly, it allows them to defer from a lottery when they don’t truly want to run that given year)

*****
The Purest Kind of Persistence is Un-Incentivized. A weighted system was intended to reward commitment and persistence to the effort of qualification and entry. And The Race has stated they wish to “reward persistence”: a characteristic worth reinforcing. 

But which type of behavior is more admirable?


A runner who applies year after year, incentivized with a “reward” of more (and more) tickets?
- or -
A runner who applies each year, with zero added incentive for repeat entry, motivated simply by the love of the race?


In a system stripped of artificial incentive, runners enter only when they’re all-in to run that year, knowing this year’s entry has no bearing on the next (or any subsequent) race year. 

One ticket, one year. No incentive, no reward, no punishment. Just be ready to race, that year. 

2. Create a stratified, multi-lottery system.

According to the mission of the race, WSER wishes to create a quality event that caters to a wide variety of runners, ranging from first-time hundred milers, to elite international runners, to veteran multi-time (10-Year Buckle) finishers. 

A stratified lottery consisting of various sub-categories could distribute entries to these desired groups, without distortion from an incentivized, geometric system. Like other races (including HR100), WSER could adopt the following sub-categories of prospective entrants: 


Never-Started. No WSER starts. (This is a notable distinction -- as opposed to finishes. See below). 

Why: of course, the event wants to include first-time runners - whose energy and excitement provide the fuel for a special day. That said, the race also does not want to be flooded by the sheer volume of first-time applicants. 

Veterans. Runners who have finished Western States multiple times. This could be five or more (as in the HR100 system), or, perhaps as many as eight finishes. 

Why: Valuing veteran finishers is an implicit mission of the race. Who, exactly, they value and encourage to continue running is up for debate, and at the discretion of the Board. Under the current geometric plan, a notable and venerated group, the "Ten Year Bucklers" will become extinct. Unless you're an elite runner (gaining entry via a "Golden Ticket" qualifier, or previous Top Ten finish), or gain entry through repetitive sponsor, volunteer, or other Special Consideration entries, it will become exceedingly difficult to win the lottery enough times to gain ten starts. 

Super-Masters. Over the age of 50. (No starts, or less finishes than a Veteran)

Why: This category recognizes that Super-Masters runners can run strong and compete well. This senior cohort also adds a degree of intrigue and magic to Western States. It also acknowledges that 50-plus year old runners simply cannot wait the five-to-ten years the current system demands.  

Everyone Else. This group has run the race at least once, but less than Veterans (and is younger than Super-Masters). 

Why: This cohort comprises all runners that have previously started Western States, yet does not have enough finishes to be considered a Veteran. 

Benefits of a stratified System: 

Consistent and pre-determined race diversity. One of the earliest concerns that prompted the new weighted lottery system was that new, first-time runners would overwhelm the lottery and crowd out the veteran runners.

Unlike a weighted system, where even multi-year entrants are drowned out by the glut of extreme ticket volumes, a stratified system protects the various cohorts of runners: race composition can be set by race leadership. Like Hardrock, Western States could appropriate a set number of entries per category, and run separate "mini-lotteries" from these groups. Doing so would allow each year's event to have a desired distribution of runners.  

Moreover, this composition could be dynamic and based on the unique composition of the entrant pool from year to year. For example, if there happen to be very few Super-Masters entrants, those entries could be shifted to a different sub-lottery. 

Such a system could also be employed to keep lottery odds consistent: entries could possibly shift from one cohort (say, Veterans) to another (Never-Starters) should demand in the latter increase far more than the former. 

Values runners’ Experience instead of “Tickets”. In a situation with such extremes of supply-and-demand, more thought should go into what types of runners should be prioritized for such limited entries. 

The current system is based solely on “tickets”: based solely on the number of years a runner has applied since his/her last race. And in this system, a single-ticket Never Starter entrant is on equal footing as a single-ticket runner with, say, three starts (but not quite a “veteran”, by number or character). 

The entry system for Western States would be far more equitable in a stratified lottery system based on both number of runs, and age, versus an arbitrary ticket count.

This is possible in a stratified system without making the overall entry standards – either by qualifier options, distances, or speed – increasingly stringent. Separate sub-lotteries preserve opportunities for these different groups. 

Indeed: it's impossible to award every interested runner an entry to Western States. However, the Board must decide the type of entrant it wishes to prioritize: First-Timers, or runners on their third or fourth run? Having already experienced Western States, why place a runner with several finishes equally amongst the Never Starter? This applies to any situation where a (1-to-4-start) “veteran” runner has the same number of tickets as a first-timer. 

That's for the Board to decide, but it would seem to me, if the goal of Western States is to capitalize on the energy of new runners and their fans, while retaining the wisdom and magic of the veterans, the Board should adopt a "Barbell Strategy" that prioritizes the extremes: 
  • Never Started (0 starts)
  • Veterans (at least X finishes)
  • Super-Masters
while giving relatively less priority to Everyone Else (1-X finishes, but at least one start). 

Premature starts (& DNFs) become more costly. An interesting facet of a stratified system is it allows the race to differentiate from a Never-Starter and Everyone Else. 

In the current system, there is no differentiation between tickets and experience. That also includes number of starts versus number of actual finishes. (While some websites such as ultrasignup.com track Did-Not-Finishes, I don’t’ believe Western States compiles this data in their lottery repository)

Note the distinction between “Never-Starter” (never run the race) versus “Never Finisher (X number of starts, but a possible DNF): A first-time runner who races, but fails to finish, would now transition to the Everyone Else cohort. 
*****

Why This Plan.
  • It is simple. It reverts to a basic lottery format.
  • It is discerning. A stratified system allows a diverse, balanced entry field based on experience (not consecutive qualification) that guarantees no single cohort of runners will overrun race entries. First-timers, veterans, and - for the first time, older runners - will be prioritized and protected groups under this plan. 
  • It removes complexity and second-order effects. This system eliminates premature entry and in doing so, should significantly decrease the number of annual lottery applicants. Community wide, it frees runners from the bondage of perennial qualification, and takes tremendous second-order strain off qualifying races. A runner enters the lottery only when s/he is most interested. This is the way it should be.
With this system, future Western States races regain an intangible quality characteristic of the first three decades of its history: a totality of qualified entrants who are unquestionably ready and passionate to run that year.

Why Other Ideas Won’t Work (or Won’t Be Heard)

Over the past several years,  I’ve heard a lot of different ideas about reforms to the Western States entry system. The vast majority of these ideas can be categorized in the following groups, and rather summarily dismissed:
1. Linear Solutions to a Geometric Problem. These ideas try to apply a “linear solution” - only a small number of increased entries - to a problem driven by a large (and geometrically-growing) demand.  
2. Cultural/Philosophical Changes. These ideas would potentially result in a substantial and pervasive cultural/philosophical shift away from the current, historic, Western States as we know it.
3. Exclusivity/Anti-Egalitarian. Similar to the cultural changes, these proposals invariably increase exclusivity and strip away egalitarianism, a core value of the race. 
4. Increased Complexity. These ideas add even more complexity to an already over-complex system. The more complexity, the harder it is to control, let alone anticipate unintended consequences. Thus any such idea will only make things worse. 

Examples include:

Argument: “Make the race bigger [somehow]” / “Have [multiple] races”
Category: Linear Solution, Cultural/Philosophical, Exclusivity/Anti-Egalitarian, Increased Complexity

This is the most often-cited - but extremely low-probability - solution. Most who make this argument fail to realize the origins of the 369 entrant limit. 

Accepting that, the second-most common suggestion it to have multiple editions of the race: one for elite runners and one for everyone else. Assuming permits could actually be obtained for two races (highly unlikely), sequestering elite athletes from the every-day runners would be a tremendous cultural sea change: increasing exclusivity and losing the egalitarian purity and powerful magic that is “Everyone from Squaw to Auburn” on the same day. Such drastic change is one of the least likely to ever occur. 

Lastly, even if the race could expand, it would need to expand at least one order of magnitude to make a palpable dent in current demand (“Linear Solution”). And if it did? Said “magic” hardly ever scales. 

Argument: “We need to do away with [insert non-lottery entrant group: sponsors, aid stations, elites, Special Consideration, international]”
Category: Linear Solution, Cultural/Philosophical

For one, the race now has over 6600 interested and qualified applicants. While the non-lottery entries number close to a hundred, a mere hundred entries is spitting into the wind when talking about 6600 entrants, and growing. 

Secondly, the race values the diversity that different entry methods bring. Separate arguments can (and should) be made for each of these non-lottery entrant groups, but I will refrain from doing so, here. 

Argument: “We need to make it harder to enter by [insert more challenging entry standard: race distance, time qualification, entry fee]”
Category: Linear Solution, Cultural/Philosophical, Exclusivity/Anti-Egalitarian

Simply making it tougher to qualify or otherwise enter won’t stem the demand. It may create a temporary dent (as did tightening qualifiers in 2013-2014), but it would do very little to sustainably curb the demand that a geometric system generates unless taken to an extreme. 

If taken to an extreme, it would create a powerful - and likely negative - cultural and anti-egalitarian impact (Charge $5000? Only sub-24 runners?). In doing so, you risk decreasing relevance of the race, and increasing negativity and bitterness. 

Runners of all abilities running from Squaw to Auburn is a big part of the Western States magic. Any “make it harder” strategy decreases race diversity and detracts from that magic. 

Argument: “How about two-to-the [insert even bigger number]?” The “Even More Tickets Argument”
Category: Increased Complexity. 

Too many tickets - and the complex second-order effects that came with them - is what got us here. More tickets won’t make it better; only worse. 
*****
Transition Plan.

There’s no easy way to transition out of the weighted system. But the potentially least-painful way would include:

1. Immediate cessation of all ticket compounding. While no more tickets could be accrued, all runners who had previously accrued more than one ticket would retain them until:
  • they are selected, or 
  • they fail to re-qualify (having already used the Lottery Bye). 
In essence, in order to be fair and minimally painful, the old ticket and qualifying system - including the requirement for consecutive qualification - would have to continue until all the multiple-ticket applicants are processed through.

For example:
  • A runner with 64 tickets and no Western States starts would retain his/her 64 tickets and enter the “Never Started” sub-lottery.
  • A runner with 4 tickets and one Western States finish would retain his/her 4 tickets and enter the “Everyone Else” sub-lottery
Of note: previous circumventions employed for lingering high-ticket entrants - such as giving away (in-person lottery or raffle) entries, or adding them to the top of a waitlist - will no longer occur. 

2. Immediate implementation of the stratified system. Regardless of ticket numbers, runners would be sequestered into their applicable cohorts. Thus, a 64-ticket “Never-Starter” is no longer competing against an “Everyone Else” with the same ticket quantity. This immediately improves equity, as the preferred cohorts will have relatively better (pre-set) odds, while “Everyone Else” will have similarly worse odds. 

3. All new applicants receive only one ticket. In this transition plan, the new applicants will unfortunately bear the brunt of absorbing the impact of the multi-ticket applicants. In year one, there will be no difference (as in the current system, first-time applicants get only one ticket). Second- and third-year applicants will no longer receive the weighted “benefit” of additional tickets.


*****
Conclusions.

There is no perfect solution, including this proposed alternative. Pure equity and fairness cannot exist in a situation with such a stark difference between supply and demand, and so many competing notions of who is most deserving for inclusion.

The first step toward the most coherent solution is a community-wide acknowledgement and acceptance of the following points:
1. Not every interested runner will be able to run the Western States 100, even once. Many will never get a chance to run.2. “Deserving runners” will be left out, and “less-deserving” runners may get to run (multiple times).
Once accepted, we can arrive at a solution that - while not necessarily “fair” (by the standards above) - is still simple, coherent, and - most importantly - free of far-reaching consequences and unintended side-effects.

As such, a third point must also be acknowledged:

3. “Persistence” - measured through consecutive application to the race lottery - cannot be incentivized 

Persistence is still rewarded, but without incentive to continue, nor punishment for stopping. In its place is increased optionality and freedom: runners can apply as many times as they want, as consecutively as they want, without invoking complex second-order effects that ripple throughout the ultrarunning community. Over time, as with the first three decades of the lottery, persistence will eventually reward them with entry, over time. 

It seems pretty simple, but - finally - a runner can simply run a race because they truly want to run that race. Including Western States. 

Dance with the one that brung you. 

Lastly:

In this proposal, a first-time runner - prioritized in the Never-Started cohort, in a lottery not over-run by premature entries - has the best chance to quickly run his/her first Western States. 

But after that, compared to the current system, will get much more difficult to get back in. One-and-done will be the most common thread connecting Western States runners. 


But that gives that one-timer - hopefully armed with a shiny buckle - an opportunity to give back: to spread that love and Western States magic and experience the race from all facets as volunteer, crew or pacer. Perhaps through those other avenues - volunteering, sponsors, or even elite performance - they earn enough finishes to be a Veteran. 


Best of all, it accomplishes the three-pronged objectives that motivated the original lottery idea:
  • to expose fresh blood to this iconic event
  • to reward and recognize veteran runners for what they have to offer
But above all, most importantly:
  • to create generation after generation of people - not just runners, but his/her friends and family  - that are positively engaged with the race, and thus become, in perpetuity, part of the Western States Family
Thank you for reading, and I look forward to any ongoing discourse that promotes the perpetual growth and improvement of this truly special event.

Tuesday, December 10, 2019

Western States 100 Has a Problem - Part II: The Non-Problem?

Before we begin, an important note:

A good friend of mine has told me,



"You have a solution to a non-problem."

He wasn't been contrarian, but facetious. And factual.

His point was that the race leadership does not believe it has a problem. 


And he is correct. They do not think they have a problem. And here is why:


They believe that interest in the Western States 100 - and the radical increase in total lottery applicants - is truly organic. 


I disagree, and so do the numbers. Since 2010:


This means the growth in Western States entry is closing in on double the overall growth in the sport (which, when broken down, is largely driven by shorter-ultra finishes, not 100-milers).

Is Western States truly increased in popularity and general appeal, versus the sport at large?

Or could there be something else (artificially) driving up those entries?

They believe that premature entry is an anecdotal problem: "several people" applying "just to pile up tickets", but not a pervaise issue.

I disagree. There are potentially hundreds - if not thousands - of Western States lottery entrants - namely the first-year/single-ticket runners - that are getting in line, but don't really want to run this year.'


And why wouldn't they?


  • A geometric system rewards premature entry
  • A geometric system, filled with premature entrants, "piling tickets", punishes patience: waiting until you're truly interested and ready

Below are the single-ticket odds by (race) year:

2014: 5.6%

2015: 4.7
2016: 4.5
2017: 2.5
2018: 2.3
2019: 1.7
2020: 1.3

Why would anyone chose patience, when the odds of entry continue to drop like a stone? 


  • Premature entry is driving down the first year odds.
  • Dwindling first (and all)-year odds is driving more premature entry.

We're caught in a vortex.

It's like a bank panic: if you want any chance, you better get in line right now. 


True, posts like these are anecdotal:





But this?



103 Respondents! And 47% percent don't really want to run this year! 

I acknowledge this is not a scientific poll (not perfectly selecting from verified entrants, as my Twitter followers are more likely to rouse rabble than the general applicant pool).


But forty-seven percent, 48 of 103 respondents, don't really want to run this year?

What is the acceptable percentage of lottery entrants that don't really want to run this year? Surely it is under 47%, or even 17%. 


The counter argument: while begrudingly acknowledging that this is a legitimate concern, the Board wishes to point out data like this:

  • single-ticket lottery winners finish at a higher rate than the overall finishing rate (76% vs 67%, in 2016, for example).
But this answer is a substitution for another question, which is:


If you're offered an entry in the race (regardless of ability and interest), will you finish it? 

The numbers indicate they will. But that doesn't mean they were fully commited and prepared, compared to the rest of the 90%+ lottery losers. 

It's like being offered a free dessert - a highly coveted, rare treat - and still eating (and finishing it), regardless of hunger. Will you turn down something that you might never get again?


Lastly:


The race leadership does not seem to acknowledge the negative second-order effects on the rest of the sport. 


A corrollary: they do "make a conscious decision to reward persistence". But on whose backs does the weight of that persistence fall? 


Qualifying standards have become tougher; races decreased (since the 50-mile qualifier was abolished). Most qualifying races, themselves, now have lotteries. And many runners chose to enter events for the sole purpose of "getting a qualifier". 


Per my letter to the Board:

The WSER qualifiers face mounting pressure (in both quantity and fervor) from runners regarding, among other things, entry into the qualifier race and finishing in the prescribed standard. Failure to do either places enormous (and increasing) strain on RDs and events, who must service a runner who's sole desire is to "run another race [WSER] other than this one". 
It also has palpable - and pervaise - effects on the yearly planning (and career trajectory) of runners, their coaches, and their friends and families:
Quite simply, people are forced, because of lottery strain, to base his/her entire yearly race schedule on "getting a qualifier". As opposed to running events organically, for their own sake -- because they're unique and desirable in their own right -- great pressure is put on runners, their coaches (and family), and race events to cater to this geometrically increasing need to finish a WSER lottery qualifying race. Indeed, the more "tickets" - and the older the runner - the higher pressure there is to register for and finish a qualifier. 
This is against the spirit of ultrarunning. Runners should enter, run and finish races, in order to run that race - not simply for the opportunity to run another race. "Dance with the one that brung you" -- don't abuse one event, simply to climb a rung closer to another. 
*****
In writing these posts, I've been criticized of (among many things) "sour grapes" - attacking the thing that I can no longer have (or, it seems, run fast enough, or finish frequently enough, to qualify to make any critique). Moreover, I've been accused of lobbing incendiary complaints into the race community without offering a solution.

And that's OK. I accept counter-criticism, and questioning of motives.



But a solution is worthless without accepting there is a problem.

This was the intent of Parts I and II: to help identify that, indeed, there is a problem. And it's only getting worse.

And it really isn't just a "numbers problem". It is an Existential Crisis:

Any sized cohort of entrants who "really don't want to run it, this year" is poisonous for the spirit of the race: like showing up to a bountiful meal, with bellies already full. (While starving folks stand, outside-looking-in)
I could go on - and will - about the issues and implications, but I will now turn the attention toward a solution:

a non-weighted, stratified lottery system.

The weighted system must go.

Thanks for reading and discussing, and please stay tuned. 

Saturday, December 7, 2019

The Western States 100 Has a Problem: Part I - Introduction

Introduction


It has nothing to do with the course, the weather, the aid stations, the volunteers, the runners, or the leadership.



It has a math problem.


And it's slowly sinking in it. And we're all sinking with it.

The weighted lottery system - designed and intended to reward committed serial entrants with a faster and (seemingly) guaranteed entry - has not only failed to deliver on that intention, but it has unwittingly accelerated the already steep demand, and thus made it exceedingly more difficult to obtain that precious commodity: the Western States entry. 


Brief Background


The compounding system, first implemented decades ago, was a well-intended system intended to reward runners who, failing to gain entry in their first year, would be guaranteed an entry the second year. The "Two-Time Loser" system rewarded the persistence of consecutive entry and functioned for many years without issue.


But that was a different time: before the proliferation of trail racing and, more notably, before the internet (and all modern technology, including online news, websites, blogs and documentary videos) "virally" spread worldwide the magic dust of what Western States was all about. 


Thus, by around 2010, the Two-Time-Loser system was inundated with so many runners that it could no longer guarantee an entry, even two years on. Something had to change. In 2011, a linear, compounding system was adopted: giving runners an additional "ticket" for each consecutive year of entry. 


This "n+1" system*  was promising, but the race entries grew even faster, and diluted the lottery even more. 


(where, in this case, n = number of years of entering the lottery and not being selected)


At some point, elements of race leadership realized that this was a problem. Based on my personal discussions with various board members and race administration, those concerns were two-fold:


  • One, they wanted to afford any interested runner the opportunity to run the race, someday.
  • Two, they did not want members of the ultrarunning community, frustrated with repeated failure, to become embittered and develop a negative attitude about the event.

Thus, to address those concerns, and reward due diligence and commitment to multiple years of serial application, along came the Geometric System. 

In 2012 or so, the geometric system was implemented. The equation, 2n-1 was used to more quickly - geometrically - grow ticket counts. While in the previous system, a five-year entrant would only have five tickets, in this new geometric system, that person would have 16 tickets. Thus, compared to a first-year entrant, this five-year lottery veteran has far better odds, and those committed runners would all eventually get selected.


Right?


A Geometric Solution Becomes a Geometric Problem

The eventual mathematical reality of a geometric system was somewhat difficult to anticipate when it was first implemented eight years ago. (And admittedly - it just might take a math degree to realize that, no matter how many years one applies, the odds will never, ever, reach 100%: the literal definition of an asymptote). 


But eight years on? People across the sport are beginning to realize a few things:


First, a weighted system does not "clear" runners. It only partially increases some odds, while diluting others. And no matter how high the odds, there will never be a "sure-thing" entry, whether a runner qualifies ten years in a row, or a hundred.


Second, the weighted system forces runners to enter before they're truly willing (let alone ready) to race. With each passing year, an increasing proportion of runners enter because they "want to run Western States someday, but not really this year". 


Why? Because with each passing year, the single-ticket (first-year) odds get lower and lower. 


Third, the converse: a weighted system punishes patience. Runners who would rather wait a year (or two, or five) to run Western States feel compelled to enter now, knowing that it may take three, five or even eight-plus years to get an entry. If then! 


Why? Because such a rewarding system is artificially increasing applicant growth - at this point, far out-stripping the overall growth in ultrarunning and 100+-miler participation. So with each passing year, the geometric growth of the ticket pool drowns out that measily single ticket. 


You're a fool to not enter "early and often", as they say. 


But these realizations are just the beginning. There are deeper effects that ripple across the landscape of the sport of ultrarunning: how we chose to train and race, how we choose to approach different race events, and indeed the sort of personal - familial and even medical - decisions runners must make to keep The Dream alive. 



But it doesn't have to be this way. 

But to fix it will take a leap of faith. 

We'll all have to take our turn. 


Settle down. Chill out. BE PATIENT. 


And enter the lottery only when we are truly willing and ready to race from Squaw Valley to Auburn. 


*****

Over the coming days, I intend to outline the various issues with the current lottery system, the consequences (many of them both negative and unintended) and impacts across the landscape of ultrarunning, and some personal anecdotes from me: a runner, a coach, a medical professional, and - more than anything - a true lover of this event, its people, and everything it stands for. 


Part I: Diving In

(The following was an analysis I prepared for and presented to several members of the Western States Board of Directors in the fall of 2017. 

Somewhere around that time it finally dawned on me the true and pervaisive consequenses of the weighted lottery. As both a runner and coach, I was worried. But as someone who's lived and breathed Western States for going on a decade, I was deeply concerned. So I gathered my thoughts and presented them to The Board in October 2017. 

The following text is verbatim from that message. Any additions are indented and [bracketed] for enhanced clarity:)


Introduction. This analysis and proposal is based on the current, incentivized geometric lottery model for Western States. At first glance, such a method was devised to "add fairness" to a lottery system where an increasing number of interested runners enter, for a frustratingly (and stangantly) small number of entries.

However, such a incentivized lottery has only made the situation worse. It's created it's own geometric rise in lottery entrants, based on the fact that entering prematurely is rewarded, while patience - waiting only until the year in which one is ready to race - is punshed, with ever-dwindling first-year ("one ticket") odds. 

Below is my analysis, followed by proposal for lottery reform. 

Problems with the current lottery system. Right now there are at least three problems with the WSER geometric (2n-1  "ticket"/year) lottery system: 

1. Incentivizes premature entry & punishes patience. This is the #1 worst problem of the WSER lottery: it gives incentive to people to prematurely enter the lottery. This mindset is pervaisive: "I need to start now...".  A very large (too large, tragically large) proportion of lotto entrants -- and thus selected runners -- enter, knowing full well they do not wish (or are not fully prepared) to race this upcoming year. However, not only are they incentivized to enter -- more years, more tickets -- but they are punished for patience. This is based on the data that suggests that with each passing year, odds for those with a single ticket (first-year applicants) decrease. This is punishment for waiting. 

2. Second-order effects. There are powerful, and overwhelmingly negative, second-order effects of the WSER lottery on the entire sport of ultrarunning. These include:

Increasing pressure on qualifier races. The WSER qualifiers face mounting pressure (in both quantity and fervor) from runners regarding, among other things, entry into the qualifier race and finishing in the prescribed standard. Failure to do either places enormous (and increasing) strain on RDs and events, who must service a runner who's sole desire is to "run another race [WSER] other than this one".  

Altering the annual landscape of ultramarathon participation. Quite simply, people are forced, because of lottery strain, to base his/her entire yearly race schedule on "getting a qualifier". As opposed to running events organically, for their own sake -- because they're unique and desirable in their own right -- great pressure is put on runners, their coaches (and family), and race events to cater to this geometrically increasing need to finish a WSER lottery qualifying race. Indeed, the more "tickets" - and the older the runner - the higher pressure there is to register for and finish a qualifier. 

This is against the spirit of ultrarunning. Runners should enter, run and finish races, in order to run that race - not simply for the opportunity to run another race. "Dance with the one that brung you" -- don't abuse one event, simply to climb a rung closer to another. 

3. Mathematics of Critical Mass. I recall RD Craig Thornley say, repeatedly, at last year's [2016] lottery at Placer High School say, "Just keep qualifying, and you'll get in!"  The problem with that statement is, eventually, it is false.

Without delving into the complexities of geometric math, if the race continues to grow in both individual entrants (which it has) and consecutive entry lottery tickets (1, 2, 4, 8, 16, 64, 128....ad nauseum), the numbers of people with higher-and-higher-ticket counts will grow to the point where a runner - even with, say, 128 tickets - will simply not get picked. 

In 2016, we faced that situation for the first time and barely escaped: 
  • 2 runners had 64-tickets (7 years of entry). Both runners gained entry: one was selected, and the other was "given" one of the five remaining "in-person" lottery spots. (While generous, this is obviously an unsustainable strategy)
However, 
  • 31 runners had 32 tickets (6 years of entry). Only 18 gained entry, and in order to "clear them" from the system, the remaining 13 had to be placed on the Waiting List -- a new (and also incentivizing) feature of this year's race. To my knowledge, most were picked (or otherwise waived their entry for this year). 

Great. But soon (within perhaps 1-2 years), those "long-waiting" applicants: >6 years or more, will completely fill the waitlist. Then what? Reserve it only for 7 year-losers? 8 years? 10 years? How long is the acceptable wait time? What degree of frustration and resentment is WSER OK with? 

[Of note, there were 9 eight-year applicants in the 2019 lottery, and all but one were selected. The Ninth was added atop the waitlist. This represents a mathetmatical cheat that, as I will discuss later, is both unfair and unsustainable. 
[Another note: 38 of 54 seven-year entrants and 88 of 126 six-year entrants were not selected this year. Within two years, the race will have surpassed a ten-year wait. WIth no end in sight]

If you're interested in some (nauseating) math, here is a fantastic analysis by Ryan Witko on the issues and challenges of the WSER lottery. I have my doubts that most WSER board members have read and understood this analysis (I have 3.5 college degrees and have taken both upper level calculus, statistics and even quantum mechanics, yet have a hard time fully absorbing it!). But the conclusions are thus:
  • both entrants and ticket counts are increasing geometrically
  • the lottery cannot "clear" all of the runners from the system before they they fail to re-qualify (either due to physical incapacitation, or - even worse - failure to gain entry to a qualifying race!).
    • An increasing number runners -- many of whom that have "queued patiently" -- qualifying and entering the lottery, year after year, per Mr. Thornley's recommendation -- will not ever be selected. 
    • This is the worst-case PR scenario for the race: runners "guaranteed" to eventually get in by the system, only to slip up one time (after, say 8-10 years of consecutive entry).
  • In short, per My Witko's analysis: "Assuming recent trends continue, we will soon reach a point where most applicants in the lottery will never actually be selected."  
In conclusion, the geometric rise in both runners and tickets are on a collision course for failure. However, that's only the half of it. That a large proportion of runners in a given year actually do not want to (or are otherwise not ready) takes a bad situation and makes it tragic, and utterly frustrating for those who are.  

Recommendations. There are only three factors that can impact the demand to run Western States:
  1. Field size. Permitting mandates a finite size. It's generally understood that the Board is working to somehow expand this. But even if the field grew to 2,000 -- that is still a very small, non-geometric increase, relative to the following two factors
  2. Lottery entrants. The total number of people interested in entering the lottery in a given year. 
  3. Lottery tickets. The total number of tickets in a yearly lottery hopper. The current 2^n-1 formula creates a geometric growth pattern of total lottery tickets.
#1 is currently a rigid figure. For #2 and 3, there are but two options:

A. Decrease demand for individual runners to [prematurely] enter.
B. Decrease the number of tickets (in order to increase the value of a single ticket)
*****

That is all for Part I. Subsequent parts - including a proposed alternative to the current system - are forthcoming.

Stay tuned, and thanks for reading.