1. Produce a ranking

Of course, the ranking algorithm should produce a overall ranking of teams. Everybody agrees with this.

2. Consider the structure of tournaments (not every game worth the same)

To me the principal weekness of the actual USAU Ranking algorithm is that it doesn't see the structure of tournaments. The algorithm can be great for a league or whatever sport where games are worth the same but not for competitive Ultimate where games are played during tournaments. For the algorithm used by USAU, a tournament is just a bunch of random games. The algorithm does not see the difference between the saturday morning game and the sunday afternoon final (apart the one day difference which won't affect much).

The following paradox illustrates how the results of games in a tournament is completely independent of the outcome of the tournament itself. Four teams (A, B, C and D) play in a tournament. The format is a round robin followed by semi-finals, final and 3rd place game: 10 games all in all. The results are (in no particular order):

• game #0: (15) A - C (7)
• game #1: (15) B - D (6)
• game #2: (15) A - D (8)
• game #3: (15) B - C (10)
• game #4: (15) A - B (12)
• game #5: (15) C - D (9)
• game #6: (12) A - D (15)
• game #7: (7) B - C (15)
• game #8: (8) A - B (15)
• game #9: (9) C - D (15)

From this information only, can you decide which team won the tournament? The answer is NO. If games #0, #1, #2, #3, #4, #5 are pool play games, then the seeding out of pool play is A(3-0), B(2-1), C(1-2), D(0-3). Games #6 and #7 are the semis, game #8 is the 3rd place game and game #9 is the final. Team D win the tournament. Final standing is D-C-B-A.

If games #0,#1,#6,#7,#8,#9 are played in the pool, then the pool outcome becomes B(2-1), D(2-1), A(1-2), C(1-2). Team B and C play in semi final game #3 while Team A and D play their semi in game #2. The final game is game #4. The 3rd place is game #5. Team A win the tournament. Final standing is A-B-C-D.

After the tournament, we can discuss about what should be the proper ranking of those four teams based on the results of the 10 games they played. But, there are good chances that we are just wrong as the above paradox shows. In a tournament all games are not equal. It is all about the context. A good ranking algorithm should consider the format of tournaments and accept that some games worth more than others. How to do this? The actual algorithm is already very complicated. How to put more value to particular games? Maybe possible, but may be complicated as well.

I believe there is an easy solution to this. The algorithm must take a step back. Looking at games is like looking at the microscope: we don't see the global picture. Which team get the trophy at the end of the tournament in front of everyone without contestation from anybody? The trophy is given at the end of the tournament to the team who reached the final and won it. This is the best team. The best team is not the one who got the best overall game results. This is why I believe ranking algorithm in competitive Ultimate should be based on the outcome of tournaments and not on the outcome of games.

3. Be predictable

In a text by Lou Burruss about ranking published during 2012, the author speak about the recursive aspect of the actual ranking algorithm making it unpredictable:

"In conversation with Sholom, he made clear there is still some mystery to this very complex process. Even after 20 years of official rankings and simulations, it is impossible to predict exactly how things will behave."

On the contrary, if the ranking system is predictable, then teams know in advance that winning a certain game will give them exactly 200 pts, let's say, and this is the amount they need to become #1 in the country, then it will be one more motivation to win the game.

This is when ranking is not anymore only about ranking. Ranking can also be a source of motivation for team to improve. But ranking will do this at its best when it is predictable. This is why I believe USA Ultimate should look into this if they want to enhance the level of competition as they claim:

"In the ongoing effort to enhance the level of competition and achieve organizational goals, USA Ultimate continues to look to improve the ranking algorithm [...]."

4. Reward what is the most valuable (winning when it counts)

In my opinion, ranking should reward what is the most valuable by the community. This should correspond to what is the most difficult to achieve. Again, this point is a necessary condition for the ranking to be used as a source of motivation for team to improve.

If a ranking values something else, then people won't value the ranking and team won't try to improve themselve to get a better ranking. For example, think about a ranking that would only consider the best differential. We all know that the best differential in a tournament can be reached by the team G who finishes 7th out of 16 teams. The team G will think they are the best in the country and they won't try to win a tournament : too much effort and useless. The team A who put all the effort to get to the final and win it will wonder if they are optimizing the good thing. They will be less motivated to win another tournament.

Naturally, the community values teams getting in the final and winning it. This is what I call the natural attractor. The ranking algorithm chosen also defines a ranking attractor. As we have seen in the last paragraph, the two attractors can be different. But, I strongly believe the best choice of ranking algorithm is when syzygy (picture here) happens, that is when the ranking attractor equals the natural attractor.

"The syzygy produces the more powerful spring tide due to the enhanced gravitational effect of the Sun added to the Moon's gravitational pull."

In this situation, the natural attractors and the ranking attractor double their forces and team then really wants to win tournaments because they get the natural recognition from the community plus they get an high rank in the official ranking. This is the best way to optimize the improvement of teams.

It is known that the actual ranking algorithm encourages teams to play a bit differently than they would if it was just about winning tournaments. In Maximizing Your Team’s Ranking: Strength of Schedule or Margin of Victory? (Dec. 2013), Scott Schriner confirms this. He wrote:

"To increase your game performance score, you will want to win more games — but also win each game by as large of a margin as possible. This might include playing your starters longer than necessary, or loading up your D Line with an O Line handler even if you don’t need that break. Of course that decision carries with it other tradeoffs: you may increase injury risk or stall the development of younger players on your team."

I believe it is a mark of respect for tournament structure that the ranking attractor correspond to the the natural attractor. Moreover, this system is trusting teams and let them all the liberty they need to win a tournament. Good team will peak at the good moment and will try new strategies or give more playing time to rookies at other moment. Teams that try to optimize every point at every game do not improve and do not push the game further.

A ranking system for competitive Ultimate

There is a important observation to be made when creating overall ranking in competitive ultimate. The output of a tournament is a ranking of teams. Why not build on this to create overall ranking? A natural system is to give points to teams according to the final ranking. In such a system, teams needs to win tournaments to get the maximal amount of points.

I think a system like tennis ATP and WTA ranking is a good inspiration for competitive ultimate. Tennis players go to tournaments they choose. Some tournaments worth more (Grand Slam) some less (ATP World Tour Masters 1000, ATP 500, ATP 250). The number of points of a tennis player is the sum of its best 18 results.

The problem with tennis compared to Ultimate is that there are no placement games. The outcome of a tennis tournament is that you either win the final, made the final, made the semis, made the quarters, made the round of 16, and so on. In Ultimate, there are placement games and finishing 9th is better than finishing 16th. Also, 18 tournaments is a lot. We need to consider less tournaments.

Since 2007, in Quebec we adapted the ATP ranking to Ultimate tournaments with placement games. In 2011, the series became so popular. We needed to adjust the system (88 teams took part in the Mars Attaque 2011 but only 32 got points). Since 2011, 1000 pts is given to the winner of a Grand Slam event, 938 pts is given to the finalist, 884 pts is given to the 3rd place, 835 pts to the 4th place and this goes until the team finishing in 50th place who obtain 1pt. The number of points is obtained from the integral of a logarithmic function. I am going to write another text on my blog where I explain the details and the math behind the system.

Example : ranking of USA Open Club team during 2013 regular season

As an example I am considering the 2013 USA Club Open seaon. I selected the following list of 14 tournaments. Those tournaments are divided into 2 groups (7 tournaments of 500 points, 7 elite tournaments of 1000 points).

Below is the resulting ranking made from the 2013 USA Club Open season with the parameters chosen above. Comments are welcome.

  Pos   Pts    Team name                     Rg   Best                         2nd best                     3rd best
+-----+------+-----------------------------+----+----------------------------+----------------------------+----------------------------+
  1     2746    Doublewide                   SC   1 (1000) Colorado Cup        2 (911) Pro Flight Finale    3 (835) US Open
  2     2670    PoNY                         NE   1 (1000) Club Terminus       3 (835) Colorado Cup         3 (835) Chesapeake Invite
  3     2619    Revolver                     SW   1 (1000) US Open             2 (911) West Coast Cup       5 (708) Pro Flight Finale
  4     2603    Sockeye                      NW   1 (1000) West Coast Cup      3 (835) Club Terminus        4 (768) Pro Flight Finale
  5     2603    Machine                      GL   1 (1000) Heavyweights        3 (835) Pro Flight Finale    4 (768) Club Terminus
  6     2387    Johnny Bravo                 SC   2 (911) Colorado Cup         4 (768) West Coast Cup       5 (708) Club Terminus
  7     2377    Ironside                     NE   2 (911) US Open              2 (911) Chesapeake Invite    8 (555) Pro Flight Finale
  8     2361    GOAT                         NE   1 (1000) Pro Flight Finale   5 (708) Chesapeake Invite    6 (653) Club Terminus
  9     2166    Chain Lightning              SE   2 (911) Club Terminus        6 (653) Chesapeake Invite    7 (602) Pro Flight Finale
  10    2140    Sub Zero                     NC   1 (1000) Chesapeake Invite   5 (708) Heavyweights         11 (432) Club Terminus
  11    2043    Ring of Fire                 SE   3 (835) US Open              6 (653) Pro Flight Finale    8 (555) Chesapeake Invite
  12    1937    High Five                    GL   3 (835) Heavyweights         7 (602) Chesapeake Invite    1 (500) No Surf
  13    1927    Rhino                        NW   5 (708) Colorado Cup         5 (708) West Coast Cup       9 (511) Club Terminus
  14    1910    Madison Club                 NC   2 (911) Heavyweights         7 (602) Colorado Cup         12 (397) Club Terminus
  15    1881    Truck Stop                   MA   4 (768) Colorado Cup         7 (602) Club Terminus        9 (511) Chesapeake Invite
  16    1685    Inception                    SC   4 (768) Heavyweights         1 (500) Col. Cup noTCT       3 (417) San Diego Slammer
  17    1492    Streetgang                   SW   6 (653) Heavyweights         2 (455) San Diego Slammer    4 (384) Cal State
  18    1455    Chicago Club                 GL   5 (708) US Open              4 (384) Mot. Throwdown       13 (363) Heavyweights
  19    1340    LA Renegade                  SW   7 (602) Heavyweights         4 (384) Cal State            5 (354) San Diego Slammer
  20    1302    Condors                      SW   1 (500) Cal State            1 (500) San Diego Slammer    15 (302) Colorado Cup
  21    1284    Madcow                       GL   2 (455) Mot. Throwdown       11 (432) Colorado Cup        12 (397) Chesapeake Invite
  22    1252    Cash Crop                    SE   10 (471) Chesapeake Invite   12 (397) Colorado Cup        4 (384) Old Dominion Q.
  23    1087    Dire Wolf                    MA   2 (455) No Surf              4 (384) Cazenovia            17 (248) Heavyweights
  24    1085    Prairie Fire                 NC   6 (653) Colorado Cup         11 (432) Heavyweights        
  25    1011    Florida United               SE   9 (511) Colorado Cup         1 (500) Old Dominion Q.      
  26    1010    Boost Mobile                 SW   8 (555) Club Terminus        2 (455) Cal State            
  27    1007    Medicine Men                 MA   12 (397) Heavyweights        5 (354) Old Dominion Q.      9 (256) Cazenovia
  28    986     Mephisto                     NE   7 (602) US Open              4 (384) Cazenovia            
  29    888     Voodoo                       NW   10 (471) Colorado Cup        3 (417) San Diego Slammer    
  30    865     Sprawl                       SW   9 (511) Heavyweights         5 (354) San Diego Slammer    
  31    835     Buzz Bullets                      3 (835) West Coast Cup
  32    833     Garuda                       NE   8 (555) Heavyweights         8 (278) Cazenovia            
  33    768     Clapham                           4 (768) Chesapeake Invite
  34    686     Oaks                         SW   5 (354) Cal State            14 (332) Heavyweights        
  35    656     Castle                       NC   5 (354) Mot. Throwdown       15 (302) Heavyweights        
  36    653     Euforia                           6 (653) US Open
  37    632     Sheet Metal                       5 (354) No Surf              8 (278) Cazenovia
  38    594     Beachfront Property          GL   3 (417) Mot. Throwdown       20 (177) Heavyweights        
  39    575     Space City Ignite            SC   7 (301) San Diego Slammer    16 (274) Heavyweights        
  40    562     Chico                             6 (326) Cal State            10 (236) San Diego Slammer
  41    555     Plex                         SC   8 (555) Colorado Cup         
  42    555     Ragnarok                          8 (555) US Open
  43    534     Gridlock                     SW   8 (278) Cal State            9 (256) San Diego Slammer    
  44    500     Ulysse                            1 (500) Cazenovia
  45    471     Brickyard                    GL   10 (471) Heavyweights        
  46    471     Furious George               NW   10 (471) Club Terminus       
  47    467     Midnight Meat Train          GL   7 (301) No Surf              14 (166) Mot. Throwdown
  48    458     Lake Effect                  GL   7 (301) Mot. Throwdown       21 (157) Heavyweights        
  49    455     Tanasi                       SE   2 (455) Old Dominion Q.      
  50    455     Powderhogs                   NW   2 (455) Col. Cup noTCT       
  51    453     Madador                           10 (236) No Surf             11 (217) Mot. Throwdown
  52    435     BD Air Show                  SW   10 (236) Cal State           12 (199) San Diego Slammer
  53    432     Oakland                      MA   11 (432) Chesapeake Invite   
  54    417     Boneyard                          3 (417) Old Dominion Q.
  55    417     Choice City Hops                  3 (417) Col. Cup noTCT
  56    417     Madcow Y                          3 (417) No Surf
  57    405     Inception-Red                     4 (384) Col. Cup noTCT       30 (21) Heavyweights
  58    399     CAKti                             11 (217) No Surf             13 (182) Mot. Throwdown
  59    384     Madcow X                          4 (384) No Surf
  60    363     Phoenix                      NE   13 (363) Colorado Cup        
  61    354     Sweet Roll                   SC   5 (354) Col. Cup noTCT       
  62    354     The Nights Watch             NE   5 (354) Cazenovia            
  63    326     Grand Trunk                       6 (326) No Surf
  64    326     Vanier Wildcats                   6 (326) Cazenovia
  65    326     Floodwall                         6 (326) Old Dominion Q.
  66    326     Enigma                       GL   6 (326) Mot. Throwdown       
  67    301     The Ghosts                        7 (301) San Diego Slammer
  68    301     Swell                             7 (301) Old Dominion Q.
  69    278     Old Growth                        8 (278) Cal State
  70    278     Burnside                          8 (278) Old Dominion Q.
  71    278     Jurassic Shark               GL   8 (278) Mot. Throwdown       
  72    278     Maverick                          8 (278) No Surf
  73    274     Lancaster                    MA   16 (274) Colorado Cup        
  74    256     Journeymen                        9 (256) Cal State
  75    256     Grantham U.                       9 (256) No Surf
  76    256     VAlhalla                          9 (256) Old Dominion Q.
  77    256     Impulse                           9 (256) Mot. Throwdown
  78    237     Hustle                            10 (236) Mot. Throwdown      32 (1) Heavyweights
  79    236     Triforce                          10 (236) Old Dominion Q.
  80    236     Jester                            10 (236) Cazenovia
  81    231     Spoiler                           12 (199) Mot. Throwdown      29 (32) Heavyweights
  82    223     Haymaker                     GL   18 (223) Heavyweights        
  83    217     Brawl                        SW   11 (217) San Diego Slammer   
  84    199     ROY                               12 (199) No Surf
  85    199     Centretown Gunners                12 (199) Cazenovia
  86    199     Warriors of Rad                   12 (199) Cazenovia
  87    199     Gnarwhal                     NC   19 (199) Heavyweights        
  88    194     INfamous                          16 (137) Mot. Throwdown      27 (57) Heavyweights
  89    182     Stonefish                    NE   13 (182) Cazenovia           
  90    166     Youngbloods                  NE   14 (166) Cazenovia           
  91    151     Flying Pig                        15 (151) Mot. Throwdown
  92    137     Freaks Uv Nature             SE   22 (137) Heavyweights        
  93    137     Firebird                          16 (137) Cazenovia
  94    137     Throw'n Together                  16 (137) Cazenovia
  95    134     Illusion                     NC   17 (124) Mot. Throwdown      31 (10) Heavyweights         
  96    119     Mufasa                            23 (119) Heavyweights
  97    112     MicroMachines                     18 (112) Mot. Throwdown
  98    102     Mad Men                           24 (102) Heavyweights
  99    100     Rust Belt War Bonds               19 (100) Mot. Throwdown
  100   89      Salvage 3                         20 (89) Mot. Throwdown
  101   86      H1N1                         NC   25 (86) Heavyweights         
  102   71      yogosbo                           26 (71) Heavyweights
  103   44      City Park Ultimate           NC   28 (44) Heavyweights         

Suppose such a ranking was used for bid allocation in USA. The next table compares the number of bid that were allocated per region for the USAU Championshipwith with the number of teams per region in the top 16 of the above ranking.

Of course, the relative number of points for tournaments is important. It must be well studied and tested to reach the good equilibrium. For example, maybe 400 or 600 points is best suited for second level tournaments instead of 500.

Pourquoi les années impaires sont plus difficiles à conquérir?

Qu'est-ce qui explique la différence? C'est que les années impaires permettent de se qualifier à des compétitions internationales. Une année sur quatre (2005, 2009, 2013, etc.), les CCU agissent comme qualification pour les Championnats du monde des clubs (WUCC). Une année sur quatre (2003, 2007, 2011, 2015, etc.), les CCU agissent comme qualification pour les Championnats du monde des nations (WUGC) auquel aucune équipe québécoise n'est jamais parvenue à se qualifier. Les années impaires, toutes les équipes au Canada considèrent les championnats canadiens plus au sérieux.

Objectif à long terme de l'ultimate compétitif au Québec

Or, je pense que c'est précisément ce que devrait être l'objectif à long terme de l'ultimate compétitif au Québec : qualifier une équipe pour les WUGC. Au cours des dernières années, on a vu les premières équipes québécoises gagner les Championnats canadiens. C'est excellent, on a brisé la glace. Maintenant, on veut viser plus haut : on veut gagner les bons championnats canadiens. Ici, j'entends les années 2015 et 2019. En 2011, avec Odyssée, nous y sommes presque parvenus, mais presque ne compte pas.

Comment gagner les bons championnats canadiens?

Je crois que la réponse doit s'inscrire dans un cycle de 4 années d'où l'image suivante. En haut et en bas, on a les années paires où les équipes québécoises ont connu de bonnes performance (12 finales des CCU). À gauche, on trouve les années de qualifications pour les WUCC où seul Méphisto a été champion en 2009 (3 finales). À droite, on trouve les années de qualifications pour les WUGC (1 seule finale, Odyssée 2011). Je reviendrai à cette image plus bas avec les opportunités et menaces.

Gagner les championnats canadiens: un objectif à long terme

Aucune équipe québécoise n'a jamais gagner les championnats canadiens à sa première année d'existence. Je pense donc qu'il est raisonnable de dire qu'on ne peut pas gagner les championnats canadiens en une seule année de préparation encore moins pour une année impaire. Voici, pour chaque équipe québécoise championne canadienne, le nombre d'années de préparation nécessaires pour gagner le championnat:

• Camelot 2006 (2e année)
• Onyx 2010 (3e année)
• Storm 2010 (4e année, 7e année pour certaines joueuses)
• Odyssée 2012 (2e année)
• Mephisto 2009 (4e année, >10e année pour certains joueurs)

On peut même dire plus. En effet, il semble qu'on puisse gagner les championnats canadiens dans un délai de 2 à 3 ans dans la catégorie mixte pour une année paire, mais qu'il faille au moins 4 ans pour gagner les championnats canadiens dans les catégories féminines et ouvertes ou pour une année impaire. Il serait intéressant de regarder le nombre d'années de préparation nécessaires pour les équipes championnes canadiennes des autres provinces.

Faut-il créer de nouvelles équipes ou poursuivre les programmes?

Je pose la question. Par programme, je veux dire des équipes qui existent pendant plusieurs années avec les mêmes joueurs possiblement avec des équipe B. Par exemple, les programmes à succès des dernières années sont Gecko (2004-2009), Méphisto (2007-2010), Méphisto-Demon-Agony (2011-2013), QUB-Quarma (2006-2013), Onyx (2008-2011), Odyssée (2011-2013), RIP (2005-2010), Storm (2004-2010), puis Storm-Mystik-Polaris-Domino (2011-2013).

Comme gagner les championnats canadiens est un objectif à long terme (particulièrement pour les années impaires), il semble que les programmes continus sont les mieux placés pour atteindre les plus hauts objectifs. La création d'une équipe sur de nouvelles bases, aussi compétitive soit-elle, est une bonne chose, mais est aussi risquée si les objectifs élevés sont à court terme, par exemple moins d'un an.

Deux exemples me viennent en tête ici, et à vous peut-être aussi. Je pense à la création de l'équipe féminine Nova au début de l'année 2013 pour une qualification aux WUCC en août 2013. Aussi à Odyssée que nous avons commencé en novembre 2010 pour remporter les championnats canadiens en août 2011. Dans les deux cas, les objectifs ont été manqués par très peu (ceci dit, Nova en terminant 4e pourrait se qualifier pour l'Italie si certains pays se désistent), mais avaient aussi été fixés pas plus de douze mois à l'avance. C'était audacieux. Que doit-on faire la prochaine fois? Comment créer une équipe qui remportera les championnats canadiens en 2015? en 2017? en 2019? Quand faut-il les créer? Je pose la question.

Opportunité et menace

Une menace importante selon moi pour réussir à se qualifier aux prestigieux championnats du monde des nations (WUGC) en 2016 ou 2020 est ...une participation aux WUCC deux ans plus tôt. Je m'explique. Si les meilleurs joueurs au Québec participent aux WUCC, c'est déjà une très belle réussite qui demandera beaucoup d'énergie, de préparation. C'est donc aussi une opportunité, car cela contribue à faire augmenter le niveau de jeu de toute la province. Par contre, les équipes qui iront en Italie à l'été 2014 ne penseront pas à autre choses d'ici là et c'est normal. Le problème, c'est que pour se qualifier aux WUGC 2016, il faut gagner les championnats canadiens en 2015. Mais si les joueurs et joueuses vont en Italie en 2014. Cela leur laissera une seule année de préparation pour gagner en août 2015.

Alors, je reviens avec la même question que plus haut. Après l'Italie, faudra-t-il créer de nouvelles équipes ou poursuivre les programmes existants pour gagner en août 2015? À regarder le passé, il semble qu'il faudra poursuivre les programmes existants.

Si les joueurs et joueuses veulent créer de nouvelles équipes (des Équipes Québec par exemple), il faut y penser dès maintenant et considérer de faire une saison de préparation à l'été 2014.