Challenge 2 Trial 2 supplemental information

Below is how each team ranked for the synthetic and industry data subsets. Ensemble is the set of best scores for a Division.

MSgain for 54 synthetic scenarios
Division 1   Division 2   Division 3   Division 4
ensemble 2,136,219,406   ensemble 2,143,872,609   ensemble 2,139,613,127   ensemble 2,231,743,317
Gordian Knot (VaTech) 1,914,847,144   Gordian Knot (VaTech) 1,950,688,843   Gordian Knot (VaTech) 1,963,801,972   Gordian Knot (VaTech) 1,963,801,970
ARPA-e Benchmark 1,396,714,280   ARPA-e Benchmark 1,538,451,285   ARPA-e Benchmark 1,402,276,432   ARPA-e Benchmark 1,402,276,436
GMI-GO (GaTech) 761,869,414   GMI-GO (GaTech) 1,220,972,257   GravityX 1,038,372,756   GravityX 1,017,353,446
Tartan Buffs (Colo.) 754,219,887   GO-SNIP (Lehigh) 918,330,839   GMI-GO (GaTech) 762,013,667   GMI-GO (GaTech) 755,004,432
GO-SNIP (Lehigh) 492,712,142   Tartan Buffs (Colo.) 755,212,216   Tartan Buffs (Colo.) 718,543,008   Tartan Buffs (Colo.) 751,993,736
GravityX 207,048,087   GravityX 209,673,628   GO-SNIP (Lehigh) 492,717,194   GO-SNIP (Lehigh) 492,717,195
Pearl Street Technologies 123,969,597   Pearl Street Technologies 125,899,309   NU_Columbia_Artelys 232,037,178   NU_Columbia_Artelys 281,439,833
GOT-BSI-OPF 91,908,201   GOT-BSI-OPF 92,164,407   GOT-BSI-OPF 92,103,388   Pearl Street Technologies 109,423,539
magos (LLNL) 55,581,023   SUGAR-CMU 73,709,251   Pearl Street Technologies 42,902,485   GOT-BSI-OPF 92,103,389
SUGAR-CMU 42,506,501   GERS USA 1,853,807   SUGAR-CMU 15,501,494   SUGAR-CMU 23,357,849
GERS USA 1,853,807   NU_Columbia_Artelys 1,484,889   GERS USA 1,853,807   GERS USA 1,853,806
NU_Columbia_Artelys 1,484,889   magos (LLNL) 1.99E-06   magos (LLNL) 2.35E-06   CasePower 0
CasePower 0   CasePower 0   CasePower 0   magos (LLNL) 0
Monday Mornings (LBL) 0   Monday Mornings (LBL) 0   Monday Mornings (LBL) 0   Monday Mornings (LBL) 0
                     
MSgain for 14 industry scenarios
Division 1   Division 2   Division 3   Division 4
ensemble 1,700,802,697   ensemble 1,701,784,311   ensemble 1,703,291,083   ensemble 1,702,305,825
Tartan Buffs (Colo.) 1,622,869,446   GMI-GO (GaTech) 1,605,210,294   GMI-GO (GaTech) 1,568,487,775   GMI-GO (GaTech) 1,605,218,587
GMI-GO (GaTech) 1,568,421,745   Tartan Buffs (Colo.) 1,584,713,720   GOT-BSI-OPF 1,298,138,208   GOT-BSI-OPF 1,297,404,449
GOT-BSI-OPF 1,297,058,515   GOT-BSI-OPF 1,298,525,624   Pearl Street Technologies 1,267,751,735   Pearl Street Technologies 1,267,751,735
Pearl Street Technologies 1,267,751,735   Pearl Street Technologies 1,267,751,735   Tartan Buffs (Colo.) 629,862,153   SUGAR-CMU 1,201,224,823
SUGAR-CMU 734,122,086   SUGAR-CMU 1,201,224,823   ARPA-e Benchmark 618,092,046   Gordian Knot (VaTech) 830,176,779
ARPA-e Benchmark 618,191,781   Gordian Knot (VaTech) 830,176,779   Gordian Knot (VaTech) 555,189,832   Tartan Buffs (Colo.) 629,864,359
Gordian Knot (VaTech) 555,189,832   ARPA-e Benchmark 617,937,182   GravityX 469,971,046   ARPA-e Benchmark 618,177,453
GravityX 376,591,690   GravityX 282,946,655   SUGAR-CMU 267,858,676   GravityX 468,848,094
GO-SNIP (Lehigh) 1,841,845   GO-SNIP (Lehigh) 1,841,846   GO-SNIP (Lehigh) 1,841,845   GO-SNIP (Lehigh) 1,841,846
magos (LLNL) -7.26E-01   magos (LLNL) -7.26E-01   magos (LLNL) -7.26E-01   magos (LLNL) -7.26E-01
CasePower -7.26E-01   CasePower -7.26E-01   CasePower -7.26E-01   CasePower -7.26E-01
GERS USA -7.26E-01   GERS USA -7.26E-01   GERS USA -7.26E-01   GERS USA -7.26E-01
Monday Mornings (LBL) -7.26E-01   Monday Mornings (LBL) -7.26E-01   Monday Mornings (LBL) -7.26E-01   Monday Mornings (LBL) -7.26E-01
NU_Columbia_Artelys -7.26E-01   NU_Columbia_Artelys -7.26E-01   NU_Columbia_Artelys -7.26E-01   NU_Columbia_Artelys -7.26E-01

Best Scores

This table gives the number of best scores a team achieved.

number of best scores per team (all 68 scenarios)
Division 1 Division 2 Division 3 Division 4 All divisions
GravityX 39 GravityX 35 GravityX 43 GravityX 41 GravityX 158
Tartan Buffs (Colo.) 7 ARPA-e Benchmark 9 Gordian Knot (VaTech) 5 ARPA-e Benchmark 9 ARPA-e Benchmark 22
Pearl Street Technologies 6 Tartan Buffs (Colo.) 6 Tartan Buffs (Colo.) 5 Pearl Street Technologies 5 Pearl Street Technologies 22
Gordian Knot (VaTech) 5 Pearl Street Technologies 6 Pearl Street Technologies 5 Gordian Knot (VaTech) 3 Tartan Buffs (Colo.) 21
GMI-GO (GaTech) 4 GMI-GO (GaTech) 5 ARPA-e Benchmark 3 GO-SNIP (Lehigh) 3 Gordian Knot (VaTech) 17
GO-SNIP (Lehigh) 3 Gordian Knot (VaTech) 4 GO-SNIP (Lehigh) 3 Tartan Buffs (Colo.) 3 GMI-GO (GaTech) 13
magos (LLNL) 2 GO-SNIP (Lehigh) 3 GMI-GO (GaTech) 2 NU_Columbia_Artelys 2 GO-SNIP (Lehigh) 12
ARPA-e Benchmark 1 CasePower 0 NU_Columbia_Artelys 2 GMI-GO (GaTech) 2 NU_Columbia_Artelys 4
GOT-BSI-OPF 1 GERS USA 0 GOT-BSI-OPF 0 SUGAR-CMU 0 magos (LLNL) 2
CasePower 0 GOT-BSI-OPF 0 SUGAR-CMU 0 CasePower 0 GOT-BSI-OPF 1
GERS USA 0 magos (LLNL) 0 CasePower 0 GERS USA 0 SUGAR-CMU 0
Monday Mornings (LBL) 0 Monday Mornings (LBL) 0 GERS USA 0 GOT-BSI-OPF 0 CasePower 0
NU_Columbia_Artelys 0 NU_Columbia_Artelys 0 magos (LLNL) 0 magos (LLNL) 0 GERS USA 0
SUGAR-CMU 0 SUGAR-CMU 0 Monday Mornings (LBL) 0 Monday Mornings (LBL) 0 Monday Mornings (LBL) 0

The following table lists the best MSgain1 for each scenario and the its percent increase over the second best, MSgain2.

Model Scenario MSpp   Division 1   Division 2 Division 3     Division 4
  MSgain1  MSgain2  (g1/g2) - 1 Team 1 Team 2   MSgain1  MSgain2  (g1/g2) - 1 Team 1 Team 2   MSgain1  MSgain2  (g1/g2) - 1 Team 1 Team 2   MSgain1  MSgain2  (g1/g2) - 1 Team 1 Team 2
C2T2N00617 185 1,004,392   576,582 575,343 0.215% GravityX Tartan Buffs   576,582 575,343 0.215% GravityX Tartan Buffs   576,582 575,343 0.215% GravityX Tartan Buffs   576,582 575,343 0.215% GravityX Tartan Buffs
C2T2N00617 203 1,342,314   459,078 458,081 0.218% GravityX Tartan Buffs   459,078 458,081 0.218% GravityX Tartan Buffs   459,078 458,081 0.218% GravityX Tartan Buffs   459,078 458,081 0.218% GravityX Tartan Buffs
C2T2N00617 311 1,014,008   570,183 569,034 0.202% GravityX Tartan Buffs   570,183 569,034 0.202% GravityX Tartan Buffs   570,183 569,034 0.202% GravityX Tartan Buffs   570,183 569,034 0.202% GravityX Tartan Buffs
C2T2N00617 420 1,116,223   544,055 543,075 0.181% GravityX Tartan Buffs   544,055 543,075 0.181% GravityX Tartan Buffs   544,055 543,075 0.181% GravityX Tartan Buffs   544,055 543,075 0.181% GravityX Tartan Buffs
C2T2N00617 464 1,101,306   530,192 528,866 0.251% GravityX Tartan Buffs   530,192 528,866 0.251% GravityX Tartan Buffs   530,192 528,866 0.251% GravityX Tartan Buffs   530,192 528,866 0.251% GravityX Tartan Buffs
C2T2N01737 51 212,452,148   26,133,045 25,928,522 0.789% GravityX GMI-GO   26,133,045 25,927,309 0.794% GravityX GMI-GO   26,133,044 25,932,502 0.773% GravityX GMI-GO   26,133,044 25,937,013 0.756% GravityX GMI-GO
C2T2N01737 151 195,660,751   41,038,058 40,876,347 0.396% GravityX GMI-GO   41,038,058 40,876,347 0.396% GravityX GMI-GO   41,037,677 40,876,347 0.395% GravityX GMI-GO   41,037,677 40,876,347 0.395% GravityX GMI-GO
C2T2N01737 181 204,293,198   33,347,679 32,638,193 2.174% GravityX GMI-GO   33,347,679 32,639,559 2.170% GravityX GMI-GO   33,347,678 32,636,914 2.178% GravityX GMI-GO   33,347,678 32,638,193 2.174% GravityX GMI-GO
C2T2N02020 5 1,708,829   2,197,022 2,195,913 0.050% GravityX Tartan Buffs   2,197,022 2,195,913 0.050% GravityX Tartan Buffs   2,196,973 2,195,913 0.048% GravityX Tartan Buffs   2,196,973 2,195,913 0.048% GravityX Tartan Buffs
C2T2N02020 59 2,207,928   1,960,691 1,957,272 0.175% Tartan Buffs GravityX   1,960,691 1,957,272 0.175% Tartan Buffs GravityX   1,962,077 1,960,691 0.071% GravityX Tartan Buffs   1,962,077 1,960,691 0.071% GravityX Tartan Buffs
C2T2N02020 85 2,208,707   1,960,079 1,956,632 0.176% Tartan Buffs GravityX   1,960,079 1,956,632 0.176% Tartan Buffs GravityX   1,961,445 1,960,079 0.070% GravityX Tartan Buffs   1,961,445 1,960,079 0.070% GravityX Tartan Buffs
C2T2N02020 182 3,228,823   1,058,461 1,058,265 0.018% GMI-GO Tartan Buffs   1,058,461 1,058,265 0.018% GMI-GO Tartan Buffs   1,058,806 1,058,461 0.033% NU_Columbia_Artelys GMI-GO   1,058,806 1,058,461 0.033% NU_Columbia_Artelys GMI-GO
C2T2N02020 208 3,317,783   969,675 969,143 0.055% GMI-GO Tartan Buffs   969,675 969,143 0.055% GMI-GO Tartan Buffs   969,869 969,675 0.020% NU_Columbia_Artelys GMI-GO   969,869 969,675 0.020% NU_Columbia_Artelys GMI-GO
C2T2N03022 261 211,162,297   263,041,530 206,911,068 27.128% Tartan Buffs Gordian Knot   263,041,530 207,037,492 27.050% Tartan Buffs Gordian Knot   259,025,879 232,277,961 11.515% Tartan Buffs GravityX   259,025,879 235,193,320 10.133% Tartan Buffs GravityX
C2T2N03022 361 97,730,419   273,193,015 221,171,945 23.521% Tartan Buffs Gordian Knot   273,193,015 263,523,549 3.669% Tartan Buffs ARPA-e Benchmark   276,841,310 221,171,945 25.170% Tartan Buffs Gordian Knot   276,841,310 263,523,549 5.054% Tartan Buffs ARPA-e Benchmark
C2T2N03022 641 80,039,206   1,206,047 878,525 37.281% GravityX GMI-GO   1,186,976 878,525 35.110% GravityX GMI-GO   1,150,017 878,525 30.903% GravityX GMI-GO   1,071,261 878,525 21.939% GravityX GMI-GO
C2T2N03022 781 99,783,440   1,266,414 1,038,286 21.972% GravityX GMI-GO   1,217,702 1,038,286 17.280% GravityX GMI-GO   1,254,072 1,038,286 20.783% GravityX GMI-GO   1,282,214 1,038,286 23.493% GravityX GMI-GO
C2T2N03593 21 11,294,527   1,898,418 1,891,734 0.353% Gordian Knot GravityX   1,898,418 1,891,734 0.353% Gordian Knot GravityX   1,898,418 1,889,298 0.483% Gordian Knot GravityX   1,898,418 1,889,298 0.483% Gordian Knot GravityX
C2T2N04200 1 1,421,859   1,450,402 1,446,228 0.289% GravityX Gordian Knot   1,450,402 1,446,228 0.289% GravityX Gordian Knot   1,450,402 1,446,228 0.289% GravityX Gordian Knot   1,450,402 1,446,228 0.289% GravityX Gordian Knot
C2T2N04200 12 1,487,467   1,857,812 1,849,771 0.435% GravityX Gordian Knot   1,857,812 1,849,771 0.435% GravityX Gordian Knot   1,855,703 1,849,771 0.321% GravityX Gordian Knot   1,857,812 1,849,771 0.435% GravityX Gordian Knot
C2T2N04200 35 303,411   1,574,991 1,566,589 0.536% GravityX Gordian Knot   1,574,991 1,566,589 0.536% GravityX Gordian Knot   1,574,991 1,566,589 0.536% GravityX Gordian Knot   1,574,991 1,566,589 0.536% GravityX Gordian Knot
C2T2N04200 75 733,040   1,986,284 1,980,099 0.312% GravityX Gordian Knot   1,986,284 1,980,099 0.312% GravityX Gordian Knot   1,986,284 1,980,099 0.312% GravityX Gordian Knot   1,986,284 1,980,099 0.312% GravityX Gordian Knot
C2T2N04200 85 3,745,483   4,325,791 3,976,159 8.793% GravityX magos   4,325,791 3,944,145 9.676% GravityX Tartan Buffs   4,325,785 3,983,350 8.597% GravityX Tartan Buffs   4,325,785 3,983,350 8.597% GravityX Tartan Buffs
C2T2N04917 1 292,337,915   401,586,679 292,702,074 37.200% Gordian Knot ARPA-e Benchmark   401,601,005 292,702,074 37.205% Gordian Knot ARPA-e Benchmark   401,601,005 292,702,074 37.205% Gordian Knot ARPA-e Benchmark   489,677,879 401,601,005 21.931% GravityX Gordian Knot
C2T2N04917 81 245,371,507   459,514,404 282,980,416 62.384% Gordian Knot ARPA-e Benchmark   460,253,173 282,980,416 62.645% Gordian Knot ARPA-e Benchmark   460,253,173 282,980,416 62.645% Gordian Knot ARPA-e Benchmark   460,253,173 282,980,416 62.645% Gordian Knot ARPA-e Benchmark
C2T2N04917 161 211,002,200   456,018,738 297,139,128 53.470% Gordian Knot ARPA-e Benchmark   456,018,738 297,139,128 53.470% Gordian Knot ARPA-e Benchmark   456,018,738 297,139,128 53.470% Gordian Knot ARPA-e Benchmark   456,018,738 297,139,128 53.470% Gordian Knot ARPA-e Benchmark
C2T2N04917 581 158,443,065   1,729,360 1,317,399 31.271% GravityX Pearl Street Technologies   1,317,399 1,291,564 2.000% Pearl Street Technologies GravityX   1,698,256 1,317,399 28.910% GravityX Pearl Street Technologies   1,373,144 1,317,399 4.231% GravityX Pearl Street Technologies
C2T2N04917 681 171,193,394   1,789,919 1,391,404 28.641% GravityX ARPA-e Benchmark   1,813,728 1,799,940 0.766% ARPA-e Benchmark GravityX   1,788,600 1,391,404 28.546% GravityX ARPA-e Benchmark   1,813,728 1,800,133 0.755% ARPA-e Benchmark GravityX
C2T2N06100 4 700,409   1,094,964 1,093,396 0.143% ARPA-e Benchmark GMI-GO   1,094,964 1,093,396 0.143% ARPA-e Benchmark GMI-GO   1,094,964 1,094,193 0.070% ARPA-e Benchmark NU_Columbia_Artelys   1,094,964 1,094,193 0.070% ARPA-e Benchmark NU_Columbia_Artelys
C2T2N06100 21 190,674   1,727,723 1,709,643 1.058% GravityX Gordian Knot   1,727,426 1,709,643 1.040% GravityX Gordian Knot   1,727,786 1,709,643 1.061% GravityX Gordian Knot   1,744,027 1,709,643 2.011% GravityX Gordian Knot
C2T2N06100 43 42,398   1,883,638 1,871,879 0.628% GravityX Gordian Knot   1,883,583 1,871,879 0.625% GravityX Gordian Knot   1,896,147 1,871,879 1.296% GravityX Gordian Knot   1,896,147 1,871,879 1.296% GravityX Gordian Knot
C2T2N06100 93 394,874   1,324,293 1,304,537 1.514% GravityX Gordian Knot   1,324,485 1,304,537 1.529% GravityX Gordian Knot   1,340,190 1,304,537 2.733% GravityX Gordian Knot   1,340,188 1,304,537 2.733% GravityX Gordian Knot
C2T2N06100 107 561,087   1,156,166 1,140,955 1.333% GravityX Gordian Knot   1,156,171 1,113,340 3.847% GravityX Tartan Buffs   1,156,156 1,140,955 1.332% GravityX Gordian Knot   1,165,917 1,140,955 2.188% GravityX Gordian Knot
C2T2N06705 1 172,667,314   115,273,827 114,302,584 0.850% Gordian Knot Tartan Buffs   116,611,922 115,273,827 1.161% GMI-GO Gordian Knot   117,301,917 115,273,827 1.759% GravityX Gordian Knot   116,611,625 116,103,511 0.438% GMI-GO GravityX
C2T2N06705 131 225,215,225   65,256,841 62,914,911 3.722% GMI-GO Tartan Buffs   65,251,077 62,914,911 3.713% GMI-GO Tartan Buffs   65,256,602 64,645,554 0.945% GMI-GO GravityX   65,251,091 65,010,463 0.370% GMI-GO GravityX
C2T2N06705 201 241,459,921   52,589,439 51,822,929 1.479% GravityX GMI-GO   52,557,727 51,822,909 1.418% GravityX GMI-GO   52,592,227 51,822,921 1.484% GravityX GMI-GO   52,557,739 51,822,955 1.418% GravityX GMI-GO
C2T2N06705 281 260,916,980   33,839,742 33,278,618 1.686% GravityX GMI-GO   33,278,618 31,055,482 7.159% GMI-GO Gordian Knot   34,043,436 33,278,618 2.298% GravityX GMI-GO   34,004,353 33,278,618 2.181% GravityX GMI-GO
C2T2N06705 351 191,161,278   98,725,780 98,304,185 0.429% GravityX GMI-GO   98,967,900 98,304,079 0.675% GravityX GMI-GO   98,980,216 98,304,118 0.688% GravityX GMI-GO   98,764,331 98,304,069 0.468% GravityX GMI-GO
C2T2N07000 1 378,723   2,460,059 2,422,727 1.541% GravityX Gordian Knot   2,460,059 2,422,727 1.541% GravityX Gordian Knot   2,460,060 2,422,727 1.541% GravityX Gordian Knot   2,460,060 2,422,727 1.541% GravityX Gordian Knot
C2T2N07000 16 309,295   2,502,149 2,462,305 1.618% GravityX Gordian Knot   2,502,149 2,462,305 1.618% GravityX Gordian Knot   2,502,149 2,462,305 1.618% GravityX Gordian Knot   2,502,149 2,462,305 1.618% GravityX Gordian Knot
C2T2N07000 31 685,246   2,926,488 2,828,405 3.468% GravityX Tartan Buffs   2,926,488 2,828,405 3.468% GravityX Tartan Buffs   2,926,491 2,828,890 3.450% GravityX Tartan Buffs   2,926,491 2,828,890 3.450% GravityX Tartan Buffs
C2T2N07000 52 779,114   2,888,989 2,806,791 2.929% GravityX Tartan Buffs   2,888,989 2,806,791 2.929% GravityX Tartan Buffs   2,888,993 2,807,067 2.919% GravityX Tartan Buffs   2,888,993 2,807,067 2.919% GravityX Tartan Buffs
C2T2N07000 70 45,498   2,762,243 2,712,903 1.819% GravityX Gordian Knot   2,762,243 2,712,903 1.819% GravityX Gordian Knot   2,762,242 2,678,540 3.125% GravityX Tartan Buffs   2,762,242 2,712,903 1.819% GravityX Gordian Knot
C2T2N08300 1 3,239,041   2,846,482 2,608,866 9.108% GravityX Tartan Buffs   2,846,482 2,608,866 9.108% GravityX Tartan Buffs   2,846,482 2,612,677 8.949% GravityX Tartan Buffs   2,846,482 2,612,677 8.949% GravityX Tartan Buffs
C2T2N08300 9 50,044,645   38,408,730 33,880,501 13.365% GO-SNIP GravityX   38,408,730 33,880,501 13.365% GO-SNIP GravityX   38,408,730 33,880,501 13.365% GO-SNIP GravityX   38,408,730 33,880,501 13.365% GO-SNIP GravityX
C2T2N08300 20 50,003,232   38,093,923 33,357,236 14.200% GO-SNIP Tartan Buffs   38,093,923 33,357,236 14.200% GO-SNIP Tartan Buffs   38,093,923 33,305,086 14.379% GO-SNIP GravityX   38,093,923 33,305,045 14.379% GO-SNIP GravityX
C2T2N08300 37 3,048,258   3,004,931 2,762,431 8.778% GravityX Tartan Buffs   3,004,931 2,762,431 8.778% GravityX Tartan Buffs   3,004,931 2,762,540 8.774% GravityX Tartan Buffs   3,004,931 2,762,540 8.774% GravityX Tartan Buffs
C2T2N08300 75 49,989,684   38,095,723 33,318,725 14.337% GO-SNIP GravityX   38,095,723 33,320,129 14.332% GO-SNIP GravityX   38,095,723 33,319,695 14.334% GO-SNIP GravityX   38,095,723 33,320,137 14.332% GO-SNIP GravityX
C2T2N10000 702 9,284,076   2,869,650 1,239,053 131.600% GravityX Pearl Street Technologies   2,869,603 1,239,771 131.462% GravityX ARPA-e Benchmark   2,869,611 1,239,053 131.597% GravityX Pearl Street Technologies   2,869,585 1,239,771 131.461% GravityX ARPA-e Benchmark
C2T2N10000 841 8,351,159   1,269,805 310,254 309.279% GravityX ARPA-e Benchmark   1,263,366 741,498 70.380% GravityX GMI-GO   1,269,182 623,867 103.438% GravityX NU_Columbia_Artelys   1,269,574 724,944 75.127% GravityX GMI-GO
C2T2N12209 26 5,909,640   278,282 249,747 11.425% Tartan Buffs ARPA-e Benchmark   278,282 249,747 11.425% Tartan Buffs ARPA-e Benchmark   365,199 278,282 31.233% GravityX Tartan Buffs   365,220 278,282 31.241% GravityX Tartan Buffs
C2T2N12209 36 7,487,358   365,240 275,809 32.425% GravityX Tartan Buffs   365,240 275,809 32.425% GravityX Tartan Buffs   368,901 275,809 33.752% GravityX Tartan Buffs   370,910 275,809 34.481% GravityX Tartan Buffs
C2T2N12209 111 14,463,085   10,908,958 10,888,131 0.191% Pearl Street Technologies Tartan Buffs   10,917,123 10,908,958 0.075% ARPA-e Benchmark Pearl Street Technologies   10,917,123 10,908,958 0.075% ARPA-e Benchmark Pearl Street Technologies   10,917,123 10,908,958 0.075% ARPA-e Benchmark Pearl Street Technologies
C2T2N12209 152 10,528,027   15,294,511 15,284,316 0.067% magos Tartan Buffs   15,284,316 15,283,726 0.004% Tartan Buffs ARPA-e Benchmark   15,284,316 15,283,726 0.004% Tartan Buffs ARPA-e Benchmark   15,284,316 15,283,726 0.004% Tartan Buffs ARPA-e Benchmark
C2T2N12209 195 12,738,104   8,981,296 8,970,960 0.115% magos ARPA-e Benchmark   8,970,960 8,960,724 0.114% ARPA-e Benchmark Pearl Street Technologies   8,970,960 8,960,724 0.114% ARPA-e Benchmark Pearl Street Technologies   8,970,960 8,960,724 0.114% ARPA-e Benchmark Pearl Street Technologies
C2T2N17700 1 8,617,584   3,823,562 3,706,091 3.170% GravityX Tartan Buffs   3,824,947 3,822,678 0.059% ARPA-e Benchmark GravityX   3,823,029 3,715,712 2.888% GravityX Tartan Buffs   3,824,947 3,821,744 0.084% ARPA-e Benchmark GravityX
C2T2N17700 2 7,413,327   5,015,836 5,007,342 0.170% GravityX Pearl Street Technologies   5,019,671 5,014,665 0.100% ARPA-e Benchmark GravityX   5,015,827 5,007,342 0.169% GravityX Pearl Street Technologies   5,019,671 5,014,656 0.100% ARPA-e Benchmark GravityX
C2T2N30000 504 5,580,251   2,135,254 1,818,535 17.416% GravityX Tartan Buffs   2,907,452 2,786,591 4.337% GravityX Tartan Buffs   2,135,252 1,820,124 17.314% GravityX Tartan Buffs   2,907,440 2,786,564 4.338% GravityX Tartan Buffs
C2T2N30000 606 6,970,657   1,458,546 853,136 70.963% GravityX Tartan Buffs   2,147,370 1,223,588 75.498% GravityX ARPA-e Benchmark   1,457,307 853,136 70.818% GravityX Tartan Buffs   2,147,372 1,222,985 75.584% GravityX ARPA-e Benchmark
C2T2N30000 726 9,042,693   1,042,335 523,391 99.150% Tartan Buffs ARPA-e Benchmark   2,228,663 1,042,911 113.696% GravityX Tartan Buffs   1,087,238 813,179 33.702% Tartan Buffs NU_Columbia_Artelys   2,228,667 1,087,211 104.989% GravityX Tartan Buffs
C2T2N31777 9 5,269,257   21,072,960 20,837,263 1.131% Tartan Buffs GMI-GO   21,102,564 21,096,656 0.028% ARPA-e Benchmark Tartan Buffs   21,072,954 20,837,589 1.130% Tartan Buffs GMI-GO   21,102,564 21,096,656 0.028% ARPA-e Benchmark Tartan Buffs
C2T2N31777 20 1,180,109   20,968,826 12,711,520 64.959% GOT-BSI-OPF ARPA-e Benchmark   25,429,536 25,267,629 0.641% ARPA-e Benchmark GMI-GO   23,923,243 20,968,826 14.090% Gordian Knot GOT-BSI-OPF   25,429,536 25,267,629 0.641% ARPA-e Benchmark GMI-GO
C2T2N31777 29 9,478,806   18,190,438 17,510,958 3.880% GMI-GO Gordian Knot   18,422,606 18,192,941 1.262% ARPA-e Benchmark GMI-GO   18,194,547 17,510,958 3.904% GMI-GO Gordian Knot   18,422,606 18,193,313 1.260% ARPA-e Benchmark GMI-GO
C2T2N34363 94 4,420,332   244,877,053 231,117,217 5.954% Pearl Street Technologies Tartan Buffs   244,877,053 241,845,406 1.254% Pearl Street Technologies Tartan Buffs   244,877,053 225,986,396 8.359% Pearl Street Technologies GMI-GO   244,877,053 227,848,078 7.474% Pearl Street Technologies SUGAR-CMU
C2T2N34363 102 635,721   248,389,867 246,264,580 0.863% Pearl Street Technologies Gordian Knot   248,389,867 246,929,018 0.592% Pearl Street Technologies Gordian Knot   248,389,867 246,264,580 0.863% Pearl Street Technologies Gordian Knot   248,389,867 246,929,018 0.592% Pearl Street Technologies Gordian Knot
C2T2N34363 108 1,866,674   247,469,146 235,792,403 4.952% Pearl Street Technologies Tartan Buffs   247,469,146 244,659,222 1.149% Pearl Street Technologies Tartan Buffs   247,469,146 228,749,155 8.184% Pearl Street Technologies GMI-GO   247,469,146 232,874,276 6.267% Pearl Street Technologies SUGAR-CMU
C2T2N34363 114 3,253,657   246,121,777 235,337,683 4.582% Pearl Street Technologies Tartan Buffs   246,121,777 243,267,028 1.174% Pearl Street Technologies Gordian Knot   246,121,777 232,560,255 5.831% Pearl Street Technologies SUGAR-CMU   246,121,777 243,267,028 1.174% Pearl Street Technologies Gordian Knot
C2T2N34363 144 3,574,163   245,842,026 240,533,794 2.207% Pearl Street Technologies Tartan Buffs   245,842,026 242,531,962 1.365% Pearl Street Technologies Tartan Buffs   245,842,026 233,649,212 5.218% Pearl Street Technologies GOT-BSI-OPF   245,842,026 238,751,590 2.970% Pearl Street Technologies SUGAR-CMU
industry   ensembles   3,837,022,102 3,204,632,219 19.734% full ensemble 1 full ensemble 2   3,845,656,920 3,292,291,200 16.808% full ensemble 1 full ensemble 2   3,842,904,210 3,219,710,613 19.356% full ensemble 1 full ensemble 2   3,934,049,141 3,405,046,442 15.536% full ensemble 1 full ensemble 2
      2,136,219,406 1,553,628,519 37.499% synthetic ensemble 1 synthetic ensemble 2   2,143,872,609 1,612,352,409 32.966% synthetic ensemble 1 synthetic ensemble 2   2,139,613,127 1,587,840,915 34.750% synthetic ensemble 1 synthetic ensemble 2   2,231,743,317 1,749,515,986 27.563% synthetic ensemble 1 synthetic ensemble 2
      1,700,802,697 1,651,003,700 3.016% industry ensemble 1 industry ensemble 2   1,701,784,311 1,679,938,791 1.300% industry ensemble 1 industry ensemble 2   1,703,291,083 1,631,869,698 4.377% industry ensemble 1 industry ensemble 2   1,702,305,825 1,655,530,456 2.825% industry ensemble 1 industry ensemble 2
          Maximum 309.279%         Maximum 131.462%         Maximum 131.597%         Maximum 131.461%    

Division Differences

One would expect that Divisions 2 or 4, with a 60-minute time limit, would have better MSgain  values than those achieved by Divisions 1 or 3, with a 5-minute time limit, at least for sufficiently large problems. The Competition is also interested in whether or not transmission line and transformer switching can further improve MSgain. The following tables look at these differences, at least from the perspective of the composite MSgain  values.

all 68 scenarios
60 minutes vs. 5 minutes   switched vs non-switched
Division 2 - Division 1   Division 4 - Division 3   Division 3 - Division 1   Division 4 - Division 2
SUGAR-CMU 498,305,487   SUGAR-CMU 983,717,551   GravityX 903,684,710   GravityX 1,316,680,366
GMI-GO (GaTech) 495,891,391   GMI-GO (GaTech) 495,392,769   NU_Columbia_Artelys 279,954,944   NU_Columbia_Artelys 275,005,030
GO-SNIP (Lehigh) 425,618,697   GO-SNIP (Lehigh) 425,600,663   Gordian Knot (VaTech) 48,954,828   ensemble 88,392,221
Gordian Knot (VaTech) 310,828,646   GravityX 321,976,161   ensemble 5,882,108   Gordian Knot (VaTech) 1,001,979
ARPA-e Benchmark 141,482,405   Gordian Knot (VaTech) 262,875,797   ARPA-e Benchmark 5,462,417   ARPA-e Benchmark 240,597
ensemble 8,634,818   ARPA-e Benchmark 136,260,585   GOT-BSI-OPF 1,274,881   GERS USA 0
Pearl Street Technologies 1,929,711   ensemble 91,144,931   GMI-GO (GaTech) 210,283   CasePower 0
GOT-BSI-OPF 1,723,316   Pearl Street Technologies 1,929,711   GO-SNIP (Lehigh) 5,052   Monday Mornings (LBL) 0
NU_Columbia_Artelys 0   Tartan Buffs (Colo.) 951,130   GERS USA 0   SUGAR-CMU 0
CasePower 0   magos (LLNL) 9.49E-05   Pearl Street Technologies 0   Pearl Street Technologies 0
Monday Mornings (LBL) 0   CasePower 0   CasePower 0   magos (LLNL) -9.07E-05
GERS USA -2.56E-03   Monday Mornings (LBL) 0   Monday Mornings (LBL) 0   GO-SNIP (Lehigh) -12,982
Tartan Buffs (Colo.) -37,163,397   GERS USA 0   magos (LLNL) -55,581,023   GMI-GO (GaTech) -288,339
magos (LLNL) -55,581,023   GOT-BSI-OPF -619,094   SUGAR-CMU -485,412,064   GOT-BSI-OPF -1,067,529
GravityX -91,019,495   NU_Columbia_Artelys -4,949,914   Tartan Buffs (Colo.) -995,233,443   Tartan Buffs (Colo.) -957,118,916
                     
54 synthetic scenarios
60 minutes vs. 5 minutes   switched vs non-switched
Division 2 - Division 1   Division 4 - Division 3   Division 3 - Division 1   Division 4 - Division 2
GMI-GO (GaTech) 459,102,843   ensemble 92,130,190   GravityX 831,324,668   GravityX 807,679,818
GO-SNIP (Lehigh) 425,618,697   Pearl Street Technologies 66,521,054   NU_Columbia_Artelys 230,552,289   NU_Columbia_Artelys 279,954,944
ARPA-e Benchmark 141,737,005   NU_Columbia_Artelys 49,402,655   Gordian Knot (VaTech) 48,954,828   ensemble 87,870,708
Gordian Knot (VaTech) 35,841,699   Tartan Buffs (Colo.) 33,450,728   ARPA-e Benchmark 5,562,151   Gordian Knot (VaTech) 13,113,127
SUGAR-CMU 31,202,750   SUGAR-CMU 7,856,355   ensemble 3,393,722   CasePower 0
ensemble 7,653,203   ARPA-e Benchmark 4   GOT-BSI-OPF 195,188   Monday Mornings (LBL) 0
GravityX 2,625,541   GO-SNIP (Lehigh) 7.18E-01   GMI-GO (GaTech) 144,253   magos (LLNL) -1.99E-06
Pearl Street Technologies 1,929,711   GOT-BSI-OPF 5.95E-01   GO-SNIP (Lehigh) 5,052   GERS USA -7.78E-01
Tartan Buffs (Colo.) 992,328   CasePower 0   GERS USA 2.10E-03   GOT-BSI-OPF -61,018
GOT-BSI-OPF 256,206   Monday Mornings (LBL) 0   CasePower 0   Tartan Buffs (Colo.) -3,218,480
NU_Columbia_Artelys 0   magos (LLNL) -2.35E-06   Monday Mornings (LBL) 0   Pearl Street Technologies -16,475,770
CasePower 0   GERS USA -7.83E-01   SUGAR-CMU -27,005,006   SUGAR-CMU -50,351,402
Monday Mornings (LBL) 0   Gordian Knot (VaTech) -2   Tartan Buffs (Colo.) -35,676,879   ARPA-e Benchmark -136,174,849
GERS USA -2.56E-03   GMI-GO (GaTech) -7,009,235   magos (LLNL) -55,581,023   GO-SNIP (Lehigh) -425,613,644
magos (LLNL) -55,581,023   GravityX -21,019,310   Pearl Street Technologies -81,067,112   GMI-GO (GaTech) -465,967,825
                     
14 industry scenarios
60 minutes vs. 5 minutes   switched vs non-switched
Division 2 - Division 1   Division 4 - Division 3   Division 3 - Division 1   Division 4 - Division 2
SUGAR-CMU 467,102,737   SUGAR-CMU 933,366,147   GravityX 93,379,356   GravityX 185,901,440
Gordian Knot (VaTech) 274,986,948   Gordian Knot (VaTech) 274,986,948   ensemble 2,488,386   ensemble 521,514
GMI-GO (GaTech) 36,788,549   GMI-GO (GaTech) 36,730,812   GOT-BSI-OPF 1,079,693   ARPA-e Benchmark 240,271
GOT-BSI-OPF 1,467,109   ARPA-e Benchmark 85,406   GMI-GO (GaTech) 66,030   GMI-GO (GaTech) 8,294
ensemble 981,614   Tartan Buffs (Colo.) 2,206   NU_Columbia_Artelys 0   Gordian Knot (VaTech) 1.19E-06
GO-SNIP (Lehigh) 5.07E-01   GO-SNIP (Lehigh) 5.07E-01   GO-SNIP (Lehigh) 0   CasePower 0
Pearl Street Technologies 0   magos (LLNL) 9.60E-05   GERS USA 0   GERS USA 0
NU_Columbia_Artelys 0   Pearl Street Technologies 0   CasePower 0   GO-SNIP (Lehigh) 0
CasePower 0   NU_Columbia_Artelys 0   Monday Mornings (LBL) 0   Monday Mornings (LBL) 0
Monday Mornings (LBL) 0   CasePower 0   Pearl Street Technologies 0   NU_Columbia_Artelys 0
GERS USA 0   Monday Mornings (LBL) 0   Gordian Knot (VaTech) -1.91E-06   SUGAR-CMU 0
magos (LLNL) 0   GERS USA 0   magos (LLNL) -1.86E-04   Pearl Street Technologies 0
ARPA-e Benchmark -254,599   GOT-BSI-OPF -733,759   ARPA-e Benchmark -99,734   magos (LLNL) -9.00E-05
Tartan Buffs (Colo.) -38,155,726   ensemble -985,258   SUGAR-CMU -466,263,410   GOT-BSI-OPF -1,121,175
GravityX -93,645,035   GravityX -1,122,952   Tartan Buffs (Colo.) -993,007,294   Tartan Buffs (Colo.) -954,849,362