Sandbox submissions are open. 

S0

***THIS DATASET IS OBSOLETE, IT NO LONGER CONFORMS TO THE PROBLEM FORMULATION*** Use S0.1 instead.

Download the revised (12/8/2022) S0 Sandbox datasets (3.1 MB). It contains 4 network models consisting of  3-, 14-, 37- and 73-buses. Each S0 network model in this set consists one scenario with 4 JSON files for each division as follows:

1. scenario_nnn.json contains the problem input data,
2. scenario_nnn_summary.json contains information about the problem input data, including the length and number of periods as in the table below,
3. scenario_nnn_solution.json contains a feasible solution file produced by the Benchmark program for this scenario, and
4. scenario_nnn_evaluation.json, the output from the Evaluation program that calculates the objective ("z") from the solution file. This will be the score for the scenario.

Download the S0.1 Sandbox datasets (8/4/2023, 4 network models, 12 scenarios, 36 files; 0.5 MB). Each S0.1 network model in this set consists of one scenario with 3 files for each division as follows:

1. scenario_nnn.json contains the problem input data,
2. scenario_nnn.json.pop_solution.json contains a feasible solution file produced by the POP (Prior Operating Point) program for this scenario, and
3. scenario_nnn.json.popsolution.log, the output from the Evaluation program that calculates the objective ("z") from the solution file. This will be the score for the scenario.

The question has arisen to what extent did the changes from S0 to S0.1 affect the results. We recalculated using the S0.1 data and have appended the results to the right of the table below. The changes from the S0 objective to S0.1 can probably be explained as resulting from the evolution of the Benchmark code, with a notable exception, the 73-bus D1 case and perhaps D2 and D3 as well. Here, at least for D1, the changes clearly affect the character of the problem, for D2 and D3 less so, although the Benchmark team believes this too could be due to a change in the Benchmark.

These small datasets are intended to help entrants create solvers that can correctly handle Challenge 3 data formats. The solutions produced by the Benchmark are not intended to be anywhere near optimal. Their purpose is to demonstrate that feasible solutions do exist and what a valid solution file would look like.

S1

***THIS DATASET IS OBSOLETE, IT NO LONGER CONFORMS TO THE PROBLEM FORMULATION*** Use S1.1 instead.

Download the S1 Sandbox datasets (55.5 MB) released December 22, 2022. It contains 4 network models consisting of  600-, 1576-, 4200- and 6049-buses. Each S1 network model in this set consists one scenario with 3 JSON files for each division as follows:

1. scenario_nnn.json contains the problem input data,
2. scenario_nnn_pop_solution.json contains a feasible solution file produced by the POP (Prior Operating Point) program for this scenario, and
3. scenario_nnn_pop_eval_summary.json, the output from the Evaluation program that calculates the objective ("z") from the solution file. This will be the score for the scenario.

Download the S1.1 Sandbox datasets (8/7/2023, 4 network models, 12 scenarios, 32 files; 37.7 MB). Each S1.1 network model in this set consists of one scenario with 2 files for Division 1 and 3 files for Divisions 2 and 3 as follows:

1. scenario_nnn.json contains the problem input data,
2. scenario_nnn.json.pop_solution.json contains a feasible solution file produced by the POP (Prior Operating Point) program for this scenario, and
3. scenario_nnn.json.popsolution.log, the output from the Evaluation program that calculates the objective ("z") from the solution file. This will be the score for the scenario. (Not available in Division 1, basically the same as S1)

As with S0.1, we recalculated using the S1.1 data and have appended the results to the right of the table below. The changes from the S1 objective to S1.1 can probably be explained as resulting from the evolution of the Benchmark code. In some cases the newer Benchmark code takes longer so the strict time limits were removed to obtain the result.

E1

***THIS DATASET IS NOT OBSOLETE, IT IS THE SAME AS E1.1***

Download the E1 Sandbox datasets (312.5 MB) released February 14, 2023. It contains 257 scenarios comprised of 8 network models consisting of  3-, 14-, 37-, 73-, 600-, 1576-, 4200- and 6049-buses from the S0 and S1 datasets, with variations on load profiles. No industry dataset was used for Event 1.

Each E1 network model in this set consists of multiple scenarios, each with 3 files (771 total files):

1. scenario_nnn.json contains the problem input data,
2. scenario_nnn.json.pop_solution.json contains a feasible solution file produced by the POP (Prior Operating Point) program for this scenario, and
3. scenario_nnn.json.popsolution.log, the output from the Evaluation program that calculates the objective ("z") from the solution file. If positive and feasible, this will be the score for the scenario.

The solutions produced by the Prior Operating Point (POP) program are not intended to be anywhere near optimal. Their purpose is to demonstrate that feasible solutions do exist and what a valid solution file would look like. The E1 results demonstrate that positive feasible solutions are possible for every scenario (see the Best results in column E). See the Event 1 Leaderboard for more results.

S2

***THIS DATASET IS NOT OBSOLETE,  IT IS THE SAME AS S2.1***

Download the (3/16/2023) S2 Sandbox datasets (15.5 MB). The zip file contains 2 network models consisting of 2000- and 6717-buses intended for use with Event 2. Each network model consists one scenario with 3 JSON files for each division as follows:

1. scenario_001.json contains the problem input data,
2. scenario_001.json.pop_solution.json contains a feasible solution file produced by the POP (Prior Operating Point) program for this scenario, and
3. scenario_001.json.popsolution.log, the output from the Evaluation program that calculates the objective ("z") from the solution file. This will be the score for the scenario.

E2

***THIS DATASET IS OBSOLETE, IT NO LONGER CONFORMS TO THE PROBLEM FORMULATION*** Use E2.1 instead.

The E2 Sandbox datasets released May 10, 2023 are available as three downloads (one for each Division): C3E2D1 (65 MB); C3E2D2 (320 MB); and C3E2D3 (245 MB). Together they contain 175 scenarios comprised of 8 network models consisting of  73-, 617-, 1576-, 2000-, 4224-, 6049-, 6708-, and 6717-buses from the S1 and S2 datasets, with variations on load profiles. The 6708-bus network is an industry dataset and is not included in the download but is available for use in the Sandbox, although results will be limited to the objective value produced by the Evaluation program. The 6708-bus dataset makes use of the "additional_shunt" feature described on page 15 of the Data Format document.

Each E2 network model in this set consists of multiple scenarios, each with 3 files as in E1:

1. scenario_nnn.json contains the problem input data,
2. scenario_nnn.json.pop_solution.json contains a feasible solution file produced by the POP (Prior Operating Point) program for this scenario, and
3. scenario_nnn.json.popsolution.log, the output from the Evaluation program that calculates the objective ("z") from the solution file. If positive and feasible, this will be the score for the scenario.

The solutions produced by the Prior Operating Point (POP) program are not intended to be anywhere near optimal. Their purpose is to demonstrate that feasible solutions do exist and what a valid solution file would look like.

E2.1

A requirement on problem data that the penalty coefficients on certain soft constraints be strictly positive has been added to the Problem Formulation. These constraints are bus real and reactive power balance, line apparent power flow limits, multi-interval energy production and consumption limits, and reserve balance constraints on regulation down, regulation up, synchronized reserve, non-synchronized reserve, ramping reserve down, ramping reserve up, reactive power reserve down, and reactive power reserve up. This requirement is posted in the May 15, 2023 version of the Problem Formulation document, and the change is mentioned in the change log item for May 15, 2023. The data checker code in C3DataUtilities now checks for positivity of these coefficients and reports an error if they are not positive. Two public datasets, the E2 2000- and 6717-bus systems released May 10, do not meet this positivity requirement and therefore fail the current data checker. We have modified these datasets by setting the relevant coefficients equal to 1 ($/pu-h), and the modified datasets pass the data checker. Download the (5/15/2023) E2.1 Sandbox datasets (276 MB) with the updated 2000- and 6717-bus systems.

S3.1

Download the (06/06/2023) S3.1 Sandbox datasets (104 MB); specify dataset=S3.1 in submission.conf, not S3. The zip file contains 6 network models consisting of 14-, 37-, 1576-, 2000-, 8,316- and 23,643-buses and 41 scenarios (123 files). Event 3 will use the 73-bus and larger networks from E2 as well as the two new detailed below, with a caveat: the 23,643-bus system is very difficult and its use is still under discussion. There are two issues: memory and time. The current evaluation platform has 256 GB of memory per node but this is not enough for the current benchmark. One of the Event 2 codes was able to find a positive feasible solution for Division 1, but it took 11.6 hours (Divisions 2 and 3 ran out of memory). Another Event 2 code found negative feasible solutions but took 2.54-, 4.16-, 2.83-hours for D1, D2, D3. Only the D3 time was within our current limits. A third code produced no results for any division after 72 hours. We may raise the time limits for this one or just see what the teams can do with it as is. The two smaller networks incorporate the features seen in other networks, including the industry network used in E2 and will be used again in E3. Putting these features in the small networks is intended to aid debugging. As with E2.1 above, these datasets only satisfy versions of the data checker later than 4/25/2023. The 06/05/2023 version of S3.1 should be deleted. It has some files mislabeled.

Do not use the following. They have some invalid data uncovered by the June 7 version of the checker and have been removed from the Sandbox.

S3.1/D*/C3S3N00037D*/scenario_001.json; S3.1/D1/C3S3N01576D1/scenario_007.json; and S3.1/D*/C3S3N23643D*/scenario_001.json. 

>>>S3.1/D*/C3S3N23643D*/scenario_003.json is now available as a replacement for scenario 1 in the Sandbox.

Each network model consists one scenario with 3 JSON files for each division as follows:

1. scenario_nnn.json contains the problem input data,
2. scenario_nnn.json.pop_solution.json contains a feasible solution file produced by the POP (Prior Operating Point) program for this scenario, and
3. scenario_nnn.json.popsolution.log, the output from the Evaluation program that calculates the objective ("z") from the solution file. This will be the score for the scenario.

New in the Sandbox 8/21/23

There will be many generation/load variations in E4 based on a base configuration for each network model. Here are the base configurations (or a derivative if the base is not available) being used for Event 4. For 73-bus D2 and D3, use any of the dataset=E1.1 scenarios. Download the new data (33.88 MB, 10 scenarios).

This applies to the 6708-bus industry network as well and we suggest you practice on the scenarios in the Sandbox from previous Events.

E3.1

Due to its size, the (06/29/2023) E3.1 Sandbox datasets come in 3 files: Division 1 (28 datasets, 124 MB), Division 2 (41 datasets, 171 MB), and Division 3 (28 datasets, 106 MB).

The Division 1 network models are C3E3N00617D1, C3E3N01576D1, C3E3N04224D1, C3E3N06049D1, C3E3N06717D1, C3E3N08316D1, and C3E3N23643D1.

The Division 2 network models are C3E3N00073D2, C3E3N00617D2, C3E3N01576D2, C3E3N04224D2, C3E3N06049D2, C3E3N08316D2, and C3E3N23643D2.

The Division 3 network models are C3E3N00073D3,  C3E3N00617D3, C3E3N01576D3, C3E3N04224D3, C3E3N06049D3, C3E3N08316D3, and C3E3N23643D3.

The industry network, C3E3N06708D[1|2|3], is not available for download but may be run in the Sandbox. See the E3 Leaderboard for the list of all scenarios.

Each network model consists one or more scenarios with 3 JSON files:

1. scenario_nnn.json contains the problem input data,
2. scenario_nnn.json.pop_solution.json contains a feasible solution file produced by the POP (Prior Operating Point) program for this scenario, and
3. scenario_nnn.json.popsolution.log, the output from the Evaluation program that calculates the objective ("z") from the solution file. This will be the score for the scenario.

Specify dataset=E3.1 in submission.conf

S4

Download the (08/03/2023) S4X_20230803.zip Sandbox datasets (9 617-bus scenarios; 2.2 MB), the X indicates these are switching enhanced datasets. This set of 9 617-bus Division 1 scenarios was constructed to demonstrate the advantages of using line switching. As stated on the Languages page, setting the submission.conf parameter experiment controls the use of switching during a Sandbox run. Similar cases will be used in Event 4.

experiment = [SW0 | SW1], where SW0 indicates that the AllowSwitching parameter passed as a runtime parameter is set to 0 (switching not allowed and if detected will result in an infeasible determination for the solution). The default is SW1, i.e., switching is allowed. To access this dataset, specify dataset=S4 in submission.conf, as well as the desired model and scenario number given in the table below.

We have run the 9 S4X datasets using the Benchmark code with and without switching and observed the following results.

Download the (08/09/2023) S4X_20230809.zip Sandbox datasets (above plus 4 73-bus scenarios; 2.85 MB).

We know that there are opportunities for beneficial switching in these scenarios beyond the numerical results we have shown, particularly for the 73-bus scenarios. For each scenario, we solved it with the benchmark solver with no switching allowed. Then we modified the initial on/off status of some branches and ran the benchmark solver again and observed improvement in the objective. In some scenarios this improvement was well over 100%. On its own, so far, the benchmark solver simply turns on branches that are off. The 73-bus scenarios have cases where a branch needs to be turned off, so the benchmark solver sees no improvement.