Sandbox submissions are open. All scenarios, Event or Sandbox, including Industry scenarios, are available for submission to the Sandbox.

S0

***THIS DATASET IS OBSOLETE, IT NO LONGER CONFORMS TO THE PROBLEM FORMULATION*** Use S0.1 instead.

Download the revised (12/8/2022) S0 Sandbox datasets (3.1 MB). It contains 4 network models consisting of  3-, 14-, 37- and 73-buses. Each S0 network model in this set consists one scenario with 4 JSON files for each division as follows:

1. scenario_nnn.json contains the problem input data,
2. scenario_nnn_summary.json contains information about the problem input data, including the length and number of periods as in the table below,
3. scenario_nnn_solution.json contains a feasible solution file produced by the Benchmark program for this scenario, and
4. scenario_nnn_evaluation.json, the output from the Evaluation program that calculates the objective ("z") from the solution file. This will be the score for the scenario.

Download the S0.1 Sandbox datasets (8/4/2023, 4 network models, 12 scenarios, 36 files; 0.5 MB). Each S0.1 network model in this set consists of one scenario with 3 files for each division as follows:

1. scenario_nnn.json contains the problem input data,
2. scenario_nnn.json.pop_solution.json contains a feasible solution file produced by the POP (Prior Operating Point) program for this scenario, and
3. scenario_nnn.json.popsolution.log, the output from the Evaluation program that calculates the objective ("z") from the solution file. This will be the score for the scenario.

The question has arisen to what extent did the changes from S0 to S0.1 affect the results. We recalculated using the S0.1 data and have appended the results to the right of the table below. The changes from the S0 objective to S0.1 can probably be explained as resulting from the evolution of the Benchmark code, with a notable exception, the 73-bus D1 case and perhaps D2 and D3 as well. Here, at least for D1, the changes clearly affect the character of the problem, for D2 and D3 less so, although the Benchmark team believes this too could be due to a change in the Benchmark.

These small datasets are intended to help entrants create solvers that can correctly handle Challenge 3 data formats. The solutions produced by the Benchmark are not intended to be anywhere near optimal. Their purpose is to demonstrate that feasible solutions do exist and what a valid solution file would look like.

S1

***THIS DATASET IS OBSOLETE, IT NO LONGER CONFORMS TO THE PROBLEM FORMULATION*** Use S1.1 instead.

Download the S1 Sandbox datasets (55.5 MB) released December 22, 2022. It contains 4 network models consisting of  600-, 1576-, 4200- and 6049-buses. Each S1 network model in this set consists one scenario with 3 JSON files for each division as follows:

1. scenario_nnn.json contains the problem input data,
2. scenario_nnn_pop_solution.json contains a feasible solution file produced by the POP (Prior Operating Point) program for this scenario, and
3. scenario_nnn_pop_eval_summary.json, the output from the Evaluation program that calculates the objective ("z") from the solution file. This will be the score for the scenario.

Download the S1.1 Sandbox datasets (8/7/2023, 4 network models, 12 scenarios, 32 files; 37.7 MB). Each S1.1 network model in this set consists of one scenario with 2 files for Division 1 and 3 files for Divisions 2 and 3 as follows:

1. scenario_nnn.json contains the problem input data,
2. scenario_nnn.json.pop_solution.json contains a feasible solution file produced by the POP (Prior Operating Point) program for this scenario, and
3. scenario_nnn.json.popsolution.log, the output from the Evaluation program that calculates the objective ("z") from the solution file. This will be the score for the scenario. (Not available in Division 1, basically the same as S1)

As with S0.1, we recalculated using the S1.1 data and have appended the results to the right of the table below. The changes from the S1 objective to S1.1 can probably be explained as resulting from the evolution of the Benchmark code. In some cases the newer Benchmark code takes longer so the strict time limits were removed to obtain the result.

S2

***THIS DATASET IS NOT OBSOLETE,  IT IS THE SAME AS S2.1***

Download the (3/16/2023) S2 Sandbox datasets (15.5 MB). The zip file contains 2 network models consisting of 2000- and 6717-buses intended for use with Event 2. Each network model consists one scenario with 3 JSON files for each scenario as follows:

1. scenario_001.json contains the problem input data,
2. scenario_001.json.pop_solution.json contains a feasible solution file produced by the POP (Prior Operating Point) program for this scenario, and
3. scenario_001.json.popsolution.log, the output from the Evaluation program that calculates the objective ("z") from the solution file. This will be the score for the scenario.

S3.1

Download the (06/06/2023) S3.1 Sandbox datasets (104 MB); specify dataset=S3.1 in submission.conf, not S3. The zip file contains 6 network models consisting of 14-, 37-, 1576-, 2000-, 8,316- and 23,643-buses and 41 scenarios (123 files). Event 3 will use the 73-bus and larger networks from E2 as well as the two new detailed below, with a caveat: the 23,643-bus system is very difficult and its use is still under discussion. There are two issues: memory and time. The current evaluation platform has 256 GB of memory per node but this is not enough for the current benchmark. One of the Event 2 codes was able to find a positive feasible solution for Division 1, but it took 11.6 hours (Divisions 2 and 3 ran out of memory). Another Event 2 code found negative feasible solutions but took 2.54-, 4.16-, 2.83-hours for D1, D2, D3. Only the D3 time was within our current limits. A third code produced no results for any division after 72 hours. We may raise the time limits for this one or just see what the teams can do with it as is. The two smaller networks incorporate the features seen in other networks, including the industry network used in E2 and will be used again in E3. Putting these features in the small networks is intended to aid debugging. As with E2.1 above, these datasets only satisfy versions of the data checker later than 4/25/2023. The 06/05/2023 version of S3.1 should be deleted. It has some files mislabeled.

Do not use the following. They have some invalid data uncovered by the June 7 version of the checker and have been removed from the Sandbox.

S3.1/D*/C3S3N00037D*/scenario_001.json; S3.1/D1/C3S3N01576D1/scenario_007.json; and S3.1/D*/C3S3N23643D*/scenario_001.json. 

>>>S3.1/D*/C3S3N23643D*/scenario_003.json is now available as a replacement for scenario 1 in the Sandbox.

Each network model consists one scenario with 3 JSON files for each division as follows:

1. scenario_nnn.json contains the problem input data,
2. scenario_nnn.json.pop_solution.json contains a feasible solution file produced by the POP (Prior Operating Point) program for this scenario, and
3. scenario_nnn.json.popsolution.log, the output from the Evaluation program that calculates the objective ("z") from the solution file. This will be the score for the scenario.

New in the Sandbox 8/21/23

There will be many generation/load variations in E4 based on a base configuration for each network model. Here are the base configurations (or a derivative if the base is not available) being used for Event 4. For 73-bus D2 and D3, use any of the dataset=E1.1 scenarios. Download the new data (33.88 MB, 10 scenarios).

This applies to the 6708-bus industry network as well and we suggest you practice on the scenarios in the Sandbox from previous Events.

S4

Download the (08/03/2023) S4X_20230803.zip Sandbox datasets (9 617-bus scenarios; 2.2 MB), the X indicates these are switching enhanced datasets. This set of 9 617-bus Division 1 scenarios was constructed to demonstrate the advantages of using line switching. As stated on the Languages page, setting the submission.conf parameter experiment controls the use of switching during a Sandbox run. Similar cases will be used in Event 4.

experiment = [SW0 | SW1], where SW0 indicates that the AllowSwitching parameter passed as a runtime parameter is set to 0 (switching not allowed and if detected will result in an infeasible determination for the solution). The default is SW1, i.e., switching is allowed. To access this dataset, specify dataset=S4 in submission.conf, as well as the desired model and scenario number given in the table below.

We have run the 9 S4X datasets using the Benchmark code with and without switching and observed the following results.

Download the (08/09/2023) S4X_20230809.zip Sandbox datasets (above plus 4 73-bus scenarios; 2.85 MB).

We know that there are opportunities for beneficial switching in these scenarios beyond the numerical results we have shown, particularly for the 73-bus scenarios. For each scenario, we solved it with the benchmark solver with no switching allowed. Then we modified the initial on/off status of some branches and ran the benchmark solver again and observed improvement in the objective. In some scenarios this improvement was well over 100%. On its own, so far, the benchmark solver simply turns on branches that are off. The 73-bus scenarios have cases where a branch needs to be turned off, so the benchmark solver sees no improvement.