Challenge 1 Evaluation | Grid Optimization Competition

The Challenge 1 Solution Evaluation detailed description is in Appendix F of the Problem Formulation.

The evaluation process will assess the provided solutions against the official challenge problem; however, Entrants are free to utilize any other formulation or modeling approach within their solution software. Entrants are required to calculate and report power flow solutions for the base case and all contingency cases (given their previously reported decision variables). With this information, all SCOPF solutions will be validated in a uniform way by forward constraint evaluation. In order to ensure fairness and to enable the use of alternative problem formulations, where appropriate, there will be two sections for each competition algorithm, referred to as Solver1 and Solver2, each generating a different output file.

Solution 1

The first solution section, Solver1, which has a strict time limit, will solve the base SCOPF problem and report the base case operating point as output. This information is recorded in the file solution1.[txt|hdf5] described in Challenge 1 Output Data and Files. This information, along with the cost function information taken from the Generator Dispatch Units, Active Power Dispatch Tables and Piecewise Linear Cost Curve Tables in the Cost Function Data File (case.rop), is used to compute the Objective Function value that is used as the scenario score.

Solution 2

The second solution section, Solver2, solves the SCOPF power flow solutions for all contingency cases (given their previously reported decision variables in the solution1 file). This information is recorded in the file solution2.[txt|hdf5], also described in Challenge 1 Output Data and Files. It is used to assess the feasibility of the base case operating point with respect to the base case constraints.

Summary

In summary, the evaluation and scoring process consists of the following steps carried out by the Competition Platform for each scenario:

Run Solver1, with a time limit specified as an input parameter, to produce solution1.[txt|hdf5] and, optionally, solution2.[txt|hdf5].
In the absence of the solution2 file, run Solver2 to produce solution2.[txt|hdf5]. There will be a time limit, also specified as an input parameter, to stop runaway solvers, but it will be generous. Solver2 is expected to finish normally. If this is not the case, contact the GO Operations Team.
Run evaluation.py (see below) with the 4 data input files and the 2 solution files as input to calculate the Objective Function value, the Cost, the Penalty (Objective - Cost) and both base and contingency constraint violations, writing the results to the submissionID_scenario_n_DetailedSolution.csv file and a one line summary the submisssionID_Score.csv file. Note: in the absence of solution files, or errors in processing them, the Objective Function value will be set to a penalty value described below as WorstCase (see the Scoring document). The Objective Function value is the Scenario Score.
The geometric mean of all the Power System Network Model Scenario Scores is computed to generate Power System Network Model Score and written to the submisssionID_Score.csv file.
The geometric mean of all the Power System Network Mode Scores is computed to generate the Dataset Score, which is also written to the submisssionID_Score.csv file.
The Dataset Score is used in Division 1 and 2 objective function scoring.
The Scenario scores are used in Division 3 and 4 performance profile scoring.

Evaluation Souce Code

The python Evaluation source code, evaluation.py is available on GitHub at https://github.com/GOCompetition/Evaluation. The latest commit was November 6, 2019, adds a check on generators/branches mentioned in the con file but not in the raw file.

The update September 13, 2019, prints out some additional information on the base case. The update July 19, 2019, checks and scrubs switched shunt swrem. Should be == 0. The update July 11, 2019, checks for unallowable characters in ID strings and the pmin >= 0. The update April 26, 2019, fixed a data conversion issue. The update April 12, 2019, included additional alerts and fixed scrub_gen_cost. The update April 5, 2019, set config parameters for revising generator cost or normalizing part.fact (default is false). The update March 29, 2019, fixed the check on bmin and bmax in view of 1e-4 tolerance and scrub commas from ROP file. The update March 27, 2019, checks that at there is at least 1 outage in each contingency and checks binit feasibility. The update March 21, 2019, removes quotes in ROP The update March 16, 2019, removed extra datasets. The update March 15, 2019 inserted additional check_data scripts. The February 18, 2019, update fixed some Python 3 issues. The update on January 16, 2019, introduced performance improvements. The update on December 6, 2018, reduced the memory requirements. The update on November 29, 2018, made the code compatible with both Python 2 and 3.

The penalty parameters (in data units) in evaluation.py are discussed in Appendix D.1 of the Problem Formulation document.

Entrants may find the input data parsing done by the Evaluation code (specifically in data.py) a useful model for their own code. The data.py file is in the search path for all python submissions so it does not need to be included in a Team GitHub repository.

The python code WorstCase, available on GitHub at https://github.com/GOCompetition/WorstCase, constructs a guaranteed feasible solution with no starting point information by insuring that the normal voltage magnitude meets normal and emergency bounds in both base and contingency cases and stays the same. While feasible, the result is far from optimal and represents something of a worst case that any solver should be able to exceed. The resulting slack objective value is used when no other objective value is found due to missing solution files or errors in processing the solution files.

Interpreting DetailedSolution.csv

The output folder for every scenario has a DetailedSolution.csv file. This file is useful for identifying which constraints may be causing you problems. The definitive explanation of the column headers in the DetailedSolution.csv files produced by the Evaluation code is in the Evaluation code (search for the first instance of csv_writer.writerow). For convenience the columns are listed here. Numbers in parenthesis refer to equations in the Problem Formulation.

'ctg', # contingency label for the current contingency, empty if base case

'infeas', # binary indicator of infeasibility for the base case or contingency of the current row - 1 indicates infeasible

'pen', # penalty value on soft constraint violations in the base case or current contingency (1,6-31)

'cost', # generator cost (base case only), 0 in contingencies (1-5)

'obj', # obj, = pen + cost, cumulative (i.e. base case + all ctgs through the current row) (1)

'vmax-idx', # bus number of maximum violation of bus voltage magnitude upper bounds (32,58)

'vmax-val', # value of maximum violation of bus voltage magnitude upper bounds (32,58)

'vmin-idx', # bus number of maximum violation of bus voltage magnitude lower bounds (32,58)

'vmin-val', # value of maximum violation of bus voltage magnitude lower bounds (32,58)

'bmax-idx', # bus number of maximum violation of switched shunt susceptance upper bounds (37,63)

'bmax-val', # value of maximum violation of switched shunt susceptance upper bounds (37,63)

'bmin-idx', # bus number of maximum violation of switched shunt susceptance lower bounds (37,63)

'bmin-val', # value of maximum violation of switched shunt susceptance lower bounds (37,63)

'pbal-idx', # bus number of maximum violation of real power balance constraints (46-48,72-74)

'pbal-val', # value of maximum violation of real power balance constraints (46-48,72-74)

'qbal-idx', # bus number of maximum violation of reactive power balance constraints (49-51,75-77)

'qbal-val', # value of maximum violation of reactive power balance constraints (49-51,75-77)

'pgmax-idx', # bus and unit id of maximum violation of generator real power upper bounds (33,34)

'pgmax-val', # value of maximum violation of generator real power upper bounds (33,34)

'pgmin-idx', # bus and unit id of maximum violation of generator real power lower bounds (33,34)

'pgmin-val', # value of maximum violation of generator real power lower bounds (33,34)

'qgmax-idx', # bus and unit id of maximum violation of generator reactive power upper bounds (35,36,61,62)

'qgmax-val', # value of maximum violation of generator reactive power upper bounds (35,36,61,62)

'qgmin-idx', # bus and unit id of maximum violation of generator reactive power lower bounds (35,36,61,62)

'qgmin-val', # value of maximum violation of generator reactive power lower bounds (35,36,61,62)

'qvg1-idx', # bus and unit id of maximum violation of generator pv/pq switching constraints of type 1 (under voltage -> reactive power at max) (94)

'qvg1-val', # value of maximum violation of generator pv/pq switching constraints of type 1 (under voltage -> reactive power at max) (94)

'qvg2-idx', # bus and unit id of maximum violation of generator pv/pq switching constraints of type 2 (overvoltage -> reactive power at min) (95)

'qvg2-val', # value of maximum violation of generator pv/pq switching constraints of type 2 (overvoltage -> reactive power at min) (95)

'lineomax-idx', # origin destination and circuit id of maximum violation of line origin flow bounds (52,53,78,79)

'lineomax-val', # value of maximum violation of line origin flow bounds (52,53,78,79)

'linedmax-idx', # origin destination and circuit id of maximum violation of line destination flow bounds (53,54,79,80)

'linedmax-val', # value of maximum violation of line destination flow bounds (53,54,79,80)

'xfmromax-idx', # origin destination and circuit id of maximum violation of transformer origin flow bounds (55,56,81,82)

'xfmromax-val', # value of maximum violation of transformer origin flow bounds (55,56,81,82)

'xfmrdmax-idx', # origin destination and circuit id of maximum violation of transformer destination flow bounds (56,57,82,83)

'xfmrdmax-val', # value of maximum violation of transformer destination flow bounds (56,57,82,83)