Frequently Asked Questions
The following questions and answers are preliminary. We are still developing the rules and formulation for Challenge 2 of the GO Competition and these FAQ responses are subject to change.
Please refer to the official rules and formulation documents once they are released.
You can download (https://gocompetition.energy.gov/sites/default/files/GO_Comp_Challenge2_...) this set of Challenge 2 questions and answers. Initial release on May 26, 2020.
The Challenge 2 formulation will expand upon the Challenge 1 formulation. Challenge 2 will now additionally consider: discrete transformer tap settings, switchable shunts, phase shifting transformers, price-responsive demand, generator ramp rates, limited “fast-start” unit commitment, and topology optimization (transmission switching). Challenge 2 will consider post-contingency response on a different timescale than the Challenge 1 approach. Challenge 2 may place additional expectations on the competitors to perform a more detailed analysis as part of the Code2 model.
Similar to Challenge 1, Challenge 2 will have multiple divisions. These divisions will feature short and long time limits and will target certain potential use cases and/or new model features unique to Challenge 2. Unlike Challenge 1, we will no longer consider performance profile scoring -- the scoring method will be the same for all of Challenge 2’s divisions.
Similar to Challenge 1, there will be a split between the “Code1” and “Code2” algorithms. Code1 will primarily handle the base case with some consideration of contingency response and Code2 will focus only on the contingency response. Code1 and Code2 will each be timed separately. The Code1 time limit will be specified by each division -- We will include divisions with short time limits (5-10 minutes) and divisions with longer time limits. For Code2, there will be a dynamic time limit based on the dataset/problem dimensions rather than the division.
There are no current plans to change the hardware that was available to all teams by the end of Challenge 1. We will no longer support Matlab software on the GO Competition platform.
Similar to Challenge 1, Challenge 2 will include two formal trials with leaderboards and one final event. Only performance in the final event will impact the final ranking. Based on group performance, ARPA-E may add an additional, optional trial to provide more development time and support to the competitor teams.
Yes, we will release an updated rules document for Challenge 2.
There is no plan to change these aspects of the rules document for Challenge 2. Please refer to the new rules document once it is available.
With some minor exceptions (which will be maintained from the Challenge 1 rules document), the Challenge 2 prizes will be cash awards rather than grant awards. Challenge 2 will award prizes to the only the highest ranking teams. The prizes will be valued based on a scale by rank (i.e. 1st place will receive a larger prize than 4th place). There will likely be fewer prizes overall, but the amounts of some of these prizes will be greater than the prizes from Challenge 1.
Challenge 1 update
According to eq. 85 of the Problem Formulation document, a generator responding to a given contingency adjusts its real power output according to its predeﬁned (oﬄine) participation factor until it hits an operational bound (min or max capacity). The real power output of a responding generator g in contingency k is p + α ∆, if the generator follows its required participation factor.
Is therefore the following statement correct?
According to eq. 207 of the same document, ∆ is not in the pu-system, but in MW. In the output file solution2.txt, the delta section is also filled in MW.
The variables and parameters of the problem can be expressed in two unit conventions: a data unit convention; and a model unit convention.
The data unit convention is used for the input and output data files and is mostly MW, MVar, MVA, etc., except for voltage magnitude values which are p.u. This choice was motivated by the default unit convention of PSSE data files. We extended this data unit convention to the output data as well, i.e. solution1 and solution2.
The model unit convention is used to express the equations of the model and is exclusively p.u.
Conversions of data parameters from data units in input data files to parameters in the model and from model variables in model units to solution data in data units in solution files are given in the formulation document.
Delta is a quantity of real power; therefore, its units are MW in the data unit convention and p.u. in the model unit convention.
Consider the equation pk[g,k] = p[g] + alpha[g] * Delta[k]. This is not the same as the formulation, for it ignores pmin and pmax, but if generator g does not hit a bound in contingency k, then this equation should hold. In this equation pk, p, and Delta are quantities of real power expressed in p.u., and alpha is a dimensionless quantity. To write the solution files, pk, p, and Delta are converted to their representations pk’, p’, and Delta’ in the data unit convention, e.g. by pk’[g]=sbase*p[g]. Then the numerical values of pk’, p’, and Delta’ are written in the solution files.
Clarification about the dimension of the participation factor alpha, respectively the governor permanent droop from appendices A.4, respectively C.11. The participation factor is announced as dimensionless, and the sum of all participation factors must be one to counteract the ∆ exactly. But looking at Original_Dataset_Real-Time_Edition_1\Network_01R-10\case.inl as an example, the values are far away from 0<=alpha<=1. Is there an error in this interpretation or in the documentation? To receive valid participation factors, each alpha must be divided by the sum of all alpha. Is this true?
This is not necessarily true. Delta[k] need not be equal to the generation lost in contingency k or the change of AC power losses or any such physically meaningful value. Delta[k] is simply any quantity of real power that ensures that values of pk satisfying the real power adjustment constraints pk[g,k] = Proj_[pmin[g], pmax[g]] (p[g] + alpha[g] * Delta[k]) also satisfy the power balance constraints. If the alpha values summed to 1, then it might be possible to interpret Delta[k] in some physically meaningful way. But this interpretation is not necessary.
E.g. suppose we have 2 generators and 1 contingency, with generator 2 going out of service in contingency 1. Suppose p’ = [100.0, 100.0] (MW), alpha = [10.0, 10.0] (dimensionless), pk’ = [[200.0, 0.0]] (MW). Then we can take Delta’ = [10.0] (MW). Delta’ is not equal to the amount of generation lost in the contingency, but it is a real power quantity expressed in MW. The alpha values do not sum to 1, but they are dimensionless.
Does a generator that is Out-of-Service in a Generator-Out-of-Service contingency no longer adjusts its real power (Generator Real Power Contingency Response) and does not try to maintain the base case voltage magnitude at its bus by adjusting the reactive power output (Generator Reactive Power Contingency Response)?
A generator that is out of service in a Generator-Out-of-Service contingency must have real and reactive power output equal to 0 in that contingency, regardless of the voltage magnitude. This is contained in equations 60 and 62.
Challenge 1 General
The Department of Energy Advanced Research Projects Agency-Energy (ARPA-E) is challenging the research and industrial communities to successfully develop and test power system optimization and control algorithms on new, publicly available, large-scale, and high-fidelity power system network models. For more information, please read here.
The GO Competition consists of multiple stages summarized below:
Beta Testing: This is the predecessor to the actual competition and allows the public to understand and test the submission processes. The relatively small and simple datasets and solutions are publicly available for download. This stage began June15, 2017, and ended October 30, 2018.
Challenge 1: The first round of the formal competition will solve a Security-Constrained OPF (SCOPF) problem. The focus will be on large-scale industry sized systems, up to, up to 60,000 buses and thousands of contingencies, with an updated problem description. Datasets will come from the ARPA-E GRID DATA program and real-world industry datasets. See the Timeline document for information on dataset release.
Start Date: Fall 2018 (Projected; Subject to Appropriation of Funding)
End Date: One year after start (Projected)
Challenge 2: Challenge 2 is expected to build on the models used in Challenge 1 and may include complicating factors such as solving larger network models, optimizing power flows over both transmission and distribution systems, and/or including unit commitment.
Start Date: At the end of Challenge 1 (Projected; Subject to Appropriation of Funding))
End Date: One year after start (Projected)
All dates are approximate and subject to change. Challenge 1 and 2 are also subject to appropriation of funding.
Final rankings for Challenge 1 will be announced following the Final Event one year after Challenge 1 starts (projected).
Challenge 1 Eligibility
Challenge 1 Registration
Yes, you need to register to participate in the competition. You must register as a team, which can be comprised of an individual or more than one individual.
You must register as a team. A team may consist of one or more individuals; thus, you can be the only member of a team.
Once you have successfully registered and created a GitHub account, click on the Create Team button located on your Account View page. Follow the directions by entering a Team Name (which must be unique to the GO Competition), the GitHub username, and the ARPA-E Competition Identifier which ARPA-E has provided to you (if you don't have one, you can fill it in later from the My Team page). Skip the last step in choosing team members. Click the Save button and you should see the My Team page with only yourself as the team members.
Once you have successfully registered and created a Team GitHub account, click on the Create Team button located on your Account View page. Follow the directions by entering a Team Name (which must be unique to the GO Competition), the Team GitHub username, the ARPA-E Competition Identifier which ARPA-E has provided to you (if you don't have one, you can fill it in later after you created your team), and choose your teammates. A team may consist of one or more individuals. If you are the only person on your team, you can skip the last part; otherwise, please make sure you choose the correct individuals and they must match the team that you provided to ARPA-E on their registration form. Click the Save button and you should see the My Team page.
First you need an account on the GO Competition website. Go to the Registration Page to fill in your personal information. Once you have successfully registered, notify the team leader and s/he will add you to their team and GitHub Team repository.
You can only register with one team per challenge.
Yes, all team members must sign the required form and return it following the directions on the back of the form.
Yes. Any teams created during the Beta Testing are only for our testing period and have no implications for the competition. New teams must be created for each challenge, but you may continue to use the same name even if the membership changes.
Challenge 1 Submission
Congratulations! Here is how you can enter your solution for evaluation:
1. Register with the GO Competition by going to the Registration Page.
2. Create a Team GitHub account.
3. Create a team. All competition submissions must be submitted by a “team.” Teams can consist of an individual or many individuals.
Note: Team Creation requires a valid Team GitHub account.
4. Return to GitHub to establish the SSH key (in My Team view).
5. Commit (save/upload) your algorithms/optimization software to your GitHub repository.
6. Submit your algorithms/software for evaluation and scoring.
7. View your results from the Accounts Page or by clicking on the My Team button and view your "My Team Submission" panel.
Ultimately each team is allowed to determine and execute their own preferred approach to this problem; however, your results will be evaluated according to the formulation described in this competition. If your team believes that you have a better, or a more efficient, approach than the official formulation, you are free to develop such an approach. However, any deviations from the official formulation produced by your approach will results in penalties (on soft constraint violations) or infeasibilities (on hard constraint violations). APRA-E recommends familiarity with the official formulation and evaluation procedure so you understand your final score.
If you have qualified for Challenge 1, submit for Trial 2 using the submission process when it is open (July 17-19, 2019). You can practice with the Original Datasets and Trial 1 datasets – Trial 2 will use the same process. Teams may no longer submit scoring in Trial 1 (it is now closed) but the datasets are available to use for development/practice. You can re-submit with the Original Datasets and Trial 1 Datasets as many times as necessary to perfect the submission process. Each team can only submit one time for Trial 2 and the Final Event.
Unit commitments are not considered in the Challenge 1 formulation.
You are allowed to add new team members at any time until the Final Event. New team members should be added using the Team Change Form on the team leader’s account page.
These penalties are calculated from the absolute MVA mismatch.
Unfortunately, we cannot share any tips or strategy to teams for this aspect of the competition. We encourage you to find novel solutions to balance the time and accuracy concerns for this problem.
This is related to the previous two questions. As stated in the response to question 23, it is possible to translate a minor error in a hard constraint to a soft constraint such that the slack variables, which are defined, are used to gain feasibility.
This question is, in part, related to the preceding question. Your output files should conform to the guidelines posted on the website. Your results will be directly fed into the evaluation code. You are encouraged to download the evaluation code and to use it for your own testing purposes. We advise that you have consistent precision for your answers to avoid any issues with numerical accuracy and/or infeasibility.
All small violations can be pushed into the slack variables that are contained in the formulation; the formulation, given the relaxations, is feasible, which should allow for teams to correct their violations in such a way to grab small violations through slacks. With that said, we adjusted the formulation document on March 29th to reduce the importance of precision level differences in voltages. Please refer to the formulation document on the website for the most up-to-date information (appendix G.4).
There should be no islanding in any of the contingencies.
All controllable shunts in this competition are modeled with continuous variables, your model may/must dynamically choose the susceptance (b) across the defined full range of operation as it finds a solution.
Yes, it is possible – there is nothing in the formulation which expressly excludes this.
This differentiates between instances of the problem with similar topologies that represent different physical power grids. Each network model has multiple scenarios which reflect small topology changes (i.e., generation or transmission element availability) or operating conditions (i.e., load profiles, unit commitments). Refer to the scoring document at https://gocompetition.energy.gov/challenges/challenge-1/scoring for a detailed description of dataset terminology.
Yes, anything your code1 produces will be available for code2 to read, though code2 will not be able to make updates to sol1 after code1 completes.
Yes, the evaluation code will look for code1 and code2 before starting either program and will produce a “scripts not found” error if either is missing. If necessary, your code2 can write the version of sol2 that your code1 created.
For divisions 1 and 3, the incumbent solutions should reflect an input that would be used in practice for an SCOPF application in real-time, with a short execution time limit. We anticipate that the majority of the incumbent solutions would be at least feasible across the base case and all contingencies (feasible without constraint relaxations). At best, the incumbent solution quality may reflect the quality of the last time period’s solution provided to an operator when solving for the next operating point in real time. The incumbent solutions will be available in the input data files. You are free to choose a different starting point for your program if desired.
The clock starts as soon as your code1 is launched. The time is measured as wall-clock time.
No additional cases will be released in the Original Dataset. However, following each Trial Event, the datasets from each trial will be released for algorithm development.
At each stage of the competition, the datasets will grow in complexity (network size and contingencies) and grow in terms of the number of scenarios. Trial 1 will be close to the Original Datasets that are released but there will be more scenarios in Trial 1 and there will be some that are larger in size. Trial 2 will grow from Trial 1 and the Final Event will grow from Trial 2.
We are not publicly releasing the exact contents of the upcoming trials, but teams should expect that the size and complexity of the networks should generally increase as we progress through the competition.
No, you will only get one submission for Trial 2.
Yes, you should be able to continue working with all currently available datasets throughout the competition, but . However, please be aware that we may have to temporarily suspend executing open submissions to the platform during the evaluation periods for Trial 1, Trial 2, and the Final Event.
If you have submitted using the Original Dataset but are not on the Original Datasets leaderboard then there may have been an error with either your submission or the online platform. Please check with email@example.com so we can resolve the issue as quickly as possible. Resubmissions of the Trial 1 datasets will not affect your standing on the Trial 1 leaderboard.
The Original Dataset Leaderboards have been suspended in preparation for the Trial and Final Events.
Once we launch your code1, you are free to utilize the full extent of the hardware as best serves your program. Your submission.conf should specify the number of nodes your program will require.
Trial 2 datasets will be released soon after the trial 2 results are posted.
We are not planning to release globally optimal solutions for our datasets, but the leaderboards and trial events should provide feedback on the relative strength of your approach.
We do not plan to provide an admittance matrix.
The website receives event messages from the evaluation platform and assigns a date/time as they are processed. It processes the most recent message first, giving the appearance of events being out of order.
We are considering this, but as of now the evaluation process remains unchanged. Please contact firstname.lastname@example.org with your specific request so we can gauge interest and potentially make future improvements to our process.
The 2sec/contingency limit will not affect submissions during teams’ development windows, but the logs for successful submissions should have adequate information for teams to gauge their code 2 performance. Teams with only partially successful code 2 attempts may have to calculate their performance based on the log information.
Several teams have requested this already, we will make more information available on the website for teams interested in replicating our singularity recipe.
No, these time limits are exclusive. The code 2 timer begins with code 2, not at the end of the code 1 timer. Any extra time remaining after code1 completes is lost.
Take the URL for the submission tar.gz and insert _output1 before .tar.gz, i.e., https://dtn2.pnl.gov/arpacomp/v1//sss-nnn_2.tar.gz becomes https://dtn2.pnl.gov/arpacomp/v1//sss-nnn_2_output1.tar.gz.
Some submissions may have a different output number; check the main tar.gz file.
The trial 2 submission window schedule has been reduced and slightly delayed in order to increase the development time for the competitors. Currently, all other submission windows and other events remain as originally scheduled. We reserve the right to alter the final event schedule.
As announced on the website, APRA-E is releasing an open source version of code 2 that many teams may find helpful. ARPA-E does not guarantee that the open source code 2 release will work for every team or that it will satisfy the new time limit for trial 2. Teams are free to use the open source release as it is presented as their code 2 or to otherwise modify and or adapt the open source release to their own processes. Every team is still responsible for having a code 2 program in their github submission for Trial 2 and the Final Event.
We will not likely provide any new data formats at this point in the competition, but if desired then please send a formal request to email@example.com. You can also check the forum to see if any other team has already prepared the data in this way and would be willing to share with the group.
If your team has coded a different approach within their code 1 for divisions 3 and 4, please specify so on the submission page. Teams that select “Yes” to the question: “For Division 3 and 4: Does your code use a different approach for these divisions?” will have their code re-run for divisions 3 and 4. Teams that select “No” will have their runs from divisions 1 and 2 applied for scoring in division 3 and 4. Please remember that your code 1 must be a single program, so separate approaches must be handled internal to your code, not as separate code packages.
There is an input parameter passed to codes 1 and 2 that defines which division is being scored -- Your program may take that input and respond accordingly, but you must submit one code set that handles all four divisions, not separate codes. Only a single submission is allowed for Trial and Final Events but you may select which divisions will be scored at the time of submission.
Please document any suspected discrepancies in the datasets and contact firstname.lastname@example.org so that we can track these reports and continue to improve the realism of our test environment. We have already implemented data quality improvements for the Trial 2 datasets based on many competitors’ observations after Trial 1.
We have expanded our platform to accommodate more submissions from all of the competitors during the development windows. For the trial and final events, each team will have full use of the software and hardware described on the GO Competition website: https://gocompetition.energy.gov/evaluation-platform. We always welcome feedback, for additional software or hardware needs, please submit a formal request with email@example.com. When submitting requests, please be aware that one of our objectives is to keep the competition hardware within a reasonable realm of the means of a typical end-customer.
The network models used in Challenge 1 will include network names (i.e., “Network 3”) which identify networks that share base topologies. Competitors can expect a combination of previously encountered networks as well as new networks in Trial 1, Trial 2, and the Final Event.
The new time constraint on sol2 is not prohibitive; it is meant to encourage teams to innovate in a space that can directly compete with existing industry software. Competitors are encouraged to make full use of the platform hardware that is available to them to solve all of the contingencies within this new time limit.
The open source code 2 prepared by ARPA-E is parallelized across both cores and nodes. The developer has also recorded several webinar videos that describe his approach and methodology for code 2 and for core and node parallelization. These webinars are available at https://youtu.be/nt9JEob7BSY and https://youtu.be/M7q72Tv7Nnk and the code 2 is available online through a public github account. We recommend that all competitors make use of these materials.
Challenge 1 Scoring/Ranking
A computer algorithm will judge the submission.
The python Evaluation source code will be made publicly available through this site once the evaluation process has been finalized and approved and the code is is deemed stable and reliable.
Yes, you will get some feedback on where your code failed via a log summary. This log summary can be viewed in the submission details; your submissions can be found in the "My Team Submissions" panel; you can reach this information by going to your Team Information by clicking on My Team in View Account. Click on an individual submission and you will find the link to download log files.
The initial score is based on your submission as it compares to other submissions at that current time for the dataset that was indicated on the submission form. The ranking is based on all current submissions.
The final results will be based on unpublished datasets, so the final standings may be different. These datasets will be published after the competition and evaluation have been completed.
No. Once you have registered, you cannot change the Team GitHub account username.
The competition uses GitHub to manage submitted solutions. All code submitted by a team must be stored in GitHub. You must grant read access to the GitHub account; this is accomplished with a public SSH key that you can find in your My Team summary.
You can view/download the full step-by-step instructions (with screen shots) found under Resources. A summary of the steps are:
1. Go to the ARPA-E Optimal Power Flow Competition login page and log in using the username and password associated with your account.
2. Once logged in, go to View Account under the Account menu.
3. Click on the My Team button to view My Team information.
3. In the "Team Information" section, copy the SSH Public Key information; this is the text with "ssh-rsa ... pnl.gov". You can also click the Copy SSH Information button.
4. Go to the GitHub login page and log in using the username/e-mail address and password associated with your account.
5. Once logged in, go to your submission repository.
6. Click the “Settings” option from the tab bar along the top of the page.
7. Click the “Deploy keys” option from the pane on the left hand side of the page.
8. Click “Add deploy key” button on the right hand side of the page.
9. Enter a “Title” (e.g. ARPA Competition) and paste the SSH Public Key information from step 3. Leave all other settings.
10. Click the “Add key” button to complete the process.