Trending Topics – Wholesale markets need reform, but flexibility, not resilience, is the key

A version of this article was posted on Greentech Media on October 31, 2017.

By Eric Gimon

U.S. electricity markets face scrutiny over revenue problems and reliability concerns as greater amounts of renewable energy and distributed resources come online, particularly after the Department of Energy’s (DOE) Notice of Proposed Rulemaking, but coal and nuclear subsidies to boost “resilience” miss the main challenge facing wholesale markets – the need for grid flexibility.

Flexibility, not fuel on-hand, is at the core of what it means for a grid to be reliable or resilient.  Where restructured wholesale markets rule and the “invisible hand” of the market is meant to provide secure and efficient real-time balancing of supply and demand, resource adequacy, and long-term cost recovery for system resources, markets and regulators have been particularly slow to evolve.  Markets, especially, are powerful tools for finding least-cost resources to meet physical grid needs, but tend to favor incumbent generation over variable resources and flexible demand- and supply-side resources against the near and long-term interests of consumers.

To reach a clean, resilient, affordable future, markets must evolve to value flexible resources – the key to reducing integration costs for variable resources.  Two recent reports from America’s Power Plan (APP) outline how markets can evolve in the short- and long-term to cost-effectively integrate ever higher amounts of variable renewable generation like wind and solar.

Flexibility is the coin of the realm

Utility-scale and distributed renewable energy resources are on a tear.  A recent Lawrence Berkeley National Lab report found utility-scale solar total installation costs have dropped 80 percent since 2010, and residential solar systems have fallen 60 percent over the same time.  DOE’s SunShot goal of $1/watt utility-scale solar has been met three years ahead of schedule, and new wind power is coming in below $20 per megawatt-hour (MWh), cheaper than running many coal plants.   But while economics and environmental goals drive increasing renewable generation, variable resources challenge our existing frameworks for grid management and investments.

One key resource need stands out for both the near-term and long-term evolution of our electricity grids and wholesale markets: flexibility.  Flexibility broadly means the grid’s ability to adjust generation dispatch, reconfigure transmission and distribution systems, and modulate demand to accommodate predictable and unpredictable balances between supply and demand.   Flexibility is an aggregate quality of networked grids, combining both the technical capabilities of all connected devices and the system’s ability to efficiently coordinate them.

Current trends have increased the need for flexibility and the opportunity to make more of it available to grid operators.  New variable generation like solar and wind are one of the biggest drivers of need for more flexibility, but aging infrastructure and inflexible power plants, outdated utility business models, and our society’s increasing dependence on reliable electric service also demand a more flexible grid.

Opportunities to unlock flexibility are everywhere: new and more flexible gas plants, storage deployed at all scales, power electronics to regulate wind and solar output as along with transmission and distribution assets, and a constellation of connected devices ready to consume electricity more intelligently.  Expanding the “balancing area” geography over which supply and demand is balanced helps too.

From NREL’s The Value of Energy Storage for Grid Applications

Restructured wholesale electricity markets, which dominate America’s electricity landscape but work best by avoiding specific technology mandates, need to find new and improved ways to surface the value of flexibility and allow current and future market participants to provide it at least cost.

Getting more flexibility today

A new research paper from APP experts Robbie Orvis and Sonia Aggarwal, “A Roadmap for Finding Flexibility in Wholesale Markets,” highlights best practices for market design and operations in a high-renewables future, focusing on ways policymakers can unlock more cheap flexibility.

The paper identifies main challenges to integrating renewables in wholesale markets: managing predictable and unpredictable variation on the bulk system, and doing the same with distributed renewables.  For utility-scale renewables, predictable variability means knowing when a wind front or windy season is coming, or when the sun is rising or setting, with associated net load ramps.  Unpredictable variability comes from sudden weather changes, like unexpected multi-day lulls.  For distributed renewable energy resources like rooftop solar or demand response, unpredictability is a function of exogenous factors like the weather, but also reflects how opaque the distribution system is to bulk system operators.  Distributed assets like PV, efficiency, or demand response can behave predictably, but grid operators need to have more data about where DERs are on the system, what kind of DERs they are, and how they are programmed.

Flexibility is the key ingredient to manage each of these challenges, empowering wholesale markets with the ability to automatically adapt to variations in net-load in cost-efficient and reliable ways.  Best practices from across U.S. wholesale markets illuminate the near-term path forward:

  • Fix market rules to unlock flexibility of existing resources

In one example, system operators can create a net generation product for distributed resources that enables aggregators to participate via fleets, making the size threshold as small as possible.  NYISO’s Behind-the-Meter net generation resource allows behind the meter storage to participate in wholesale electricity markets, including being dispatched beyond the meter.

  • Create and modify products to harness the flexibility of existing resources and incent new flexible resources

Higher scarcity pricing and reserve adders are one of many ways to do this.  ERCOT’s high scarcity price and Operating Reserve Demand Curve adder creates additional value for flexible units during times of system stress.  Where necessary, system operators can create new products for flexibility or products that reward flexible resources, even if just for a limited number of years.  Finally, system operators can pay for reliability services that are of increasing importance but are currently uncompensated, like frequency response.

A clean, high renewables future is within sight. Policy-makers need only look at the best practices of their colleagues around the country to understand how to manage the transition as it happens.

Paying for flexibility and other resources in the future

Over the long term, however, more significant structural changes are likely required to integrate low-cost renewables and manage the major resource base transition animating wholesale markets.  Wholesale markets will need to reach a stable end-state where they can successfully manage real-time dispatch, resource adequacy (especially flexibility) and long-term cost recovery for clean resources.

A second APP research paper, “On Market Designs for a Future with a High Penetration of Variable Renewable Generation,” offers two possible paths for wholesale electricity markets to manage three key challenges in a high renewables future:

  • How will the market pay for the long-term provision of electricity when marginal costs are zero much of the time?
  • How do grid operators make operational decisions of which zero-cost assets to dispatch in times of surplus?
  • What will be the roles of distributed resources, especially the controllable ones? What price signals will they follow and how will they be dispatched?

The first path is an evolved version of today’s markets which becomes increasingly dependent on flexibility from storage and demand-side’s ability to shift consumption for its viability.  The second path splits the market into a long-term “firm” market which covers most consumer needs and a “residual market” which operates much like todays’ spot markets but trades in both withdrawals and injections of electricity against the “firm” market deliveries.

These paths address the challenges above in diverse ways but they also overlap by leaning on long-term contracting, a natural way to align with the investment needs of capital-heavy fuel-light assets.  They also both avoid capacity remuneration mechanisms commonly seen today – in other words, capacity markets.

In addition, the two general paths above for the evolution of wholesale market design in a future with a high penetration of variable renewable energy and distributed resources reveal several important themes:

  • Alignment: Markets must be aligned with physical and financial realities of the underlying for viable market design.
  • Optimization: Markets need the tools to optimize both near-term dispatch and long-term investment in grid assets.
  • Risk Management: Markets must be able to toad value by managing risk by shifting risk from one set of parties to another (customers to generators) and by reducing risk through pooling (lowering costs as well).  Any future market design needs to provide this function.


The transition from today’s legacy grid into a low-cost, low-carbon engine for our future economy is an especially pronounced challenge for restructured wholesale electricity markets because they function through a combination of direct regulatory interventions and dynamic market forces.  The two new APP research papers focus on the start and finish of this transition.  They identify changes needed to provide the opportunity for and manage meaningful increments in renewable generation today and provide a vision for how the future grid might function, consistent with engineering and financial realities.

But another important part of overcoming this challenge includes managing a timely transition to a cleaner grid where many fossil-fueled assets will need to retire before the end of their useful life, and where the grid may sometimes be long or short on some of the resources it needs.  Lessons from today and a vision for future help us on the path, but much work remains to be done.