Why the NEM is stuggling to meet it's objectives

By Tyson Vaughan, Technical Director – Energy Policy, Mott MacDonald, and Dr Stephen Hinchliffe, Energy Advisory Leader – Asia-Pacific, Mott MacDonald

Global energy markets and grids are in transition, and Australia’s National Electricity Market is no different. Here, we take a look at recent rule changes within the market, and the influence these have had as we increase the concentration of renewable energy in the grid.

Up until the late 2010s, Australia's National Electricity Market (NEM) had ample generating capacity. This was primarily because from the 1960s to the 1980s, state governments built most of the coal power stations that we rely on for base load today.

However, a view was formed that state governments built too many power stations, triggering the breakup of the state utilities and the development of the NEM.

However, this excess supply helped make sure that the new market was competitive, and this helped deliver some of the lowest electricity prices in the world during the 1990s and 2000s.

More recently, a series of coal plant retirements during the 2010s, combined with a lack of support for new generation capacity, significantly reduced the size of the generation fleet compared to demand.

The market suspension event in June 2022 revealed how tight supply is in the NEM.

Gas generators did not bid into the market to generate (because the cost of burning gas to make electricity was higher than the maximum market price). Some coal-fired generators could not physically generate because they were shut down for maintenance, or had wet coal supplies due to flooding caused by a La Niña event – arguably made more powerful because of global warming. Output from solar PV and wind turbines was low because it was the middle of winter and wind conditions were light. And, ironically, because winter was unusually cold (again, thanks to climate change) demand was high for heating.

This series of events, coupled with high periods of electricity prices hitting the ceiling price, caused AEMO to suspend the market and dispatch generators, irrespective of their preferred, commercially driven, dispatch position.

Coal and gas are set to remain expensive in the medium-term. Russian gas embargoes are likely to continue, keeping gas prices high in the medium-term. La Niña is forecast to peak in November 2022, but rain and flooding will threaten coal stocks for months to come. As such, this market volatility, and the potential for AEMO to intervene again, remains.

The cost of running coal plant, exacerbated by their advanced age, market requirement to flex more than their design parameters intended, and carbon risks means that more than 85 per cent of the NEM's coal plants are expected to retire by 2034. This, combined with expected demand increases from electrification (transition from gas and liquid fuels to electricity, and technologies like hydrogen electrolysers), provides a clear signal that the market will face supply shortages over the next decade.

Variable Renewable Energy (VRE) has been demonstrated to be the cheapest form of new generation at least since 20171, and is cheaper to operate than fossil fuel generation. If we are to guarantee a cost-effective and reliable electricity supply for consumers and industry; and reduce carbon emissions, we need to significantly increase Australia's generation capacity, and that generation should come from renewables and storage. However, this does not necessarily mean that costs will fall, as the capital and operating cost of required new transmission lines to connect distributed VRE, coupled with the cost for both short duration (intra-day firming) and long duration storage (inter-day firming to cover periods of wind and solar droughts) are high.

The relatively moderate pace of roll out of new VRE is not due to lack of interest. At the time of writing, there are 165GW of projects proposing to connect to the NEM – more than three times current installed VRE capacity. However, market and regulatory barriers, particularly around connection, prevent these generators from coming online quick enough to replace declining fossil fuel generation. This potentially leads to increased supply constraints and hence on-going high electricity prices.

Considering this, and considering market failures, it is arguable there is a need for governments to help coordinate and support investment in generation, storage, and transmission networks like they did during the 1960s to the 1980s.

New renewables policy

Given this, it might surprise some to know that most of the recent rule changes completed by the Australian Energy Market Commission (AEMC), the NEM rule-maker, do not provide this needed support to VRE developers and operators.

That said, there are some rule changes that do help clean energy technologies. The Integrating energy storage systems into the NEM rule change, for example, clarified obligations for storage technologies and enabled large-scale storage to co-locate with generation in hybrid projects. It also maintained the existing framework for network charges, allowing storage participants to obtain a negotiated shared transmission service with their relevant transmission network service provider. Broadly, under a negotiated agreement, storage participants would not have to pay for transmission use of system charges if they do not charge at times that would trigger the need for new network investment.

However, some rule changes, like the Semi-scheduled generator dispatch obligations, make regulatory requirements more stringent for renewable generators. Nevertheless, rule changes like this are generally reasonable because they seek to make the obligations equitable across all participants.

And this is what many of the AEMC's rule changes have focused on: making the rules technology-neutral to be “fair” to all market participants. They have also focused on creating new markets for system services. This is in line with the National Electricity Objectives (NEO) of a safe, efficient, and reliable power system, the guiding principle for the AEMC.

The normative aspect of the NEO is that it recommends broadly that the market, rather than governments, is best placed to coordinate investment and decide what is the least-cost mix of generation and transmission. However, this fails to consider that maybe markets are not best placed to solve extremely complex problems involving long-term investment decision making that feature split incentives and have significant impacts on public welfare.

If the energy transition was simply about the market seeing a signal to invest, then we should have been flooded with new power plant connections without government assistance. However, despite high prices, historically low interest rates, access to low-cost finance, a glut of market information and clear signals about tight supply, the market simply isn't delivering the amount of generation and transmission necessary to replace existing supply (at least, outside of government support).

It may be that the market is simply becoming too complicated for investors to handle without government guidance. The premise behind the NEM is that the private sector is almost always better placed to make allocative and rational economic decisions than government. Unfortunately, examples like the solar farms stranded in the “rhombus of regret” in Northwest Victoria, or how changing (increasingly volatile and declining) marginal loss factors caught so many VRE operators and owners off-guard, show that investors are not as omniscient as they are in economic textbooks. Both investors and analysts have told us that it is simply becoming too hard to reasonably forecast

both sides of the ledger for new projects, particularly in new markets.

The number and frequency of rule changes are increasing to try to enable the market and the power system to facilitate new supply. But while these rule changes are helpful, they do not make the market less complex.

State support for new development is helping to overcome this problem. Many state ministers have determined that the states should lead development of their own capacity mechanisms rather than the Energy Security Board. The NSW Government is set to underwrite billions of dollars of transmission and generation projects. The Victorian Government has announced that, if re-elected, it would renationalise part of the state's electricity system, spend $1 billion on building new generation and explore whether it can sell electricity directly to consumers. This echoes statements from the Queensland Government that public ownership of generating assets is the best way to deliver a smooth transition.

Governments have recognised, as they did during the 1960s, that they need to guide and coordinate investment in projects that provide a public service –particularly when they are major capital investments for which the private sector struggles to obtain investment approval.

By way of example, we set out in the two tables several recent and proposed rule changes and their general impact on VRE development.

This shift in market dynamics and potential increase in government involvement brings up a broader philosophical question about the nature of competition in the market. Some perceive that VRE is not legitimately competitive because most projects, up until recently, have relied on some sort of government support to connect. However, it must be remembered that most coal and gas plants in the NEM were entirely paid for with government support.

Under this orthodoxy, state and federal government interventions are seen as impure distorting influences that ruin an otherwise perfect market.

These are, as we see it, the big questions that governments, market operators, market rule changes, market proponents and consumers need to contemplate:

The power system comprises distributed generation (dispatchable, semi-dispatchable or self-dispatchable); short, medium, and long duration storage, aging and retiring centralised fossil fuel generation, increasing distributed energy resources (DER). Is it becoming too complex to allow markets alone to operate?

Is there a need for increased government intervention (as has always been the case – think Large-scale Renewable Energy Target and the Small-scale Renewable Energy Target)?

Do state governments need to continue to introduce central planning?

Is there a need to start from scratch and build a new energy market that doesn’t just contemplate centralised large-scale generation, transmission, and distribution single way power flow, but is designed for our future, complex system of distributed generation, DER, storage, and reverse power flow requirements?

1 CSIRO GenCost Database