The ALP’s original purpose, the mass participation of working men and women in parliamentary democracy, has dissolved. Very few unionists are still active in the party. This is clear from statistics released by the former NSW assistant general secretary Luke Foley. Only 16 per cent of party members belong to a trade union affiliated to the ALP. By far the biggest category (55 per cent) is concessional membership, that is, people outside the workforce, mainly retirees.

When extrapolated to the 2011 national presidency vote, Foley’s figures expose the emptiness of Labor’s structure. Of the party’s 11,665 members participating in the ballot, only 1860 (16 per cent of the total) were likely to belong to a Labor-affiliated union. This represents, on average, just 12 trade unionists in each federal electorate – not enough for a rugby team.

As a party of working class activism, Labor now resembles a Hollywood back lot. There’s nothing behind the facade. Walk through the doors of the main street saloon and you find yourself wandering around an empty car park. Grassroots union activism in Australia has ended and, with it, the energy which once powered political Labor has faded. As ALP historian and former NSW minister Rodney Cavalier has recorded.

Contested union ballots, like attendances at union meetings, reveal how very few members take the slightest interest in the affairs of their unions. The proportion of members of affiliated unions who belong to the ALP is fewer than 0.5 per cent. Belonging to the ALP is not part of the life of a modern Australian worker.

In organising Labor Party numbers, it is many decades since union officials tried to recruit unionists into the local branches. Union power is now exercised through centralised control: union secretaries donating money and staff to marginal seats and rounding up the numbers at state and federal Labor conferences. The grand old party of working class participation has become a virtual party, top heavy with union/factional bosses. In no other part of society – whether in the corporate sector, community groups or government agencies – could an organisation function this way and expect to survive. This is the core delusion of 21st-century democracy, that political parties can fragment and hollow out, yet still win the confidence of the people.

For a party of reform like Labor, the condition is doubly debilitating. Social change needs to be a bottom-up process, with reformers drawing ideas from the community, testing those ideas at a local level and using them in the development of national policy. Labor has always been strongest when it has followed this technique: advancing a reform agenda but not getting too far ahead of public opinion.

Rediscovering this process in an era of widespread public disengagement from party politics is a difficult task, but one essential to Labor’s future. It is the key to effective organisational reform. Otherwise the party will continue to shrivel, living off the glories of the past but unable to attract and inspire the next generation of social reformers.

Labor’s identity crisis

The corrosion of Labor’s culture has produced a crisis of Labor identity. The party is confused on economic policy, not knowing whether to embrace former prime minister Paul Keating’s legacy of micro-reform and productivity growth or to accede to the sectional demands of union/factional bosses and the anti-competitive comfort of industry welfare. As union coverage tends to be highest in old, declining industries such as smokestack manufacturing, the demand for protection and subsidisation is disproportionately strong inside the Labor movement. This is at odds with Keating’s creation of a “miracle economy,” delivering 20 years of uninterrupted growth and wealth creation through the liberalisation of economic policy.

The union oligarchy is also at odds with the rise of Australia’s new aspirational class: the free agents of the new economy (the start-up entrepreneurs, contractors and information-rich specialists) who resent the intervention of outsiders, whether in the form of excessive government regulation or trade union collectivism.

This is a problem not just in economic management, but also the development of social policy. Given a choice between traditional left thinking – reflected in the paternalism of welfare-state programs – and the aspirational demand for individual entitlements and flexibility, Labor still sides with the former.

In combination these problems have damaged the party’s electoral standing. The aspirational class is the fastest-growing part of the electorate, yet much of the ALP’s policymaking is still directed towards a declining working class constituency. The underlying voting trends in recent years have been alarming.

Generations of Australians who grew up with the assumption of an evenly balanced two-party system have had to recast their thinking. In March 2011, the Labor primary vote at the NSW election fell below 25% – a figure more befitting an interest group than a mainstream party.

Twelve months later in Queensland, Labor’s parliamentary representation was reduced to the size of a netball team.

I should say something about my motivation in writing this essay. It comes from an obligation to correct a lingering personal failure. Along with Rudd and Julia Gillard, I was a Labor leader drawn from the post- Keating generation of parliamentarians. We were tasked with revitalising the party’s agenda after federal Labor’s heavy defeat in 1996, but in large part, nearly two decades later, this goal remains unfulfilled.

I had a short period in the job, just fourteen months. Nonetheless, I made no lasting progress in the development of new Labor thinking. Looking back, I came to the leadership too young (at forty-two years of age), with too little life experience (not yet having built a home and raised a family) and with too much of my policy thinking still a work in progress. I had done a lot of writing about new ideas for the party, but very little of it was settled in the public arena – that is, tested over time and able to withstand criticism, both from inside the Labor movement and by political opponents.

Rudd and Gillard, in their differing ways, have done much better. Rudd delivered an exhilarating election victory in 2007 and, for several years, inspired a generation of true believers to believe again. He is a highly intelligent person with a phenomenal work ethic – the sort of leader who could have successfully addressed Labor’s structural challenges. Unhappily, he failed in this task because he put public popularity ahead of the development of public policy.

The legitimacy of Rudd’s leadership rested entirely on his standing in the opinion polls. When this dissolved in the first half of 2010, his caucus colleagues, most of whom could not abide the man personally, removed him from the prime ministership. He left office with a newly discovered contempt for Labor’s factional system, the so-called faceless men. More tellingly, the key policy issues of his leadership remained unresolved: he failed to develop an effective framework for dealing with climate change, mining taxation and border protection. This was a terrible waste of Rudd’s talent – a tragedy made worse by his subsequent role as Labor’s parliamentary Destabiliser-in-Chief.

Gillard’s great skill has been on the practical side of government: negotiating successfully with the independents after the 2010 election, navigating a series of legislative achievements through a hung parliament and holding together the disparate sub-factions and fiefdoms of Labor’s caucus. Her methodology is that of a transactional leader, a tough-minded doer, rather than a policy wonk or inspirational figure. There is little evidence of Gillard wanting to tackle Labor’s identity and structural problems, of her leading a renewed push for party reform. In large part, this is a product of the fraught political environment in which she finds herself. She is the modern equivalent of the eighteenth-century clergyman and pamphleteer Emmanuel Joseph Sieyès, who, when asked what he had done during the French Revolution, replied, “I survived.”

Last chance reform

One of the catchcries of the thoroughbred breeding industry, in which I take a keen interest, is “You only need one” – meaning that one good galloper can set up a breeder for life, increasing the value of young horses related to the star performer. The Labor Party is no different. Its most successful periods of reform are characterised by one inspiring leader, one creative spark for policy-making success. Invariably, the spark has come from the leader’s personal experience, from perspectives and ideas gleaned from grassroots political participation.

The party’s four great reformers, John Curtin, Ben Chifley, Gough Whitlam and Paul Keating, each followed this pattern: the young Curtin crusading for social justice with the Salvation Army on the working-class streets of Brunswick, Melbourne; Chifley, first and foremost a son of Bathurst in central NSW, a servant of the local community, not just as a federal MP, treasurer and prime minister but also as an Abercrombie Shire councillor; Whitlam, raising a young family in Cabramatta in Sydney’s south-west, gathering ideas for education, health and urban policy from his electorate; and finally, the attentive young Keating, bruised by his father’s frustration in trying to raise bank finance for the family’s small manufacturing business – a personal catalyst for the deregulation of Australia’s financial system in the 1980s.

You only need one. One quality leader to emerge with a bundle of policies and know-how derived from hands-on experience in the electorate. This is, from the time of its founding, how the ALP is supposed to function – giving working people from the grassroots of politics a voice in the legislative process. The ideal was for mass participation: every working person a unionist, every unionist a branch member.

While in the modern era, the ideal is no longer attainable, the policy leadership process can be. No one expects large numbers of people to join parties anymore – apathy and busy lifestyles have ended this possibility. At a local level, the remaining political energy for Labor now revolves around ambition: young people and established activists getting involved, hoping to progress through the party and become members of parliament. Labor needs to harness this personal drive and activity, using it as a way of forcing pre-selection hopefuls to embed themselves in community politics.

The pathway to parliament should not be through union- factional patronage. It should be based on the Curtin / Chifley / Whitlam / Keating precedent: a capacity to work with local organisations, learning lessons about the public’s needs and converting this experience into Labor policy. Primary pre-selections are the best mechanism for re-establishing this culture. They ensure that nobody can become a Labor candidate without forging a reputation for community engagement, and winning the trust of branch members, community groups and progressively minded residents.

The party has lost a lot over the past 30 years. All it has left is the ambition of new people joining. Unless it makes good use of this motivation, Labor’s membership crisis will continue to worsen. New South Wales general-secretary Sam Dastyari’s vision for primary pre-selections can heal the party’s organisational split: the party above ceding power to branch members and community voters in the selection of parliamentary candidates. This would shift Labor’s centre of gravity, taking the arrogance out of the factional system and emphasising the importance of community-based policy development.

Centralised candidate selection by state executives and the national executive would end, including for upper-house members. There is no reason why senators and state members of legislative councils could not be chosen through 50/50 primaries, cleaning out the sub-factional rotten boroughs. As the union / factional system has intensified, the quality of ALP upper-house members has deteriorated. The first priority for these MPs is factional organisation, ignoring the importance of policy work. Their contact with the electorate tends to be desultory, weakening the party’s community credentials. With upper-house candidates subject to state-wide primaries, it would be possible for state branches to ballot for lower-house seats on the same day – a Super-Saturday effect.

In establishing this system, I would add an extra stage to the primaries process: the vetting of candidates to encourage genuine community involvement. One of the dangers of primaries is poorly motivated candidates who circumvent the system. An obvious risk is the targeting of ethnic groups with enticements and sweetheart deals – a backdoor form of ethnic branch-stacking. Excessive campaign expenditure could be another problem, as per the US experience. Thus the primaries should operate according to the following structure and timetable:

• Nine months before the scheduled primary, the party calls for expressions of interest for a particular seat.

• Those who nominate go through an assessment process by a panel comprising party elders (members respected across factional lines, such as Faulkner and Iemma), local branch office-bearers, a Labor Council representative and a state branch official.

• In determining eligibility to contest the primary, the panel applies certain criteria (a track record of local community engagement, public speaking ability and policy-making credentials) and sets certain tasks (such as organising public meetings on local issues and presenting policy papers to Labor think-tanks). The objective is to maximise the party’s community presence and gauge the ability of candidates not only to win their seats, but also to serve as parliamentary frontbenchers.

• The vetted candidates are announced two months before the primary ballot, giving them ample time to run for the position (within strict campaign-expenditure limits).

There are other things Labor can do to improve its participatory credentials. Many of these ideas are set out in the Faulkner / Bracks / Carr report – such as allowing like-minded organisations (not just trade unions) to affiliate to the party, directly electing National Conference delegates and state branch presidents, and adopting a community-organising model for local branches. While helpful in opening up party forums, these are not culture-changing proposals. The point of last-chance reform is to recognise how the long-term decline of an organisation can only be fixed by transforming its structure and ethos. Given the impossibility of breaking the union nexus, Labor’s last hope is the introduction of primary pre-selections.

A post-Left electorate

The authentic Labor tradition is not one of grand theories and dogma. It is a tradition of community engagement – the legacy of Curtin, Chifley, Whitlam, Keating and, at state level, McKell. This is an empirical style of policy development, responding to the needs of people who look to Labor for better opportunities in life. If any of these leaders represented a country town or outer-suburban constituency today, what would they find?

Compared to their period in Parliament, they would notice an all-encompassing change. That is, the disintegration of the traditional institutions of the Left: organised labour, working class solidarity and mass meetings of a political nature. There is no longer any central mechanism for bringing together progressively-minded people in local electorates. If Labor is to have a future, it must be a post-Left future. It can no longer rely on the politics of mass scale and common membership.

Electoral opinions and interests are now highly dispersed, spread across individuals, diverse family types and small-scale community groups. Majority beliefs still exist, of course, but in a notional way, without the organisational rallying points of the past.

Few community bodies can claim to speak on behalf of a significant proportion of citizens. With higher levels of education and better access to information, people are more likely to make up their own minds on issues, bypassing the views of interest groups and media commentators.

The most pervasive public belief is economic aspiration. The sons and daughters of the working class have had a taste of financial success and they want more. In this task, they do not expect governments to control economic outcomes, but rather to foster an environment in which individuals, through hard work and enterprise, can advance themselves. Flexible and effective service delivery is all-important as people juggle the demands of work, education and family.

These values are underpinned by an earthy, no-nonsense approach to politics. If the two-party system appears negative and overly adversarial, people lose interest and disengage. If parents see teachers struggling to achieve results in their local public school, they promptly switch to the non-government sector. The most significant aspiration of all is for the education of the next generation.

If taxpayers see evidence of welfare recipients taking advantage of the system, they want governments to take a tougher approach. If local residents experience problems with ethnic crime gangs, they become sceptical about the composition of Australia’s migration program.

On the big sweeping claims of public life such as the impact of climate change, people want hard evidence: expert information which corresponds with their own experience. Big ideas are still possible in politics, but first they must prove their credibility and relevance on a smaller scale.

A similar type of empiricism is applied to social values. Some of the staples of conservative politics are in decline. After several generations of marriage failure and patched-up family arrangements, society is far more tolerant of diversity in the home. The traditional nuclear family model, with its white picket fence imagery, is no longer the only acceptable family structure. Gay relationships, unmarried couples and shared parenting (in which children from broken relationships move between one home and another) have a legitimacy which was unthinkable 30 years ago.

Labor needs to be a conduit for these majority values. As a party of electoral politics, it has no choice but to move with the electoral tide. Despite the wishes of many on the Left, it is not possible to vote in a new electorate. The challenge for Labor is to shape, through community engagement, public opinion for egalitarian purposes.

In some areas such as family values, social trends are moving in its favour. In other fields, most notably economic aspiration, the party needs to rethink its policies and the way it communicates them to the electorate.

This is a forgotten aspect of the Whitlam-Keating leadership model: the power of persuasion. Whitlam spent his years in opposition arguing and re-arguing the case for reform, such as universal health insurance and needs-based school funding.

Similarly, Keating saw himself not just as an economic reformer but also as a public educator, explaining the workings and advantages of open-market economics. Contrast this approach, for instance, with the current government’s precipitate introduction of a carbon tax and the political problems that followed.

This is where Labor’s culture needs to change. Inside the party, factional power-brokers have grown accustomed to telling people what to do. When applied outside the party, this technique is inadequate – setting back the reform cause with its overtones of arrogance and deceit. The skills of argument and public persuasion have been lost. Among the many union officials who have become Labor senators, some can barely string two sentences together.

This is another compelling reason for the introduction of community pre-selections: ensuring the party’s best debaters and persuaders can win seats in Parliament.

Accepting the Keating settlement

Among the mistakes of modern Labor, none has been more frustrating than its failure to capitalise on the Keating economic legacy.

In the history of the Labor movement, one can readily draw up a long list of policy failures. White Australia, industry protection, population decentralisation, broad-acre public housing estates, indigenous welfare policy and Rudd’s education revolution come to mind.

It is far harder to identify enduring achievements – policies which were so correct, so successful that not even Coalition governments dismantled them. Here one thinks of Medicare, engagement with China and some of the Hawke government’s education initiatives (such as lifting school retention rates).

This is what makes Labor’s ambivalence about Keating’s program so bizarre. Among the achievements of ALP governments since federation, the 1983 to 1996 reforms stand out: creating the policy settings for 20 years of low inflationary growth and unprecedented wealth creation.

For the first time, suburban workers grabbed a significant slice of Australia’s economic expansion. They were able to move into better jobs or obtain capital funding and start their own enterprises.

The National Centre for Social and Economic Modelling (NATSEM) has calculated that since 1984, real disposable incomes in Australia have increased by 20 per cent – leaving the average family $224 per week better off in real terms.

Importantly, this rising tide has lifted all boats, as measured by income. As the 2012 NATSEM report concluded:

A particularly surprising result is that, in terms of percentage gain over the cost of living, we find the lowest income households have actually managed a small claw-back compared to the highest income group. Quintile one income growth (the lowest 20 per cent band) exceeded the cost of living over the past 27 years by 27 per cent, while the top income quintile outpaced (living costs) by only 19 per cent.

Given that welfare recipients experienced an 11.9 per cent increase, the result for wage earners in the bottom income quintile was very strong. In the other income bands, there is further evidence of the equalising impact of Keating’s reforms. The second quintile (20-40 per cent of income levels) experienced a large gain in real disposable income (30.5 per cent), while quintiles three (19.5) and four (14.6) also improved, but by smaller amounts.

For all the commentary about “Howard’s battlers” and Rudd’s “working families under financial pressure”, the statistical evidence suggests a different dynamic. When it came to raising living standards for the lowest paid, Keating’s role was paramount, overshadowing the contribution of his two prime ministerial successors.

This is what Labor is supposed to achieve: to lift up the bottom, to break down class barriers and to make social mobility possible. Not surprisingly, this process has ignited further aspiration, even greater ambition for the economic success of the next generation.

What was the old working class supposed to do, having enjoyed more money in its kick? Go back to low-paid factory work and rented fibro cottages?

The practical argument against Keating’s achievements is nonsensical. How can any of Labor’s true believers campaign against such a stunning record of success?

Yet this is what large sections of the Labor movement have been doing. When Keating retired from Parliament, having set up Australia’s miracle economy, he was sceptical about the durability of free-market economics inside the party. “I got all those changes through,” he told me in his retirement years, “but our people never really believed it, they never really believed in markets and competition.”

He was right. One of the first decisions of Kim Beazley and Simon Crean in opposition was to distance themselves from Keating economics. “Labor Buries Keating,” the Business Review Weekly screeched in March 1997, as Crean heralded a return to industry planning and subsidisation. This was an electoral gift to the Howard government, in its declaration that the ALP no longer saw itself as responsible for national economic outcomes and reform. A legacy once so abandoned is doubly difficult to retrieve.

The record of the Rudd and Gillard governments has been no better. Since 2007, two major statements of economic ideology have been issued, both in the Monthly, both deviating from the Keating ethos of competition and deregulation. In February 2009, Rudd declared that:

The great neoliberal experiment of the past thirty years has failed … the emperor has no clothes. Neoliberalism, and the free-market fundamentalism it has produced, has been revealed as little more than personal greed dressed up as an economic philosophy.

The NATSEM figures render such a claim ridiculous. Far from walking the streets naked, Australia’s neoliberal reforms have been a sartorial success. Due to reductions in tariffs, average prices for footwear and clothing in Australia have fallen since 1984 – making them more affordable for emperors and citizens alike.

With the passing of time, it is too easily forgotten how pompously wrong Rudd was in his analysis of the global financial crisis. Much like a Hegelian theorist, he opined in the opening of his essay that:

From time to time in human history there occur events of a truly seismic significance, events that mark a turning point between one epoch and the next, when one orthodoxy is overthrown and another takes its place. The significance of these events is rarely apparent as they unfold: it becomes clear only in retrospect, when observed from the commanding heights of history.

Four years later, history’s verdict is still clear: open, free markets work better than any leftist alternative.

The second essay, by the Treasurer, Wayne Swan, in March 2012, launched a populist critique of market economics and its leading beneficiaries – in particular, the Australian mining billionaires Clive Palmer, Andrew Forrest and Gina Rinehart.

Swan’s polemic was the perfect expression of an age-old Labor dilemma: how to regulate and restrict capitalists without harming the economic prospects of the workers they employ.

In his day job, the Treasurer is responsible for making mining and other companies more profitable. It’s what we call the national economic interest.

Yet as a Labor partisan, never truly comfortable with a system of profit-based capitalism, Swan wants to damage Palmer, Forrest and Rinehart. The two goals, of course, are incompatible. Whether we like it or not, what’s good for capital investment in the economy is also good for employment levels, wages and working conditions.

Quarterly Essay

Mark Latham is a columnist for the AFR. He was leader of the Australian Labor Party and leader of the federal opposition from December 2003 to January 2005.

This is an edited extract from Quarterly Essay 49, Not Dead Yet: Labor’s Post-Left Future by Mark Latham, to be published on Monday March 11. Also available as an ebook. www.quarterlyessay.com

Labour Gets Left On The Outer