Last month, Australia’s highest court refused to hear an appeal brought by conservative activist Bernard Gaynor. The reservist had his commission with the army terminated in 2013 for provocative public comments, and has since been engaged in a legal battle with the Australian Defence Force. Gaynor won in the Federal Court, lost in the Full Federal Court, and with the High Court declining special leave to appeal, the one-time Senate candidate’s litigious crusade is now over.

Some might view this latest development in a positive light – Gaynor’s political views are deeply offensive to certain sectors of society. Prior to his termination, he tweeted: “I wouldn’t let a gay person teach my children and I am not afraid to say it”; published a press release on “Defence’s gender-bending preoccupation” and issued another entitled “Government and Defence blinded on Islam”. More recently, Gaynor lobbied to remove an ADF imam and campaigned against halal certification. His comments often evoke One Nation senator Pauline Hanson, whose position on immigration he has endorsed.

But the High Court’s refusal to even consider an appeal from Gaynor should be deeply troubling, regardless of your political persuasion. The Full Federal Court finding that the ADF did not impermissibly encroach on Gaynor’s constitutionally protected free speech has broader ramifications. They may be well-worn to the point of cliché, but the words of English writer Evelyn Beatrice Hall are apt. “I disapprove of what you say,” she mused in 1906, “but I will defend to the death your right to say it.”

To elucidate the nuances of this topic, several preliminary observations are necessary. Firstly, Australians do not enjoy an American-style constitutional right to free speech; we have no First Amendment. But 25 years ago, the High Court “discovered” in the Constitution an implied protection for political communication. This, the judges argued, was a necessary implication from the system of representative government established by our founding document.

Yet such judicial activism created a shield, not a sword. The implied freedom has been consistently conceptualised as a limitation on legislative power, rather than an individually enforceable right. It was on this ground that the Full Federal Court accepted the ADF’s appeal: an error from the primary judge supposedly “led his Honour to look at [Gaynor’s] constitutional argument through an incorrect prism”.

Finally, that political communication may be deeply offensive does not lessen the protection it is afforded. “History,” a High Court judge once wrote, “teaches that abuse and invective are an inevitable part of political discourse.”

With these points in mind, the High Court’s disinclination to involve itself in the Gaynor case becomes all the more problematic. The comments that led to Gaynor’s termination were made while he “was not on duty, not in uniform and not doing anything connected with the ADF except criticising it and certain of its members”.

Similarities with the public sector employment context are striking. Earlier this month the Australian Public Service Commission published new guidance on social media use by public servants, continuing its attempt to limit the participation of government employees in political debate. The Full Federal Court’s decision against Gaynor buttresses these efforts.

In both contexts, the government is prosecuting legitimate objectives. The ADF is implementing a long-overdue process of cultural change within the Australian military, reform to which Gaynor was stridently opposed; the effectiveness of public servants depends on them acting impartially, and the APSC’s guidance is aimed at protecting that impartiality. But the dangers of silencing dissenting voices, when expressed as private opinion, are self-evident in a democratic society. Striking the right balance may not be easy, but it is certainly a task within the High Court’s remit.

The drawbacks with the status quo are demonstrated through a simple counterfactual: what if Gaynor had instead vocally praised the ADF’s approach? He would still be an army reservist. Indeed, this is what the APSC wants of public servants – its guidance stressed “this doesn’t stop you making a positive comment on social media about your agency.” But the contest of ideas – the Miltonian concept underlying much of the philosophical grounding for free speech – becomes moot if only one side can speak. Such “content-based” restrictions on expression should be subject to exacting scrutiny.

Whether we agree with their views or not, every member of Australian society should be able to engage in political debate without fear of retribution by the state. That includes public servants and those enlisted in the armed forces. In refusing to hear Gaynor’s appeal, the High Court has missed a significant opportunity to protect freedom of expression in this country. Our democracy is poorer for this omission.

The last word should go to the first instance judge in this lengthy saga, Justice Robert Buchanan. ”[Gaynor’s] commission was terminated,” Buchanan concluded, “because of the publication of his private views about political matters.” Last month, the High Court failed to appreciate the deeply troubling implications of that fact.