In August 2021, Britain’s last ambassador to Afghanistan, Sir Laurie Bristow, climbed on to a table holding a kitchen knife. He and a member of his security team had a small but important task. They unscrewed a portrait of the queen from the wall. Around them was “incessant gunfire”. Some of it came from heavy calibre weapons. A large TV screen nearby relayed the news on a loop. It was grim. The Taliban were at the gates of Kabul.

From time to time, Bristow recalls, there were great crashes from the roof as soldiers destroyed sensitive equipment. The UK was shutting its embassy and relocating to a military facility inside Kabul airport. Soon, soldiers and diplomats would depart. The US-led campaign in Afghanistan – a 20-year saga of wishful thinking and blunders – was ending in ignominy and farce. And, as Bristow describes in his compelling memoir Kabul: Final Call, with betrayal and human disaster.

The previous week, the Taliban had overrun a series of provincial capitals. Afghanistan’s foreign-backed republic was falling apart. That might have been predicted. In February 2020, Donald Trump announced the US was pulling its forces out. The Biden administration stuck to this decision and gave a deadline of September 2021 for the exit of Nato troops. Afghanistan’s government, many thought, might stagger on until Christmas.

Bristow arrived in Kabul in June 2021, as Afghanistan’s future looked precarious. He understood it might fall to him to shutter the embassy and evacuate staff, as well as Afghans who worked with British forces. What nobody had anticipated was the speed with which the situation unravelled. The Taliban – chased out in 2001, in the aftermath of 9/11 – controlled rural areas and key roads. In nine giddy days they reconquered the entire country.

What went wrong? As Bristow tells it, the west failed because of bad strategy and a loss of will. After the attacks on the twin towers, a military response from the US and its allies was inevitable. Its goal: to exterminate al-Qaeda. As a young reporter, I watched the Taliban’s northern army surrender outside Mazar-i-Sharif. The five-year-old emirate ended in “a wilderness of shimmering desert and telegraph poles”, I wrote in 2001. It was “vanishing into history”.

This prediction turned out to be wrong. After a period in Pakistan’s tribal regions, the Taliban returned. They waged a brutal and increasingly effective insurgency against international and Afghanistan government troops. The conflict cost billions. Meanwhile, George Bush’s administration invaded Iraq. A surge by the next US president, Barack Obama, didn’t bring results. By 2021, the public had lost interest in Afghanistan, seeing it as a for ever war with few benefits.

Washington and London’s mistake, in Bristow’s view, was to seek a military solution to what was a political problem. The international coalition also failed to address the “egregious” behaviour of its Afghan allies. Bristow portrays Afghanistan’s then president, Ashraf Ghani, as an aloof academic, surrounded by toadies. As the Taliban closed in, he gave speeches from his fortified Arg palace about digital governance. Afghanistan’s then defence minister, Bismillah Khan Mohammadi, was more clear-eyed; the Taliban tried to murder him. Overall, Afghanistan’s ministers were a corrupt and predatory bunch, beset by factional squabbles.

As the situation got worse, Bristow received media requests for interviews. He relayed these to special advisers working for then foreign secretary Dominic Raab. There was no reply. When Kabul fell, Raab was on holiday to Crete. Other senior mandarins were missing. Bristow doesn’t say much about the infamous campaign allegedly involving former prime minister Boris Johnson’s wife, Carrie, to rescue Afghan cats and dogs. But he notes that “the priority of some in London” was to spare senior figures from “embarrassment”.

Kabul: Final Call is full of glorious details. Bristow was previously ambassador to Moscow. He swapped a 19th-century mansion overlooking the Kremlin for a Barratt Homes-like residence in Kabul. It had a shiny grand piano and a garden with a small lawn and mynah birds. Before flying out, Bristow picked up a flak jacket and helmet. During security alerts, he took cover in a sweaty armoured “wardrobe” and leafed through a back number of the Economist.

In the run-up to Kabul’s fall, there were numerous ominous signs. Rockets thudded into the green zone. The French ambassador sent home his chef. After relocating to the airport, Bristow found himself in a “real-life” version of Apocalypse Now, as desperate Afghans tried to board a plane. Thousands besieged the perimeter. The Baron hotel – used by the UK as an evacuation handling centre – became a chaotic refugee camp. He spoke to Downing Street from a laptop propped up on the bar.

In contrast to his masters in Westminster, Bristow comes across as decent, serious, analytical and quietly heroic: a brave public servant doing a tough job. He pays tribute to the young British soldiers sent to guard the airport’s gates. And to his staff, who had to inform Afghan families they did not qualify for evacuation. “You are told in return that you are condemning them to death,” he recalls. It was a harrowing experience, he writes, that haunted those involved.

Some hoped the Taliban who seized Kabul in 2021 might be more moderate than their hardline predecessors. This, it turned out, was a delusion. The Afghans, by the Norwegian writer and journalist Åsne Seierstad, tells the story of what happened next, after the last flights carrying Bristow and his US counterparts took off. She traces the lives of three Afghans: a Taliban commander, a young female law student and a prominent women’s rights activist.

Despite earlier assurances, the Taliban closed down girls’ secondary schools, banned women from the workplace and ordered them to cover up. They were forbidden from travelling farther than 45 miles (72kms) without a male guardian.

Seierstad, the bestselling author of The Bookseller of Kabul, gives a heartbreaking account of the first day of school. Senior girls wearing regulation black dresses and white headscarves arrived, keen to learn. They had “expectant faces”. The same morning, the Taliban’s education ministry ordered female secondary schools to remain closed. In some provinces, Taliban fighters stormed classrooms, beating girls with rods and berating them. In 2022, women were banned from university.

It is Afghan women and girls who are paying a terrible price for our missteps. Since the Taliban took power poverty, hunger and infant mortality have soared. Bristow argues we need a “proper reckoning” of why the campaign in Afghanistan ended the way it did. A lack of resolve contributed to Vladimir Putin’s decision to invade Ukraine, he thinks. The only “glimmer of light” he sees comes from courageous Afghans who defended their right to live freely and in peace.

Luke Harding’s Invasion: Russia’s Bloody War and Ukraine’s Fight for Survival, shortlisted for the Orwell prize, is published by Guardian Faber (£10.99)

 Kabul: Final Call: The Inside Story of the Withdrawal from Afghanistan August 2021 by Laurie Bristow is published by Whittles Publishing (£18.99). To support the Guardian and Observer order your copy at guardianbookshop.com.