Few towns in the US brim with as strong a sense of place as New Orleans. Its centuries-old French Quarter boasts density and antiquity where the average American city is diffuse and facelessly modern. Sat in the swampy delta where the Mississippi river meets the Gulf of Mexico, New Orleans has had more in common with old Caribbean emporiums like Havana and San Juan than other American cities. Locals readily speak of the particularity of their city, its mix of cultures, its famous music traditions and Creole and Cajun cuisine, its tangled history of French and Spanish colonialism, British invasion, and slavery.

I was in New Orleans recently, my visit coinciding with a festival in the French Quarter. Troupes of college students roamed the narrow streets, stuffing their faces with crawfish sausages while drinking tankards of ‘hurricanes’ (a suspiciously bright red cocktail). Brass bands and fiddlers occupied every street corner. A gaunt, elderly woman flung bead necklaces down from a wrought-iron balcony. These were glimpses of the New Orleans of cliché, a city whose customs are now well packaged for tourists. Visitors queued outside ‘Napoleon’s House’, a beautiful building that might have hosted Napoleon if privateers from New Orleans had succeeded in their scheme of freeing the French emperor from British captivity and bringing him to the Americas.

The city’s history is not only the source of amusing trivia. A curious sign hung on the windows of various boutiques: ‘Save Our Monuments.’ It protested the decision of the city council, led by the left-of-centre mayor Mitch Landrieu, to dismantle four public memorials. Three of the monuments are dedicated to figures from the American Civil War who fought on the side of the pro-slavery Confederacy — the generals Robert E Lee and PGT Beauregard and the Confederate president Jefferson Davis. The fourth commemorates an attempted coup in 1874 (nearly a decade after the end of the Civil War) by a paramilitary group composed of former Confederate soldiers. All these monuments were erected in the late-19th and early-20th century, a period that saw the consolidation of racial segregation and discriminatory laws against blacks in New Orleans and the wider American south.

The council verdict has sparked a legal and cultural battle about how the past is remembered in New Orleans. Supporters of the decision insist that the monuments celebrate a dark and poisonous history. The memorials take pride in the legacy of the Confederacy, which waged a rebellion in order to maintain the barbaric institution of slavery. “We, the people of New Orleans, have the power and we have the right to correct these historical wrongs,” Landrieu said in December. “The Confederacy was on the wrong side of history and humanity.” The mayor’s opponents — and there are many — argue that history is history and that it does no good to suppress the symbols of the past in the present.

Predictably, debate about the fate of the monuments is racially coded. Though Landrieu is white, black lawmakers and activists back his efforts. For over a century, blacks in New Orleans have had to walk beneath lofted statues of men who strove to preserve their enslavement. Defenders of the monuments tend to be white. They don’t see the heroes of the Confederacy as agents of the subjugation of blacks, but rather as signs of the unique cultural heritage of the American south, a region that still defines itself against the ‘Yankee’ north. Many white New Orleans residents feel that the city council’s decision strikes directly at their sense of identity.

I’m torn by these debates. On one hand, those who want to get rid of these statues ignore a possible teaching moment. Monuments reveal as much about the people who make them as the people they represent. The Confederates and their segregationist progeny were undoubtedly in the wrong. But instead of knocking it down, the citizens of New Orleans could reinterpret a statue honouring a Confederate general as a window into the hateful society that thought him worth honouring.

On the other hand, public monuments seek to represent a society’s conception of itself. By taking the Confederates off their pedestals, the city of New Orleans would send a strong and overdue message. Over 150 years after the end of the American Civil War, it would finally recognise that blacks have as much a stake in the civic life of New Orleans as whites do, that their memory of the past cannot be subordinated to the historical memory of whites.

I’m wary of the impulse to demolish and erase, but in this instance I think the city council of New Orleans has it right. Its move to take down the monuments actually makes New Orleans a more inclusive place, acknowledging that the landscape of the city belongs to all its citizens.

This isn’t only a dilemma pertinent to New Orleans. Americans are still wrestling over other symbols from the Civil War, especially the Confederate ‘battle flag.’ In recent months, students at Oxford University and Princeton University have protested against the ennobling on their campuses of Cecil Rhodes and Woodrow Wilson respectively, both early-20th century leaders with deeply racist views. In India, too, memorials to historical figures are often the subject of great contestation (and cynical politicisation).

Such debates often invite scholarly contributions in puzzling out the real meaning of a historical figure (for example, how horrible, really, was Rhodes?). But they aren’t actually about the past. Battles over history show us how we see ourselves in the present and who we imagine ourselves to be.