The Arch of Triumph was "pulverised" by the militants who control the city, a Palmyra activist told AFP news agency.
It is thought to have been built about 2,000 years ago.
IS fighters have already destroyed two ancient temples at the site, described by Unesco as one of the most important cultural centres of the ancient world.
"The Arch of Triumph was pulverised. IS has destroyed it," Mohammad Hassan al-Homsi, an activist from Palmyra told AFP on Monday.
The London-based Syrian Observatory for Human Rights, a group monitoring the conflict, said sources on the ground had confirmed the destruction.
Syrian antiquities chief Maamoun Abdul Karim also confirmed the news, and told Reuters news agency that if IS remains in control of Palmyra, "the city is doomed".
Unesco's director general Irina Bokova has said the destruction constitutes a "war crime" and called on the international community to stand united against IS efforts to "deprive the Syrian people of its knowledge, its identity and history".
IS believes shrines or statues represent idolatry, and should be destroyed.
In August, the group destroyed the ancient Temple of Baalshamin - one of the city's best-known buildings built nearly 2,000 years ago.
The group has also published photos of militants destroying what it said were artefacts looted at Palmyra.
IS militants captured the historic site from Syrian government troops in May, amid a series of setbacks for forces loyal to President Bashar al-Assad.
Last week, Russia, one of Mr Assad's key backers, launched air strikes in Syria, saying it was targeting IS and other extremist groups.
But members of the US-led coalition that has also been carrying out air strikes in Syria have strongly criticised the move, saying Russia appeared to be targeting more moderate anti-Assad rebels than IS.
Turkey, one of the countries to criticise Russia's intervention, said it had intercepted a Russian warplane on Saturday in Turkish airspace, and had summoned the Russian ambassador in protest.
Syria's conflict, which began in 2011, has left more than 250,000 dead and about half the country's population displaced.
Posted by Women Against Shariah on Monday, October 5, 2015