In the first section, a wall of about "a dozen kilometres, 3.5 metres high, was built in Nineveh province", in the Sinjar area of north-west Iraq, a senior officer told AFP.
Iraq, whose border with Syria is more than 600 kilometres long, will use the wall to stop ISIS members infiltrating its territory, the source said, without giving details of how long the wall would be.
Iraq in 2018 said it had begun building a fence along the Syrian border for the same reason.
The Syrian Observatory for Human Rights, a war monitor, said the wall was being built in an area facing the town of Al Shaddadi, in the south of Syria's Hasakeh province.
Many prisoners are thought to have escaped, with some crossing to Syria's north, the Observatory said.
ISIS overran large parts of Iraq and Syria in 2014, declaring a "caliphate". Baghdad declared victory over the extremists in late 2017 after a grinding campaign.
But a low-level insurgency has persisted, flaring up particularly in rural and mountainous areas between Iraq's autonomous Kurdistan region and northern outskirts of the capital.