The leadership of the United States describes victory as an Iraq that is a
"strong ally in the Middle East" defeat is described as a situation where extremists govern
"a new terrorist state in the heart of the Middle East with large oil reserves that could be used to fund its radical ambitions". But if Iraq is to be democratic, there is a big problem. No population in the Middle East will, given the choice, elect leaders who are US allies in the war against terror. No Middle East population will keep extremists out of power if they fairly choose their own leaders.
Victory in Iraq, as the US defines it, requires the US to have leverage in Iraq sufficient to overrule its voters to ensure that Iraq is an ally in the war, and to prevent "extremists" from being brought to power there democratically. Leverage in Iraq is nearly impossible to maintain without boots on the ground. So the US needs a military presence in Iraq to prevent defeat.
"Extremism" as used by the US in the context of the Middle East means disbelief in the legitimacy of Israel. No group that does not accept Israel's legitimacy, regardless of distinctions such as secular vs. Islamist vs democratic vs tyrannical fails to be described as extemist. No group that does accept Israel, regardless of any other beliefs or policies, is described as extremist. Following the US lead, I use extremism and non-acceptance of Israel interchangeably.
Enter the Kurds. Most Kurds live in Turkey, where Kurdish separatists have been Turkey's primary security concern for over a generation. A wealthy independent Kurdistan from what is now Iraq is a mortal threat to Turkey because there would be no way to prevent it from supporting Kurdish separatists in Turkey. Kurds are viewed similarly by, but are substantially less threatening to Iran and Syria.
The Kurds of Iraq though, landlocked and surrounded by neighbors of various degrees of hostility, can use a US military presence to ensure that the neighbors do not intervene it its affairs. Without that presence, for example, Turkey and Iran would follow the PKK back into its bases and Iraq and flush them out there. The Kurdish militias, good though they would be if put against other Iraqi militias, are entirely unequipped to fend off national armies of Turkey or Iran. Turkey would also intervene to ensure justice for both the Sunnis and the Turkmen in the area of Kirkuk - denying that city and the oil from that area to any Kurdish confederation or state.
The US needs the Kurds because without the Kurds, the US cannot keep a military presence in Iraq, which means the US is defeated when extremists are voted into power. Specifically, the US needs the Kurds to have enough independence that they can host a US and even an Israeli(!) military presence despite the fact that the Sunnis and Shiites of any central government are sure to vote against that. This requirement means that de-facto partition of the country, at least, is necessary to prevent a US defeat in Iraq.
Further than that, the Kurds know the US both needs their assent and intends to remain indefinitely. That gives the Kurds leverage in their dealings with other Iraqis that allows them to make demands that otherwise would be unreasonable. The Kurds want their mini-state to have the power to make oil exploration deals independent of the central government. They want to limit the amount of revenue that is sent back to Baghdad and they want Kirkuk ceded to their region.
The leverage they bring is that if the Kurds feel forced, they can declare independence, and the US is forced to support them, otherwise the US would be defeated in Iraq. Without the knowledge of US support, the Kurds would not have a credible threat to secede. Secession would bring immediate defeat at the hands of Iraq plus Turkey and Iran.
A relationship has formed between the US and the Kurds where the Kurds are free to ask for and get economic arrangments and political and military autonomy that represent a de-facto partition of Iraq in themselves. The fuel of this relationship is the US imperative to avoid defeat, to prevent the acquisition by extremists of unchecked power in Iraq and control of Iraq's resources.
I have earlier commented that a foreign invader alone can be enough to begin a civil war spiral when formerly illegitimate organizations become legitimate fighters of the occupation. I used the example of Crips and Michigan Militia in an invasion of Michigan. In the case of Iraq, that is a very small part of the story.
The US relationship with the Kurds, and the de-facto secession the US must support for the Kurds puts Iraq's Shiites in the position that they are nearly forced to secede themselves.
In Iraq, Shiite areas produce approximately 60% of the oil, Kurdish areas the other 40%. So the Kurds who are about 20% of the population, and have the most oil per capita are going to get to keep their oil. The Shiites, which much less oil per capita are to share theirs with the Sunnis, who have none.
The Shiites are not going to accept that. As much as Iran would prefer Iraq remain stable, there is no way, if Iraq is going to break up anyway, for them to force the Shiites to stay with the Sunnis.
And if Iraq is going to break up, that means there is a new importance for creating facts on the ground. The question of whether the Shiite parts of Baghdad will connect with the Shiite state to the South and similar strategic questions are the major motivation for the current fighting. This race to create facts on the ground is not due to ethnic hatred, nor it is the inevitable result of the defeat of the Baathists, nor is it proof that only a tyrant can rule Iraq. This race, and the violence it brings with it, are due to the US need to effectively partition the country to maintain a military presence.
If the US was willing to accept defeat in Iraq, willing to accept an extreme government having full control or Iraq's resources including oil, then the civil war could be wound down even now. It certainly was never inevitable.
The constitution was negotiated with the understanding that the US intended to maintain a long-term presence in Iraq. If the US was willing to leave now, it could be amended so that oil revenues are divided equally per-capita among all provinces. Kurdistan could maintain cultural, and linguistic independence but its foreign policy would be set in Baghdad. Under those conditions, the Shiites would also remain in Iraq. The Kurds could not refuse this arrangement if the US was willing to leave because if Turkey and/or Iran intervene, which they would be willing to do, the Kurds would end up with less.
But since the US is not willing to accept defeat, Iraq will be violently broken up. This violent civil war is not the unpredictable outcome of a poorly executed occupation. It is the only possible result of an occupation of Iraq by the US given the way the US defines victory and defeat.