The council voted 12 to two, with one abstention, in favour of a fourth round of sanctions, including tighter finance curbs and an expanded arms embargo.
The US welcomed the move and said Iran must choose a "wiser course".
But an Iranian foreign ministry spokesman said on Iranian TV the decision was an "incorrect step" that would "complicate" the situation.
The US and its allies fear Iran is secretly trying to build a nuclear bomb, but Tehran insists its programme is aimed solely at peaceful energy use.
Heavy weapons
The Security Council resolution was opposed by Turkey and Brazil. They had earlier brokered a deal with Iran on uranium enrichment. Lebanon abstained.
The new sanctions were passed after being watered down during negotiations with Russia and China on Tuesday.
There are no crippling economic sanctions and there is no oil embargo.
Those passed include prohibiting Iran from buying heavy weapons such as attack helicopters and missiles.
They also toughen rules on financial transactions with Iranian banks and increase the number of Iranian individuals and companies that are targeted with asset freezes and travel bans.
Hailing the vote, the US ambassador to the UN Susan Rice said: "The Security Council has risen to its responsibilities and now Iran should choose a wiser course."
UK Foreign Secretary William Hague said the decision sent a "strong statement of international resolve", increasing the pressure on Iran to meet its obligations.
However, both Turkey and Brazil spoke out in opposition, saying the deal they had brokered with Iran had not been given time.
Brazilian Ambassador to the UN Maria Luiza Ribeiro Viotti said: "We do not see sanctions as an effective instrument in this case. They will most probably lead to the suffering of the people of Iran and will play into the hands of people on all sides who do not want dialogue to prevail."
Turkey's envoy Ertugrul Apakan said the Turkey-Brazil deal had created "a new reality" on Iran's nuclear programme and Turkey was "deeply concerned" that sanctions would have a negative impact.
The BBC News website's world affairs correspondent, Paul Reynolds, says this new round of sanctions is unlikely to have any more effect on Iranian policy than the first three.
Iran's vital economic interests have not been targeted, he says, and Tehran has in any case developed systems of evasion.
Iran had tried to ease international concerns by agreeing the deal with Turkey and Brazil. It would see Tehran deposit 1,200kg of low-enriched uranium with Turkey, in return for reactor fuel.
But the deal has not been accepted by world powers and on Wednesday, the US, Russia and France outlined their concerns in letters to the IAEA.
The letters were not made public, but US envoy to the IAEA Glyn Davies said the deal "would still leave Iran with substantial stocks [of low-enriched uranium], decreasing the confidence-building value of the original proposal".
Three earlier rounds of UN sanctions blocked trade of "sensitive nuclear material", froze the financial assets of those involved in Iran's nuclear activities, banned all of Iran's arms exports and encouraged scrutiny of the dealings of Iranian banks.
