Russian Foreign Minister Sergei Lavrov has warned that Europe could be returning to what he called the "nightmare of military confrontation".
At a European security conference in Sweden, Mr Lavrov floated the idea of a new European security pact to try to stop Nato from expanding further east.
US Secretary of State Antony Blinken warned of "serious consequences" if Russia sought conflict with Ukraine.
The meeting comes as Russia boosts its military near Ukraine's border.
Ukraine says Russia has amassed more than 90,000 troops there.
Moscow denies it is preparing an attack on Ukraine and accuses Kyiv of its own military build-up.
As tensions rise, Russia said on Thursday it had arrested three suspected Ukrainian security service agents.
One of the three was accused of planning a terrorist attack, while the other two had been seeking to gather intelligence, Russia's Federal Security Service said.
Mr Lavrov and Mr Blinken met at a conference of the Organisation for Security and Co-operation in Europe (OSCE).
US officials said the two sides had agreed to further dialogue on Ukraine.
Mr Blinken hinted at the possibility of direct talks between US President Joe Biden and his Russian counterpart Vladimir Putin in the near future.
Russia's foreign ministry hoped a Putin-Biden summit would take place in the coming days but no date had been set, Interfax news agency reported.
The build-up of Russian troops near Ukraine's border was among the topics discussed at an in-person summit between the two leaders in Geneva in June.
Earlier this week, President Putin said Russia would seek US guarantees that Nato will limit its presence in Ukraine, which is not a member of the security alliance.
But on Thursday, Mr Lavrov said in a speech that Nato was refusing to constructively consider proposals to de-escalate tensions and prevent dangerous incidents.
"The alliance's military infrastructure is being irresponsibly brought closer to Russia's borders in Romania and Poland, deploying an anti-missile defence system that can be used as a strike complex," he said.
"American medium-range missiles are about to appear in Europe, bringing back the nightmare scenario of a military confrontation."
He warned Nato against turning countries neighbouring Russia, ie Ukraine, into "bridgeheads of confrontation", and said he hoped Russian proposals for a new security pact would be carefully considered.
At a joint news conference with the Russian foreign minister, Mr Blinken said the best way to avoid a crisis was through diplomacy and urged Russia to pull back its forces.
"The United States is willing to facilitate that but... if Russia decides to pursue confrontation there will be serious consequences," he added.
On Wednesday, Ukrainian President Volodymyr Zelensky called for direct talks with Russia over the more than seven-year conflict with Russian-backed separatists in eastern Ukraine.
He also said that Kyiv's goal was to "liberate" Crimea, annexed by Russia in 2014, but made no mention of using force.
A large part of the recent Russian military build-up is in Crimea.
Troops are also massing near Ukraine's eastern Donbas region, the name for parts of Luhansk and Donetsk regions under the control of separatists.