Dhaka had wanted to begin its long-held plan this month to move 100,000 people to the mud-silt island of Bhashan Char, amid growing frustration with the presence of the squalid tent settlements in its southeastern border towns.
Bangladesh has said thousands of Rohingya families have volunteered to relocate, with some 3,500 of the Muslim minority due to be moved between mid-November to February during calm seas.
But the plan was in doubt as the U.N. has not supported the relocation so far, Bangladesh disaster management and relief minister Enamur Rahman told AFP.
"This has become uncertain," Rahman said of the relocation to the island, which takes around three hours to reach by boat.
"They (U.N. agencies) still haven't agreed to the relocation plan."
Aid agencies including the UN refugee agency UNHCR, the International Organisation for Migration (IOM) and the World Food Program (WFP), which held meetings with the government, told him the island was "isolated" and "flood-prone".
The agencies set out a list of conditions that had to be met, including a regular shipping service between the islet in the Bay of Bengal and the mainland, Rahman added.
The organisations provide humanitarian aid to the nearly one million Rohingya in the vast camps, including 740,000 who fled a military crackdown in Myanmar in August 2017.
"We won't do anything forcefully," he said, adding that at least two ships were set to ply the waters between the site and the mainland.
A U.N. official told AFP on Sunday that "U.N. agencies cannot support a move for which (they) have no technical information".
Dhaka is due to hold another round of talks with the agencies on Wednesday, Rahman said.
Global activist group Fortify Rights said last month it interviewed 14 Rohingya at three camps, including some who appeared on lists of refugees allegedly willing to go, and found none had been consulted "and all opposed it".
Other groups have also expressed misgivings about moving people to the island, which is regularly hit by devastating cyclones.
The agencies set out a list of conditions that had to be met, including a regular shipping service between the islet in the Bay of Bengal and the mainland.