Around 400 troops fled the Kharg after it caught fire in unclear circumstances, state media reported, with 20 suffering injuries.
The British-built ship was named after the island that serves as the main oil terminal for Iran.
Firefighters battled for almost 21 hours to save the vessel before it sank at around 8:30 a.m. local (1 a.m. ET) on Wednesday morning near the Iranian port of Jask, some 790 miles southeast of Tehran on the Gulf of Oman near the Strait of Hormuz — the narrow mouth of the Persian Gulf.
Iranian officials offered no explanation about what started the blaze, which began at around 11 a.m. local time (2:30 a.m. ET) on Tuesday, the semiofficial Fars news agency reported. However, they said an investigation had begun.
Thick, black smoke could be seen rising from the ship — one of a few vessels in the Iranian navy capable of providing replenishment at sea for its other ships — in a video published by Fars early Wednesday.
Pictures circulated on Iranian social media also showed sailors wearing life jackets evacuating the vessel as a fire burned behind them. NBC News was unable to verify these images.
Iran's navy typically handles patrols in the Gulf of Oman and the wider seas. It recently launched a slightly larger commercial tanker called the Makran it converted into serving a similar function as the Kharg, which was also able to lift heavy cargo and serve as a launch point for helicopters.
The country's paramilitary Revolutionary Guard operates in the shallower waters of the Strait of Hormuz and the Persian Gulf.
The fire comes after a series of mysterious explosions targeting ships in the region.
Israel and Iran have blamed each other for reported attacks on cargo ships since late February, escalating a yearslong shadow war in Mideast waters between the two countries.
Iran has refused to recognize Israel since its Islamic Revolution in 1979 that toppled the U.S.-backed Shah. Israel sees Iran's nuclear program as a threat to its existence.
The shipping incidents have occurred since President Joe Biden took office in January, pledging to rejoin Iran's 2015 nuclear containment deal with world powers if Tehran returns to full compliance with the accord.
The deal was abandoned by former President Donald Trump in a move welcomed by Israel.
The sinking of the Kharg also marks the latest military disaster for Iran, which has aging military equipment and has suffered a series of fatal accidents in recent years.
On Tuesday, a malfunction in the ejector seats of an Iranian F-5 dating back to before the revolution killed two pilots while the aircraft was parked in a hangar.
In 2020, during an Iranian military training exercise, a missile mistakenly struck a naval vessel near Jask, killing 19 sailors and wounding 15. Also in 2018, an Iranian navy destroyer sank in the Caspian Sea.