What about between Bahrain and Saudi Arabia (looks to be about 5 miles) and Bahrain and Qatar (looks like less than 2 miles from one of Bahrain's islands to Qatar).
Here is Middle Island which has Saudi customs on one side and Bahraini customs on the other - but try as I might, I just cannot find whether either country owns it, or if the border runs down its middle! Does anyone out there know?
UK territories do not form a part of the UK, even though they are under its jurisdiction (which is why Anguilla - Sint Maarten above has also been omitted).
And here is the closest non-bordering country pair in the world! The leaf-shaped island in the middle distance is Namibia, all of the green river-bank in the foreground is Zimbabwe, and between them, the strip of dry land with a road running across it, is Botswana (just) keeping them apart.
I'm going to be extra nit-picky here though: this should really still be classified as a "water border." Namibia owns no territory on the south side of the Chobe River, thus you must cross water in order to get from Namibia to Zimbabwe at this shortest distance.
Plus if you look at the way the borders are drawn on Google Maps (admittedly not perfect), the narrowest they ever are is over water. I know that's not your criteria, but just added reference.
The only reason I got that was because I was on google earth and I saw Namibia, Botswana, Zimbabwe and Zambia and I wanted to see if they made a four corners.
Yes, that's a pair that I tried repeatedly, because Bougainville (PNG for now) and the Solomons are in the same archipelago, and just as close as PNG and Australia (which are also less than 4km apart at Saibai Island, not 6km)
Saudi Arabia and Egypt are only 5km apart across the Straits of Tiran at Sharm El-Sheikh, and Egypt is also only 7km from Jordan at the northern end of the Gulf of Aqaba (at Taba/Eilat/Aqaba)
Japan and Russia both claim the Habomai Islands, which are part of the broader Kuril Islands dispute. The Habomais are uninhabited, with one tiny island about 50 metres across, called Rifovy (Рифовый) by the Russians, is 5 km / 3 miles from Cape Nosappu on Hokkaido. The nearest inhabited disputed island is Kunashir/Kunashiri, 15 km / 10 miles away.
Bahrain-Saudi Arabia is currently under investigation (a chat with a Bahrain government official!)
Plus if you look at the way the borders are drawn on Google Maps (admittedly not perfect), the narrowest they ever are is over water. I know that's not your criteria, but just added reference.
Mongolia-Kazakhstan
Nepal-Bhutan
Pakistan-Tajikistan