Not alone but generally they aren't considered to be Middle Eastern countries, at least not since Russia got involved heavily in the region.
https://www.quora.com/Is-the-Caucasus-the-Middle-East
This site generally considers the Caucasus to be part of Europe, as does the rest of the internet increasingly it seems.
Personally, I always considered them to be part of Asia, not the Middle East though. Started to think I might have to give in and join in with Jetpunk's border scheme.
They're featured on the Middle East map quiz. Whether or not they are Asia or Europe doesn't really matter that much except perhaps from a historical perspective since the "Near East," "Middle East," and "Far East" were defined by their relative position to Europe.
Personally, I always considered them to be part of Asia, not the Middle East though. Started to think I might have to give in and join in with Jetpunk's border scheme.