Well if you are on a diet you are naturally going to have less energy and therefore have less energy for your sex drive. However, this effect is most common with low protein diets rather than low carb.

But if you really find it a problem, for the love of God eat something!