Libido for both men and women are virtually the same, although they can be affected in very different ways. The Libido is affected by both physical and emotional or psychological factors. While testosterone drives the libido, it can also be affected by psychological and physiological aspects. Depression, fatigue and even one’s self image can affect your libido. These would be examples of how the psyche can affect the level of Libido.