I think I might have the same recorder. Tascam DR-40? If so, you have one mic plugged into one XLR port, but stereo mode is on. If you hit the REC MODE button and change it to MONO instead of STEREO, the clicking will go away.
FAQ SITE:
http://tascam.com/product/dr-40/faqs/
Just a thought.
Cody