Citrus Nutrient Deficiency

Moisture stress (too much or too little) and nutrient deficiencies can cause improper fruit set. Keep plants uniformly moist but not wet, and use a well balanced, acid-based, fertilizer formulated for citrus to help prevent these environmental stresses. Low humidity could also cause developing fruits to abort. Make certain that your plant has proper water scheduling. Environmental factors such as dry winds in addition to nutrient and or water problems can cause fruit drop.
Overwatering, poor drainage or improper pH may also be conditions to look into. Make sure the soil partially dries out between waterings. And make sure your fertilizer includes micronutrients- iron zinc and manganese. A good soil might also be worthwhile: http://soiltesting.tamu.edu