We present a systematic evaluation of the robustness of ChatGPT (in both zeroand few-shot settings) under input perturbations for Named Entity Recognition (NER) task using automatic evaluation. Our findings suggest: (1) ChatGPT is more brittle on Drug or Disease entity perturbations (rare entities) as compared to those on widely known Person or Location entities, and (2) the quality of explanations for the same entity considerably differ under various Entity-specific and Context-specific perturbations; the quality significantly improves using incontext learning.