When appropriate. You don't have to use curse words, but, like in real life, there are times when profanity or expletives just seems right for your characters and a "well, fiddle de dee" would break immersion.
Hahahaha. I can just picture it already, this big tough guy walk into a bar. “You bunch of fiddle de dee. Where is my money?!” Hahaha. Just glorious. I think I will have to include this in my story somehow.... it is just too good to pass.