It depends firstly on the scale you are in and then what you are trying to do. When you have sharps in the key signature, you'll most likely use sharps as accidentals. You'll choose your accidental depending on where you want to move afterwards. The case usually is sharp when you move up, flat when you move down.
How do you know when to use sharps or flats?
Some people say you should pick based on what 'direction' your travelling. Eg, if you're heading to a higher note you should use 'sharp' and if you're heading to a lower note you should use 'flat'.