Is it OK to strum without a pick?
Most beginners should start with a pick before attempting to strum without. And, if you know how to do it one way, you can always learn the other. But strumming without a pick can allow you more control over your strumming and help you gain access to more tonal possibilities too.
Is it better to learn guitar with or without a pick?
Strumming chords on a steel-string guitar tends to be richer with a pick, and you get a louder volume in general. The downside to using a pick is that string skipping (jumping over one or more strings) can be more difficult than when playing with fingers, e.g. when playing arpeggio patterns.