Scrub vigorously with a DRY stiff bristle nylon brush. An extra firm tooth brush would possibly work. I use a nylon grout brush (you can google for examples) to clean polish compound out of wood pores and it works with not too much effort and no marring of the wood.
How do I get the gunk off my rosewood fretboard?
Clean the fretboard – Use fine steel wool to remove stubborn gunk from Rosewood/Ebony/Pau Ferro fretboards, and apply Lemon Oil to re-hydrate. Use a damp cloth to clean Maple fretboards. Polish the guitar body – For Poly-finished (gloss) guitars, spray guitar polish onto a soft cloth and wipe down.
What household items can I use to clean my frets?
The quick answer is that the best way to clean a guitar fretboard with household items is to just use things like Q-tips or cotton swabs to remove the dirt that is specifically located to the side of the fret and use cotton balls (or even face-cleaning pads) for the middle part in between frets.