Exfoliating, help! how to exfoliate properly?

alrite, so i have acne prone skin, everyday i use a rough sponge to exfoliate the ingrown haird on my beard with clearasil ultra, i then use cold water to close the pores. on weekend when i shave i have a normal lukewarm shower, and then use st ives apricot scrub to exfoliate. afterwards i clean my face with cold water. i don't like those little grains from the scrub being left around on my face, coz when i try to use this light mosturizer, those little grains come in the way. any tips on how i can get over this problem? are there any steps which im doing wrong? i was thinkin of using clearasil after using the scrub, is this a good idea? please help!