Manyo,
It works this way.
You have already laid down a few coats of primer filller to start off with. You guidecoat, and then do all your block work sanding. In your case, you have decided to lay down more primer filler over what you had already put on, because of rub through issues and the like. At this point, you may choose to do a little bit more block sanding or not.
The main point now is, that you must prepare your primer filler for your color coats, by hand sanding wet, (without using any blocks) the entire surface with 800 Grade wet & dry. Using a coarser grit like 500 could cause sanding scratch marks that will appear in your finished color coats. If that happens, you will have no choice but to refinish your panels again. That is something you don't want to happen. Sanding scratch marks will appear in your metallic and pearl finishes . Solid colors on the other hand should tolerate the use of 500 grade to do the final rub down before color, but i would still stick to using 800w&d, wet.
Before you move into hand sanding for your final color coats, lay down a fine coat of black guide coat. That will help you to see any imperfections in the primer filler. Once the guide coat has been sanded away, as well as any imperfections, there should be no real need to keep on sanding.
Also, make sure when you hand sand down, keep your fingers together. If you spread them, you will get tram track marks in your final finish. Use the palm of your hand, and/or the fingers that are closed together, when free hand sanding down in wet or dry sanding applications. Sanding in circular motions produces good even results as well. Don't sand in forward/ backwards straight lines fashion.That causes uneven sanding marks as well.
Hope that helps,
Greg.
(Pro painter.)