So then, what's scoop with applications like CrazyBump
Make bumps, then convert then to normal maps

You can't paint over normal maps the same way you can paint on bumps.
When you look at a bump map, you can easily see the depth by understanding the greyscale.
Unless you have a pretty well trained eye (much more than i even imagine i have) you won't be able change a normal map by painting on it, unless it's a matter of minor fixes.
Also, grayscale maps are used on a much more usefull thing that is called "displacement maps", which only work on powerfull CGI render engines, which precalc subdivisions of the faces and then move those fake vertexes based on the displace map to create a real relief (which exists only during render time), and then precalc the lighting over all that subdivided surface.
Results on those are amazing as the geometry is actually deformed, and you won't lose the effect at the edges as a bump or normal, which are texture artifacts.
The drawback is the huge rendertime.