I'm trying to implement a smudge tool like one you would find in Gimp or Photoshop. I've tried lots of variations but they all have problems. The basic method I've tried for smudging from position P1 to P2 on an image is:
I suspect that you algorithm constantly renders your rectangles on top of each other, likely averaging their values to get the new color value below. Depending on how the particulars of you program is set up, I would wager that the RGB values are going to 0x000000 due to this repeated averaging. This also explains why smaller step distances blacken faster, as more steps equates to more averaging. I had a similar problem with a color changer that was repeatedly blurred going black on the edges.
A good tutorial on creating your own smudge tool can be found here: http://losingfight.com/blog/2007/09/04/how-to-implement-smudge-and-stamp-tools/
Unfortunately, the examples are all in Objective-C but the text explains fairly well what is going on.
Hope this helps.