Corey Richardson Corey Richardson - 3 months ago 12
Linux Question

GCC + Inline Assembly - Unexpected Segfault on notl

Code: http://paste.pocoo.org/show/422081/ (link is 404 missing).

Alright so I'm trying to NOT generic chunks of memory and decided I was done diddling with unwieldy casts and figured I'd drop down into inline assembly (I think it actually improved readability). I've managed to narrow this segfault to one specific function. It's at line 22 of the paste.

Yet it always segfaults. As you can see, str is passed as argv[1].

I can manually do stuff to argv[1] (for example

argv[1][0] = 'q'
) so I'm not entirely sure why that not doesn't work, especially considering that earlier in
nots
it successfully runs
not8
and
not2
. Is there something funky going on that I don't know about? What's going wrong here?

Also a generic code review would be nice; I'm fairly new to C.

Answer

Your problem is the password variable. You allocate just one element and then you do password++ in the for loop. So the second time you do the loop you are in nowhere land.

The code shows that you are using much too complicated concepts than you master at the moment.

  • Don't do inline assembler when you don't know yet how to handle pointers.
  • Don't cast the return of malloc.
  • unsigned char is generally the correct type to inspect individual bytes of an object.