Here is my code. I tried to decrypt an affine cipher. My .txt files include two cipher: alpha and beta. Alpha is an odd number because gcd(alpha, 26)==1 but decryption doesnt work correctly. For example: the encrypted word UVOVE, when decrypted, turns out in the AAAAA word but it should be "BAHAR". Keys are: alpha:25 and beta:5
fscanf(key_alpha, "%d", &alpha);
fscanf(key_beta, "%d", &beta);
while((c = getc(cipher)) != EOF)
putc(((toupper(c) - 65 - beta) / alpha)% 26 + 65, decrypt); // 65 = the numeric value of A in ASCII
E(x)=ax+b mod 26
D(x)=a^(-1)x-b mod 26
a^(-1) mod 26 is not just simply 1/26. It is a multiplicative inverse of a number a. It means that a^(-1) is such that a*a^(-1) = 1 mod 26.
If a=25 => a^(-1)=25.
Use euclidean algorithm to find multiplicative inverse mod 26.
If you want to avoid problem with negative numbers you can just use additive inverse of b mod 26. Additive inverse of b mod 26 is a number b' such that b + b' = 0 mod 26. In your case b'=21.
putc(((toupper(c) - 65 + add_inv_beta) * mul_inv_alpha) % 26 + 65, decrypt);