Basic Reverse Engineering With Live Example

For this "tutorial" we will use a real program called PDF2Word.
I stumbled upon this program when trying to convert a pdf document to a word document (duh..).

The funny thing about this program is that it costs $39.95 and that it is released under the GPL.

Let's start shall we?

1: Obtain a copy of the program athttp://www.verypdf.com/pdf2word/index.html
We will use version 2.6 in this tutorial.

2: Obtain a copy of ollydbg at http://www.ollydbg.de/

Start PDF2Word and you will see a screen with title"Please register .."
Type in an emailadress and a bogus Registrationkey and hit OK.
Write down the error message you get since we will need it later on:
"Series number error, please check it and try again."

Now close pdf2word and start Ollydbg. From within ollydbg go to file, open and browse to
C:\Program Files\PDF2Word v2.6\ and select pdf2rtf.exe.

Once the program is loaded rightclick somewhere in the code table and select Search for
Then select All referenced text strings".
A new window will open with all text strings in the program. Scroll up and rightclick once more.
Now select Search for text and put in the errorstring you had earlier in the program:
"Series number error, please check it and try again.".
deselect "Case Sensitive" and press OK.

You will see the following line highlighted:

00429F6F | PUSH pdf2rtf.00468270 |ASCII "Series number error,
please check it and try again."

Now press to put a breakpoint on it. Directly above you will find some other strings
which might be of interest as well so put a breakpoint on them to. These are:
"Thank you registered" and "Thank you registered VeryPDF PDF2Word v1.6."

Now double click on one of the lines and press to start the program.

The registration window will come up again, so now put in an emailaddress, a dummy registration-key
and press OK.

Go back to Olly and you will see the following line highlighted:

00429F6F PUSH pdf2rtf.00468270 ;|Text = "Series number error, 
please check it and try again."

This is our errormessage again and the program has stopped right before displaying it.
You can see this errormessage is part of a MessageBoxA call, this is the
API responsible for displaying the Message. If you look a couple of lines earlier you will
see another MessageBoxA call where instead of our errormessage now the
message "Thank you registered VeryPDF PDF2Word v1.6." is located.
This means that if we have the correct serial it will display the registered message, and if
we don't we will get the error.
The program will need to decide if your serial is correct before it can display any of these
messages. The code to do so is usually located close to the messages so scroll up a bit until
you see a piece of code containing a JNE,JE,JNZ or JZ. These are jumps that occur when a specific
event is met.
Usually the event is tested in the code directly in front of it.
In this case you should end up at the following lines:

00429F2E . 85C0 TEST EAX,EAX 00429F30 . 74 39 JE SHORT pdf2rtf.00429F6B

As you can see EAX is tested against itself which in this case will jump to 00429F6B if
EAX has the value 0.
if we trace the jump we will see that it will jump over the registered message and end up
right before the errormessage. This means that if EAX = 0 we will get the errormessage and
our serial is wrong.

At this point we could remove the JE SHORT pdf2rtf.00429F6B code and fill it with
NOP so we will always get the correct message since the jump is never made, but that won't help
here cause if you restart the program it will still ask you to register.

So we need to continue our search. We need to figure out where EAX is getting it's value from.
2 lines above TEST EAX,EAX we see

00429F26 . E8 F5F7FFFF CALL pdf2rtf.00429720

This is a call to a function somewhere else in the program and you can bet your life it is this
function that sets the EAX value.
And so we need to figure out what this function does. To do this we will put another breakpoint
at CALL pdf2rtf.00429720 by highlighting the line and press .
Now we want to restart the program to make it break on the function call. This is done by
pressing +. Select YES to the question asked (press left arrow and hit enter) and the program is restarted.
Now press again to let the program run.
You will see the registrationbox popup again, so put in an emailaddress and a bogus serial and press OK.

As you can see the program will break on the CALL pdf2rtf.00429720 line.
Now press to step into the call.
The first 4 lines are not of interest to us so we will start analyzing the code from 00429725.
The code we have there is

00429725 |. 8B7424 3C MOV ESI,DWORD PTR SS:[ESP+3C] 00429729 |. 57 PUSH EDI 0042972A |. 8A06 MOV AL,BYTE PTR DS:[ESI] 0042972C |. 8A4E 01 MOV CL,BYTE PTR DS:[ESI+1] 0042972F |. 8A56 0E MOV DL,BYTE PTR DS:[ESI+E] 00429732 |. 884424 18 MOV BYTE PTR SS:[ESP+18],AL 00429736 |. 32C0 XOR AL,AL 00429738 |. 884C24 30 MOV BYTE PTR SS:[ESP+30],CL 0042973C |. 8A4E 0F MOV CL,BYTE PTR DS:[ESI+F] 0042973F |. 884424 19 MOV BYTE PTR SS:[ESP+19],AL 00429743 |. 884424 31 MOV BYTE PTR SS:[ESP+31],AL 00429747 |. 884424 25 MOV BYTE PTR SS:[ESP+25],AL 0042974B |. 884424 0D MOV BYTE PTR SS:[ESP+D],AL 0042974F |. 8A46 02 MOV AL,BYTE PTR DS:[ESI+2] 00429752 |. 3C 24 CMP AL,24 00429754 |. 885424 24 MOV BYTE PTR SS:[ESP+24],DL 00429758 |. 884C24 0C MOV BYTE PTR SS:[ESP+C],CL 0042975C |. 75 52 JNZ SHORT pdf2rtf.004297B0

I've copied the entire block until the first check of the serial above to safe space, so refer
to above code in this explanation.

1st line --> move our entered serial to ESI
2nd line --> not important
3rd line --> move the first byte (character) of our serial into AL
4th line --> move the second byte (first+1) of our serial into CL
5th line --> move the 13th byte (first+E) of our serial into DL
6th line --> move content of AL (first character of our serial) into [ESP+18]
7th line --> clear the contents of AL
8th line --> move the content of CL (second character) into [ESP+30]
9th line --> move the 14th byte (first+F) of our serial into CL
10th,11th,12th,13th line --> clear the contents of [ESP+19],[ESP+31],[ESP+25] & [ESP+D] since AL still is empty.
14th line --> move the third byte (first+2) of our serial into AL
15th line --> compare content of AL with 0x24 (hex for the $ sign)
16th line --> move the 13th byte of our serial into [ESP+24]
17th line --> move the 14th byte of our serial into [ESP+C]
18th line --> Jump to 004297B0 if AL is not equal to 24 ($ sign).

If you trace the jump in the 18th line you will see that it jumps to the following code:

004297B0 |> 5F POP EDI 004297B1 |. 5E POP ESI 004297B2 |. 33C0 XOR EAX,EAX 004297B4 |. 5D POP EBP 004297B5 |. 83C4 30 ADD ESP,30 004297B8 \. C3 RETN

Which means so much as restore values, set EAX to 0 and return from where we were called.
If we let this happen then EAX will be 0 which will give us the errormessage.

So what do we know now from this code?

- AL should be equal to 0x24 or we will get the errormessage
- the program moves the third character of our serial into AL before comparing it to 0x24
- the program moves the 14th byte of our serial into CL

from above we can conclude that the third character of our serial should be a $ sign and
that our serial should be at least 14 characters long since the 14th character is moved.

So our serial will be something like: ..$...........

It's time for the next piece of code:

0042975E |. 8B3D 4C964400 MOV EDI,DWORD PTR DS:[<&MSVCRT.atoi>] ; msvcrt.atoi 00429764 |. 8D5424 0C LEA EDX,DWORD PTR SS:[ESP+C] 00429768 |. 52 PUSH EDX ; /s 00429769 |. FFD7 CALL EDI ; \atoi 0042976B |. 8BE8 MOV EBP,EAX 0042976D |. 8D4424 1C LEA EAX,DWORD PTR SS:[ESP+1C] 00429771 |. 50 PUSH EAX 00429772 |. FFD7 CALL EDI 00429774 |. 03E8 ADD EBP,EAX 00429776 |. 83C4 08 ADD ESP,8 00429779 |. 83FD 0A CMP EBP,0A 0042977C |. 75 32 JNZ SHORT pdf2rtf.004297B0

Let's analyze it:

1st line --> move the address of the function MSVCRT.atoi to EDI
atoi is a function which converts ASCII characters to integers (numbers).
2nd line --> put the 16th byte of our serial into EDI
3rd line --> push our byte to the stack as an argument to atoi
4th line --> call atoi. the result will be in EAX
5th line --> move content of EAX into EBP
6th line --> move the first byte of our serial into EAX
7th line --> push our byte to the stack as an argument to atoi
8th line --> call atoi. the result will be in EAX
9th line --> add EAX to EBP and store the result in EBP
10th line --> not important to us
11th line --> compare EBX with the value 0x0A (10)
12th line --> if EBX is not 0x0A then jump to 004297B0 (put 0 into EAX and return).

NOTE:You probably wonder how I figured out which byte is used to put into atoi, well here
it is: I've put in serials multiple times with different digits as values and compared them to
the output of atoi in EAX. This resulted in the corresponding bytes.

What do we know from this code?
- our 16th byte is put into EDI, this means that our serial must be at least 16 characters long
- the 16th byte is put into atoi, which means it much have a value between 0-9
- the 1st byte is put into atoi as well, which also means it must have a value between 0-9
- the value of our first byte is added to the value of our 16th byte and together they must be
equal to 0xA (10).

So now we have the following serial: 1.$............9
as you can imagine the values of our 1st and 16th byte can be anything as long as they both
are digits and when added to eachother are equal to 10 decimal.

Time for the next piece of code:

0042977E |. 8D4C24 24 LEA ECX,DWORD PTR SS:[ESP+24] 00429782 |. 51 PUSH ECX 00429783 |. FFD7 CALL EDI 00429785 |. 8D5424 34 LEA EDX,DWORD PTR SS:[ESP+34] 00429789 |. 8BE8 MOV EBP,EAX 0042978B |. 52 PUSH EDX 0042978C |. FFD7 CALL EDI 0042978E |. 03E8 ADD EBP,EAX 00429790 |. 83C4 08 ADD ESP,8 00429793 |. 83FD 0A CMP EBP,0A 00429796 |. 75 18 JNZ SHORT pdf2rtf.004297B0

Let's analyze it:

1st line --> our 15th byte is put into ECX
2nd line --> push our byte to the stack as an argument to atoi
3rd line --> call atoi. the result will be in EAX
4th line --> our 2nd byte is put into EDX
5th line --> move the result of atoi into EBP
6th line --> push our 2nd byte to the stack as an argument to atoi
7th line --> call atoi. the result will be in EAX
8th line --> add the result of our 15th and 2nd byte together and store in EBP
9th line --> not important to us.
10th line --> check if our 15th byte + our 2nd byte added together is equal to 0x0A (10).
11th line --> if not, then another jump to 004297B0

As you can see this code is very similar to the codeblock before, so I won't explain it any
further.

Now we have the following serial: 12$...........89
We still need to analyze some code, but we're getting somewhere ;-)

Lets take a look at the final piece of code from this call:

00429798 |. 807E 03 24 CMP BYTE PTR DS:[ESI+3],24 0042979C |. 75 12 JNZ SHORT pdf2rtf.004297B0 0042979E |. 8A4E 05 MOV CL,BYTE PTR DS:[ESI+5] 004297A1 |. 33C0 XOR EAX,EAX 004297A3 |. 80F9 23 CMP CL,23 004297A6 |. 5F POP EDI 004297A7 |. 5E POP ESI 004297A8 |. 5D POP EBP 004297A9 |. 0F94C0 SETE AL 004297AC |. 83C4 30 ADD ESP,30 004297AF |. C3 RETN

Analysis:

1st line --> since ESI still contains our serial, the 4th byte is now checked against value 0x24
2nd line --> if not equal, then goto the famous 004297B0
3rd line --> move our 6th byte to CL
4th line --> check if our 6th byte is equal to 0x23 (# sign)
5th,6th,7th line --> not important
8th line --> sets the byte in AL to 1 if our check above is equal
9th line --> not important
10th line --> return to where this call was called from.

What we see here is that our 4th byte is compared to 0x24 ($ sign remember?) as well and that
the 6th byte is compared to 0x23 (the # sign).

If that is correct then AL is set to 1 meaning EAX won't be 0 and so causing the program to
give the registered message.

In short:

-check if 3th == $
-check if 1st + 16th == 10
-check if 15th + 2nd == 10
-check if 4th == $
-check if 6th == #

All other characters are of no importance to create a valid serial so our result will be:

12$$.#........89

You can fill in the remaining dots with anything you like and as long as the rules above are
correct you can change the 1th,2nd,15th and 16th byte as well.

Now close OllyDbg and start PDF2Word.
Enter your email-address and enter one of your newly created serials.

When you press ok you will be thanked for registrering. You're welcome ;-)
The result of your entered information is put in a string and written to
%WINDIR%\system32\pdf2word.dat

In case you want another serial just delete the dat file and put in a new one.

Of course this is a very simple protection, but it is a nice example to cover the basics of
reverse engineering.

Final Note:
The explanation of above code pieces isn't completely correct in an assembly way, but it
should be easier to understand like this for non-assembly gurus.