(Writing)

OpenGL .exe from scratch

I've wanted for a while to make an .exe completely from scratch, and I finally got to do that this past month. The program opens a window and displays a spinning cube.

"From scratch" is a bit vague, but this is what I mean here:

• Program (written in C) writes the .exe file format and leaves room for x86 assembly/data to be added
• x86 is written manually and assembled to bytecode; I used this assembler, but will write my own simple assembler soon for educational purposes
• System (Windows) and OpenGL calls are done through assembly (don't worry, I did not rewrite OpenGL in x86!). Note that I did not use any OpenGL-adjacent libraries like glfw, glm, etc.

The goal was to learn the .exe file format so that I can use it for compiler backends in the future. I'm also making a guide on how to recreate these steps.

I started by making a small executable that just returned a value (e.g. 1234). For the most part I used the PE Format webpage which is maintained by Microsoft, along with some online resources like PE 101.

After that it was time to move on to the OpenGL demo. I made a quick prototype in C and then rewrote it in x86. After some debugging, I finally got this to appear:

You can (hopefully) recreate this on your end, too! Start by downloading these files to the same folder:

• og_opengl.c; the main C program (please excuse the giant binary arrays!)
• import_table.c; auto-generated code which writes to the .exe's import table
• run_output_exe.py; a Python script that compiles og_opengl.c, runs it to generate the output .exe, and then runs that output .exe

Use the command python run_output_exe.py to automate the process.

Additionally, the following x86 was written manually, "cleaned" of comments using a Python script, and then assembled using an online assembler to produce the binary arrays in og_opengl.c:

• the window callback; output goes in og_opengl.c --> text_raw_wndproc[]
• the 3D rotation function; output goes in og_opengl.c --> text_raw_transform[]
• the main function; output goes in og_opengl.c --> text_raw_main[]

This is about 600 lines of x86 in total. Note that I did (could) not use any symbolic names/addresses for this project, but a proper compiler would handle this more elegantly.

Some tools I used:

• Visual Studio on the command line (cl command) for compiling everything
• Python for automation and for generating some C code (i.e. the import table)
• HxD for viewing the raw hex contents of my output .exe
• PE Tools and PE File Browser to see what parts of the .exe were formatted correctly/incorrectly
• OllyDbg to step through the program and examine the values of registers

(Writing)