CTF 101 Series: What is Reverse Engineering?
A Capture-the-Flag or “CTF” is a cybersecurity competition designed to test and sharpen security skills through hands-on challenges that simulate real-world situations. MetaCTF offers training in eight different categories: Binary Exploitation, Cryptography, Web Exploitation, Forensics, Reconnaissance, Reverse Engineering, CyberRange, and Other / Miscellaneous. This blog is part of a series that explains what each category covers, why the subject is important, and how to approach these problems.
What is Reverse Engineering?
Reverse engineering is the practice of examining a piece of technology (e.g. software, a computer, or a car) to understand how it works. It is a useful tool when you do not have any documentation, like source code or design documents. The end goal of reverse engineering is typically to either modify or recreate the technology.
Hackers can use reverse engineering to exploit unsecured applications, APIs, and other software. Once they understand how an application works, they can make it behave in ways the designers did not intend.
For example, a hacker could look at how your app calls your API to find vulnerabilities like hidden API endpoints, i.e. endpoints your team has launched but is not actively using (and therefore probably not actively testing). The hacker could deduce how the endpoints work by sending commands and analyzing the responses, then try to exploit the app by sending commands using inputs or sequences that your development team did not anticipate.
How to solve a Reverse Engineering CTF challenge
First, figure out the type of application you are hacking. This information will inform the tools you use.
Next, play with it as a regular user would. Your goal is to take an inventory of the available functionality.
Once you understand the available functionality, use one or more tools to understand how the app works. For a standard desktop app, use a disassembler or debugger like Ghidra. For a mobile app, use a tool like Android Studio. And in either case, if the app is generating any network traffic, monitor that via proxies. The end goal of this phase is to create a mental model of how the app works and communicates with the network.
Finally, look for a way that you can deliver a command or other input to get the flag. Once you understand how the application functions, provide malicious input to the various attack surfaces. For example, you could launch command injection or directory traversal attacks in a HTTP request, or you could upload a specially crafted file designed to allow you to get the flag.
Resources to learn more about forensics training
- Malware reversing and techniques: https://www.youtube.com/c/OALabs
- Collection of resources: https://github.com/wtsxDev/reverse-engineering
- Reverse Engineering for Beginners: https://beginners.re/
Want to learn more about cybersecurity training options for your team? Schedule a demo here or email us at contact@metactf.com.