Reverse Engineering Tools: Difference between revisions

From iPhone Development Wiki
(→‎IDA: since when???)
(41 intermediate revisions by 9 users not shown)
Line 1: Line 1:
'''This is a draft that needs your help. Can you improve it? Add some details!'''
While developing a tweak, you may find these tools useful to analyze how iOS and apps work, and to find where to interpose your functionality.
While developing a tweak, you may find these tools useful to analyze how iOS and apps work, and to find where to interpose your functionality.


== Runtime analysis ==
== Dynamic analysis ==


The following tools are useful for analyzing a program during runtime.
The following tools are useful for analyzing a program during runtime.


=== GDB / LLDB ===
=== GDB / LLDB ===
When writing software, a debugger can help determine what is causing a crash, to find backtrace information on certain points of a program, and so on. Attaching the debugger to normal processes running on the iPhone can be done with the description on [[Debugging on iOS 7]].
 
When writing software, a debugger can help determine what is causing a crash, to find backtrace information on certain points of a program, and so on. Attaching the debugger to normal processes running on the iPhone can be done with the description on [[debugserver]], and see [[Debugging on iOS 7]] for more context.


=== Cycript ===
=== Cycript ===
[[Cycript]] allows you to run your own code in an attached process out-of-the-box, with some JavaScript-syntax goodies to make writing code more convenient. It allows for useful runtime analysis of a program (such as for instance getting the complete view hierarchy, or checking out the properties of an object), and it allows for easy prototyping of a tweak (by hooking methods with a Substrate bridge, changing objects freely and calling functions, etc.).
[[Cycript]] allows you to run your own code in an attached process out-of-the-box, with some JavaScript-syntax goodies to make writing code more convenient. It allows for useful runtime analysis of a program (such as for instance getting the complete view hierarchy, or checking out the properties of an object), and it allows for easy prototyping of a tweak (by hooking methods with a Substrate bridge, changing objects freely and calling functions, etc.).


=== Logify ===
=== Logify ===
While not a runtime analysis tool, [[Logify]] takes an Objective-C header file containing a class interface and generates a [[Logos]] file hooking all methods in the given class, and for each hook logging the call of the method (with parameters) to the [http://theiphonewiki.com/wiki/System_Log syslog]. Logify allows for convenient analysis of what methods of a class get called during runtime, and when.
While not a runtime analysis tool, [[Logify]] takes an Objective-C header file containing a class interface and generates a [[Logos]] file hooking all methods in the given class, and for each hook logging the call of the method (with parameters) to the [http://theiphonewiki.com/wiki/System_Log syslog]. Logify allows for convenient analysis of what methods of a class get called during runtime, and when.


=== weak_classdump ===
=== weak_classdump ===
When <code>class-dump</code> (described below) can't analyze an executable and generate header files with class interfaces (due to encryption, malformed binaries etc.), another option is to get these definitions from the runtime. [https://github.com/limneos/weak_classdump <code>weak_classdump</code>] is a [[Cycript]] tool which attaches into a project and generates <code>class-dump</code>-like output files.
 
When <code>class-dump</code> (described below) can't analyze an executable and generate header files with class interfaces (due to App Store app encryption, other encryption, malformed binaries etc.), another option is to get these definitions from the runtime. [https://github.com/limneos/weak_classdump <code>weak_classdump</code>] is a [[Cycript]] tool which attaches into a project and generates <code>class-dump</code>-like output files.


<code>weak_classdump</code> can be used to dump a single class, like this:
<code>weak_classdump</code> can be used to dump a single class, like this:
Line 29: Line 31:


It can also be used to dump all the classes in a bundle (in this case, the main bundle):
It can also be used to dump all the classes in a bundle (in this case, the main bundle):
<source lang=javascript>
<source lang=javascript>
iPhone$ cycript -p Skype weak_classdump.cy; cycript -p Skype
iPhone$ cycript -p Skype weak_classdump.cy; cycript -p Skype
Line 37: Line 40:
See the [[Cycript Tricks#Weak Classdump .28Cycript based class-dump.29|<code>weak_classdump</code> section of Cycript Tricks]] for another example.
See the [[Cycript Tricks#Weak Classdump .28Cycript based class-dump.29|<code>weak_classdump</code> section of Cycript Tricks]] for another example.


== Executable analysis ==
=== InspectiveC ===
 
[[InspectiveC]] allows you to log message hierarchies of certain objects, classes, and selectors. It is very useful if you're trying to figure out how a certain method or class works without having to go into the assembly. You can temporarily use InspectiveC in your tweak to log objects as needed.
 
=== Runtime View Debugging ===
 
==== Reveal ====
[https://revealapp.com Reveal] is a macOS App designed for UI Debugging. In terms of UX, it appears to replicate the XCode storyboard layout, offering a plethora of layout tools and the ability to edit UI in real-time.
 
It is worth noting that version 24 and 25 exhibit terrible performance in most use cases. A free trial is offered, and it's advised that you evaluate the product before purchasing, as for some users, it has completely failed to work as advertised. It is a powerful debugging application when it works properly.
On iOS 13, and/or Version 24, a change was made that broke Reveal's ability to load into SpringBoard. You will need to use or copy the below project's fix for this issue below:
 
[https://github.com/ApexTweaks/RevealLoader Reveal Loader] will dynamically load the RevealServer framework into applications the user selects, and will automatically load itself into SpringBoard without requiring user intervention.
 
When you load an application using Reveal, the application will appear to become unresponsive, as Reveal will "pause" execution in order to "snapshot" the current UI state. This is expected and may take several minutes to complete.
 
==== Lookin ====
 
Lookin is an alternative to Reveal that, in addition to being free, performs much better and offers many more features than Reveal. The installation method is identical to Reveal.
 
==== Spark Inspector ====
[http://www.sparkinspector.com Spark Inspector] has  a three-dimensional view of your app's interface and the ability to change view properties at runtime
 
==== FLEX ====
[https://github.com/Flipboard/FLEX FLEX] is an in-app debugging and exploration tool for iOS.
 
[https://github.com/NSExceptional/FLEXing/tree/statusbar-activation FLEXing] will help you load (the up-to-date) FLEX into your applications by holding the status bar.
 
== Static analysis ==


The following tools can be used to analyze an executable.
The following tools can be used to analyze an executable.
=== Clutch ===
[https://github.com/KJCracks/Clutch Clutch] decrypts app executables, plugins and frameworks. Requires iOS7 and above:
<source lang=bash>
iPhone:~ root#  Clutch -b com.atebits.Tweetie2
...
Finished dumping com.atebits.Tweetie2 to /var/tmp/clutch/2AC9E1CA-450D-4267-8601-D76375B0354B
Finished dumping com.atebits.Tweetie2 in 27.3 seconds
iPhone:~ root#
</source>


=== dumpdecrypted ===
=== dumpdecrypted ===
App Store app executables are encrypted. [https://github.com/stefanesser/dumpdecrypted dumpdecrypted] can generate a decrypted executable out of it:
App Store app executables are encrypted. [https://github.com/stefanesser/dumpdecrypted dumpdecrypted] can generate a decrypted executable out of it:


Line 50: Line 94:
Application.decrypted #decrypted, generated executable
Application.decrypted #decrypted, generated executable
</source>
</source>
(Or see weak_classdump above.)


=== class-dump, class_dump_z, classdump-dyld ===
=== class-dump, class_dump_z, classdump-dyld ===
From a given executable, [http://stevenygard.com/projects/class-dump/ class-dump] and [https://code.google.com/p/networkpx/wiki/class_dump_z class_dump_z] will generate header files with class interfaces. This allows for an analysis of what methods exist in the executable, which can help you guess which ones to hook to get given functionality.


All default (private and public) libraries on iOS are combined into a big cache file to improve performance in <code>/System/Library/Caches/com.apple.dyld/dyld_shared_cache_armX</code> (see [[dyld_shared_cache]] for more details). If you want to class-dump private frameworks, you can either install Xcode and class-dump the frameworks on your Mac using the above tools, or you can use [https://github.com/limneos/classdump-dyld classdump-dyld], which works right on your device. Remember that the resulting files are not the original headers, so use them with caution.
From a given executable, [http://stevenygard.com/projects/class-dump/ class-dump] and [https://code.google.com/p/networkpx/wiki/class_dump_z class_dump_z] will generate header files with class interfaces. (class-dump may produce better headers than class-dump-z for recent binaries.) This allows for an analysis of what methods exist in the executable, which can help you guess which ones to hook to get given functionality.
 
All default (private and public) libraries on iOS are combined into a big cache file to improve performance in <code>/System/Library/Caches/com.apple.dyld/dyld_shared_cache_armX</code> (see [[dyld_shared_cache]] for more details). If you want to class-dump private frameworks, you can either install Xcode and class-dump the frameworks on your Mac using the above tools, or you can use [https://github.com/limneos/classdump-dyld classdump-dyld], which works right on your device (classdump-dyld can also be installed via its package hosted on BigBoss). Remember that the resulting files are not the original headers, so use them with caution.


You can also find other developers have done this process for many frameworks and compiled this information into repositories:
You can also find other developers have done this process for many frameworks and compiled this information into repositories:
* [https://github.com/nst/iOS-Runtime-Headers iOS-Runtime-Headers]
* [https://github.com/nst/iOS-Runtime-Headers iOS-Runtime-Headers]
* [https://github.com/rpetrich/iphoneheaders/ iphoneheaders]
* [https://github.com/rpetrich/iphoneheaders/ iphoneheaders]
Line 62: Line 110:


=== Disassemblers ===
=== Disassemblers ===
Disassemblers are useful when you need an in-depth analysis of a binary. These programs convert the compiled code into assembly for your examination. Assembly is hard to understand for beginners and is platform-dependent (ARM assembly is very different from x86 assembly), so you need a good knowledge of assembly to find disassemblers useful.  
 
Disassemblers are useful when you need an in-depth analysis of a binary. These programs are designed to aid and facilitate reverse engineering of compiled software.
 
Although all can "Disassemble", that is, provide assembly code, some can also provide near-perfect C pseudocode from the assembly. This is called decompiling, and IDA, Hopper, and Ghidra all have powerful decompilers bundled with them.


==== IDA ====
==== IDA ====
[https://www.hex-rays.com/products/ida/ IDA] (Interactive Disassembler) is a popular program for disassembling binaries. It supports a [https://www.hex-rays.com/products/ida/processors.shtml plethora] of processors. IDA has tons of features and has been in development for more than a decade.


It is a commercial application, and it requires some time getting used to it. For analyzing Objective-C applications, KennyTM's [https://github.com/kennytm/Miscellaneous/blob/master/fixobjc2.idc fixobjc2.idc script] is useful for exposing Objective-C method definitions and calls.
IDA, or IDA Pro, (Interactive Disassembler) is a very popular program for disassembling binaries. It supports a [https://www.hex-rays.com/products/ida/processors.shtml plethora] of processors.
 
IDA has a massive amount of features and has been in development for more than a decade. It's typically regarded as the industry standard for Reverse Engineering. Recent versions include unrivalled dyld_shared_cache tools. These have been documented in the page linked below.
 
[https://iphonedevwiki.net/index.php/IDA_Pro A much more extensive writeup on using IDA for iOS Research.]
 
{| class="wikitable"
|-
! Subproduct Name
! Key Features
! Includes Decompiler
! Includes Debugger
! Approximate price
|-
| IDA Pro
| "Full Version". Capable of disassembling/debugging most binary types, both 32 and 64 bit.
| With Purchase
| Yes
| ~$4248 With 1 Decompiler
|-
| IDA Home
| "Lite Version". One processor type per license.
| "Cloud Decompiler"
| Yes
| ~$370/year
|-
| IDA Freeware
| even "lite-er' version. x86 and x86_64 only. Presumably good for simulator binaries.
| Cloud Decompiler
| No
| $0
|}


==== Hopper ====
==== Hopper ====
[http://www.hopperapp.com/ Hopper] is quite new and only supports a small subset of the features that IDA has. It is fast and has a nice user interface. However, the produced assembly code is not as good as the one produced by IDA.
 
[http://www.hopperapp.com/ Hopper] is a newer disassembler and decompiler that offers an excellent choice for hobbyists that don't have several thousand to spare.
 
Some crucial Hopper features: 
* arm64e support
* Basic dyld_shared_cache handling
* Support for .development files from Internal dumps
* Excellent UI and UX. Feels modern and powerful.
* String, function, and other search tools leave other disassemblers in the dust.
 
Downsides:
* Only local x64 Debugging
* Missing some crucial features for iOS
* Pseudocode cannot be edited, and is often difficult to read.
 
The standard License is $99.
 
A free, evaluation copy of the program is offered which limits functionality and showcases an older version of the program. While it's more suited to x86_64 binaries, feel free to evaluate the product and see if it works well for you.
 
==== Ghidra ====
 
[https://ghidra-sre.org/ Ghidra] is a free, very powerful reverse-engineering tool released by the NSA. The pseudocode it generates is on par with IDA, and offers an alternative to Hopper's pseudocode, which can be difficult to work with.
 
For those who can't afford expensive licenses, Ghidra is more than enough for any developer or engineer.
 
==== BinaryNinja ====
 
[https://binary.ninja/ BinaryNinja] is a newer Disassembler.
 
''More information/experience needed here''
 
==== jtool ====
 
[http://www.newosxbook.com/tools/jtool.html jtool] is a project by morpheus which provides a powerful command-line utility for static analysis of Mach-O caches, objects, files, and more. Documentation is available on the linked page.
 


==== otool ====
==== otool ====
'''write me <sup>please</sup>'''
 
The [http://www.unix.com/man-page/osx/1/otool/ otool] command displays specified parts of object files or libraries.  It can also disassemble:
 
Example usage:
 
<source lang=bash>
bash$ xcrun -sdk iphoneos otool -arch arm64 -tV FaceCore
/Applications/Xcode.app/.../PrivateFrameworks/FaceCore.framework/FaceCore:
(__TEXT,__text) section
0000000000001100 stp fp, lr, [sp, #-16]!
0000000000001104 add fp, sp, 0
0000000000001108 stp x20, x19, [sp, #-16]!
000000000000110c sub sp, sp, #16
...
</source>


=== strings ===
=== strings ===
[http://unixhelp.ed.ac.uk/CGI/man-cgi?strings strings] is a simple utility that will print all the strings in a given binary.
[http://unixhelp.ed.ac.uk/CGI/man-cgi?strings strings] is a simple utility that will print all the strings in a given binary.


Example usage:
Example usage:
<source lang=text>
 
<source lang=bash>
bash$ strings crash_mover
bash$ strings crash_mover
moveLogsAtPath
moveLogsAtPath
Line 88: Line 219:


=== nm ===
=== nm ===
[http://unixhelp.ed.ac.uk/CGI/man-cgi?nm nm] is a utility that displays the symbol table of a given binary.
[http://unixhelp.ed.ac.uk/CGI/man-cgi?nm nm] is a utility that displays the symbol table of a given binary.


Example usage:
Example usage:
<source lang=text>
<source lang=bash>
bash$ nm CoreTelephony
bash$ nm CoreTelephony
000234c4 t +[CTCall callForCTCallRef:]
000234c4 t +[CTCall callForCTCallRef:]

Revision as of 04:25, 2 July 2021

While developing a tweak, you may find these tools useful to analyze how iOS and apps work, and to find where to interpose your functionality.

Dynamic analysis

The following tools are useful for analyzing a program during runtime.

GDB / LLDB

When writing software, a debugger can help determine what is causing a crash, to find backtrace information on certain points of a program, and so on. Attaching the debugger to normal processes running on the iPhone can be done with the description on debugserver, and see Debugging on iOS 7 for more context.

Cycript

Cycript allows you to run your own code in an attached process out-of-the-box, with some JavaScript-syntax goodies to make writing code more convenient. It allows for useful runtime analysis of a program (such as for instance getting the complete view hierarchy, or checking out the properties of an object), and it allows for easy prototyping of a tweak (by hooking methods with a Substrate bridge, changing objects freely and calling functions, etc.).

Logify

While not a runtime analysis tool, Logify takes an Objective-C header file containing a class interface and generates a Logos file hooking all methods in the given class, and for each hook logging the call of the method (with parameters) to the syslog. Logify allows for convenient analysis of what methods of a class get called during runtime, and when.

weak_classdump

When class-dump (described below) can't analyze an executable and generate header files with class interfaces (due to App Store app encryption, other encryption, malformed binaries etc.), another option is to get these definitions from the runtime. weak_classdump is a Cycript tool which attaches into a project and generates class-dump-like output files.

weak_classdump can be used to dump a single class, like this:

iPhone$ cycript -p Skype weak_classdump.cy; cycript -p Skype
'Added weak_classdump to "Skype" (1685)'
cy# weak_classdump(SkypeAppDelegate, "/tmp/")
"Wrote file to /tmp/SkypeAppDelegate.h"

It can also be used to dump all the classes in a bundle (in this case, the main bundle):

iPhone$ cycript -p Skype weak_classdump.cy; cycript -p Skype
'Added weak_classdump to "Skype" (1685)'
cy# weak_classdump_bundle([NSBundle mainBundle], "/tmp/SkypeHeaders")

See the weak_classdump section of Cycript Tricks for another example.

InspectiveC

InspectiveC allows you to log message hierarchies of certain objects, classes, and selectors. It is very useful if you're trying to figure out how a certain method or class works without having to go into the assembly. You can temporarily use InspectiveC in your tweak to log objects as needed.

Runtime View Debugging

Reveal

Reveal is a macOS App designed for UI Debugging. In terms of UX, it appears to replicate the XCode storyboard layout, offering a plethora of layout tools and the ability to edit UI in real-time.

It is worth noting that version 24 and 25 exhibit terrible performance in most use cases. A free trial is offered, and it's advised that you evaluate the product before purchasing, as for some users, it has completely failed to work as advertised. It is a powerful debugging application when it works properly.

On iOS 13, and/or Version 24, a change was made that broke Reveal's ability to load into SpringBoard. You will need to use or copy the below project's fix for this issue below:

Reveal Loader will dynamically load the RevealServer framework into applications the user selects, and will automatically load itself into SpringBoard without requiring user intervention.

When you load an application using Reveal, the application will appear to become unresponsive, as Reveal will "pause" execution in order to "snapshot" the current UI state. This is expected and may take several minutes to complete.

Lookin

Lookin is an alternative to Reveal that, in addition to being free, performs much better and offers many more features than Reveal. The installation method is identical to Reveal.

Spark Inspector

Spark Inspector has a three-dimensional view of your app's interface and the ability to change view properties at runtime

FLEX

FLEX is an in-app debugging and exploration tool for iOS.

FLEXing will help you load (the up-to-date) FLEX into your applications by holding the status bar.

Static analysis

The following tools can be used to analyze an executable.

Clutch

Clutch decrypts app executables, plugins and frameworks. Requires iOS7 and above:

iPhone:~ root#  Clutch -b com.atebits.Tweetie2
...
Finished dumping com.atebits.Tweetie2 to /var/tmp/clutch/2AC9E1CA-450D-4267-8601-D76375B0354B
Finished dumping com.atebits.Tweetie2 in 27.3 seconds
iPhone:~ root#

dumpdecrypted

App Store app executables are encrypted. dumpdecrypted can generate a decrypted executable out of it:

iPhone$ DYLD_INSERT_LIBRARIES=dumpdecrypted.dylib /var/mobile/Applications/.../Application.app/Application
iPhone$ ls Application*
Application #original executable
Application.decrypted #decrypted, generated executable

(Or see weak_classdump above.)

class-dump, class_dump_z, classdump-dyld

From a given executable, class-dump and class_dump_z will generate header files with class interfaces. (class-dump may produce better headers than class-dump-z for recent binaries.) This allows for an analysis of what methods exist in the executable, which can help you guess which ones to hook to get given functionality.

All default (private and public) libraries on iOS are combined into a big cache file to improve performance in /System/Library/Caches/com.apple.dyld/dyld_shared_cache_armX (see dyld_shared_cache for more details). If you want to class-dump private frameworks, you can either install Xcode and class-dump the frameworks on your Mac using the above tools, or you can use classdump-dyld, which works right on your device (classdump-dyld can also be installed via its package hosted on BigBoss). Remember that the resulting files are not the original headers, so use them with caution.

You can also find other developers have done this process for many frameworks and compiled this information into repositories:

Disassemblers

Disassemblers are useful when you need an in-depth analysis of a binary. These programs are designed to aid and facilitate reverse engineering of compiled software.

Although all can "Disassemble", that is, provide assembly code, some can also provide near-perfect C pseudocode from the assembly. This is called decompiling, and IDA, Hopper, and Ghidra all have powerful decompilers bundled with them.

IDA

IDA, or IDA Pro, (Interactive Disassembler) is a very popular program for disassembling binaries. It supports a plethora of processors.

IDA has a massive amount of features and has been in development for more than a decade. It's typically regarded as the industry standard for Reverse Engineering. Recent versions include unrivalled dyld_shared_cache tools. These have been documented in the page linked below.

A much more extensive writeup on using IDA for iOS Research.

Subproduct Name Key Features Includes Decompiler Includes Debugger Approximate price
IDA Pro "Full Version". Capable of disassembling/debugging most binary types, both 32 and 64 bit. With Purchase Yes ~$4248 With 1 Decompiler
IDA Home "Lite Version". One processor type per license. "Cloud Decompiler" Yes ~$370/year
IDA Freeware even "lite-er' version. x86 and x86_64 only. Presumably good for simulator binaries. Cloud Decompiler No $0

Hopper

Hopper is a newer disassembler and decompiler that offers an excellent choice for hobbyists that don't have several thousand to spare.

Some crucial Hopper features:

  • arm64e support
  • Basic dyld_shared_cache handling
  • Support for .development files from Internal dumps
  • Excellent UI and UX. Feels modern and powerful.
  • String, function, and other search tools leave other disassemblers in the dust.

Downsides:

  • Only local x64 Debugging
  • Missing some crucial features for iOS
  • Pseudocode cannot be edited, and is often difficult to read.

The standard License is $99.

A free, evaluation copy of the program is offered which limits functionality and showcases an older version of the program. While it's more suited to x86_64 binaries, feel free to evaluate the product and see if it works well for you.

Ghidra

Ghidra is a free, very powerful reverse-engineering tool released by the NSA. The pseudocode it generates is on par with IDA, and offers an alternative to Hopper's pseudocode, which can be difficult to work with.

For those who can't afford expensive licenses, Ghidra is more than enough for any developer or engineer.

BinaryNinja

BinaryNinja is a newer Disassembler.

More information/experience needed here

jtool

jtool is a project by morpheus which provides a powerful command-line utility for static analysis of Mach-O caches, objects, files, and more. Documentation is available on the linked page.


otool

The otool command displays specified parts of object files or libraries. It can also disassemble:

Example usage:

bash$ xcrun -sdk iphoneos otool -arch arm64 -tV FaceCore
/Applications/Xcode.app/.../PrivateFrameworks/FaceCore.framework/FaceCore:
(__TEXT,__text) section
0000000000001100		stp	fp, lr, [sp, #-16]!
0000000000001104		add	fp, sp, 0
0000000000001108		stp	x20, x19, [sp, #-16]!
000000000000110c		sub	sp, sp, #16
...

strings

strings is a simple utility that will print all the strings in a given binary.

Example usage:

bash$ strings crash_mover
moveLogsAtPath
Could not open and lock %s: %s. Proceeding with copy anyway.
Extensions
...

nm

nm is a utility that displays the symbol table of a given binary.

Example usage:

bash$ nm CoreTelephony
000234c4 t +[CTCall callForCTCallRef:]
0001ee90 t +[CTEmailAddress emailAddress:]
000199b8 t +[CTMessageCenter sharedMessageCenter]
0001db54 t +[CTMmsEncoder decodeMessageFromData:]
...