Resources for learning malware analysis and reverse engineering abound for the Windows platform and PE files, but by comparison there’s very little literature or tutorials for those who want to learn specifically about how to reverse macOS malware and macOS malware analysis techniques.
In this series of posts, you’ll take a sample file and use native tools and techniques to understand what a file does and to build a list of
IoCs (Indicators of Compromise) that can be used in detection.
As there’s a lot of ground to cover, the tutorial is split over several parts. In Part 1, you’ll learn how to set up a safe environment to test malware on macOS. You’ll install all the tools you need (bonus: doesn’t cost a cent!) and learn where you can source samples of macOS malware from. You’ll examine an application bundle and its contents to understand how it works and find an interesting encrypted text file. In Part 2, you’ll learn the fundamentals of static analysis of Mach-O binaries, the native executable file type for macOS. In Part 3, you’ll use dynamic analysis techniques to execute a malware file in a controlled manner and read code from memory.
If you have ever wanted to learn how to reverse malware on macOS, this is the place to start! Continue reading…
For small enterprise developers like myself, commercial analytics aren’t really an option. And while there are open source alternatives like UKCrashReporter, I’ve been lucky enough to suffer relatively few crashes and typically just ask users to manually submit crash reports (stored in ~/Library/Logs/DiagnosticReports) as and when they occur.
Choosing a method of collecting crash reports is one thing, but scouring them for useful information is quite another, and the reams of data often included can seem a little overwhelming.
On a good day, your crash report may include a direct reference to the method or function that’s caused the issue, like this one from an early version of my troubleshooting app DetectX Swift:
On other days, though, you might get presented with nothing but a bunch of unhelpful memory addresses:
It’s at this point that you need to enlist the help of the built-in command-line tool atos. In the remainder of this post, we’re going to learn what it does and how to use it to find the line in our source code that caused the crash.
How does atos help?
The beauty of
atos is that it will give you a reference to the source file, function and even line number in your code where the crash occurred. Given that it can do this for a crash that occurred on another user’s machine that you can’t reproduce locally and don’t have access to, this seems like nothing short of magic.
Here’s an example of what
atos might provide from a crash log like the one above:
ViewController.checkReg(isCli:) (in DetectX Swift) (ViewController.swift:4042)
The output from
atos tells me the name of the Swift source file, the name of the function and — here’s the truly wonderous part — even the line number: 4042.
This is all part of the magic of symbollication, which is such an esoteric word I’m not even going to try to spell it again :-). Instead, let’s just get down to the practical nitty-gritty of how to get this kind of data out of our users’ crash reports.
How to use atos
The method is fairly simple and goes like this:
1. Create a folder called “CrashWork”.
2. Go to Xcode’s ‘Organizer’, and right-click on the archive of the version of the app that crashed.
Choose “Show in Finder”. That takes you to the
.xcarchive file in Finder, from which right-click again and choose “Show Package Contents” to open the package:
3. Click on the
.dSYM file and make a copy of it. Switch back to your CrashWork folder and paste the copy in there. Grab a copy of the same version of your app that crashed (you can also get that from the Products folder in the .xcarchive package, if you don’t have one handy elsewhere) and place it in the same folder:
Important: Be sure you’re working with the same version of your app as the user;
atos needs that to match up the addresses in the crash report, and any other version will produce incorrect results or an error.
4. It’s now time to head on over to the Terminal.
cd into your CrashWork directory:
5. The format of the command is generally going to be like this (see man atos for options):
atos -o <path to executable> -arch x86_64 -l <load address> <address>
<path to executable> is the path all the way to the Mach-O binary in your app bundle in the CrashWork folder, so the path in my example looks like this:
atos -o DetectX\ Swift.app/Contents/MacOS/DetectX\ Swift -arch x86_64 -l <load address> <address>
You need to be careful to get the
<load address> and the
<address> the right way around. They are the reverse order of what you see in the crash log:
Now my complete example looks like this:
atos -o DetectX\ Swift.app/Contents/MacOS/DetectX\ Swift -arch x86_64 -l 0x10a10e000 0x000000010a16a601
6. Hitting ‘return’ produces the magic:
No matter how many times I visit this page, I always find something new that I hadn’t tried before. So the title of this post is more of a suggestion than a description. Why not take a poke around Apple’s shortcut list and discover ten that you don’t already know? I’m going to give a quick round up of ten shortcuts that I use on a regular basis.
1. ⌘ ⌃ Space
Command-Control-Space is a universal shortcut (i.e., you can use it no matter what application is currently active) for bringing up the character viewer (see the screenshot, top of page). In order for it to work, the cursor must be in a text field where character entry is possible, but you can use it literally anywhere: text documents, search fields, browser bars, save dialogs, etc. I used it heavily in creating this post!
2. ⌘ ⌃ ⏏
Command-Control-Eject saves you having to trawl all the way up to the icon top left of your screen and choose ‘Restart…’. It’ll give you the chance to save any unsaved work or cancel the shutdown. This shortcut has a cousin, too: add the option key (⌘ ⌥ ⌃ ⏏ ) to the mix and you get Shutdown instead of Restart.
3. ⌘ ⌫
One of my pet annoyances is continually being asked if I want to save ‘Untitled’ documents, those that I’ve never saved before, when trying to quickly quit an app. Having to take my fingers off the keyboard to move the cursor down to that ‘Delete’ button is time-consuming, and tabbing to it takes multiple hits of the tab key. Fortunately, ‘Command-Delete’ will let you discard the document immediately from the keyboard in one quick shot.
4. ⌘ G
Most people are familiar with invoking ‘Find’ on a page by using ‘Command-F’, but have you ever got annoyed by having to mouse up to those tiny little jump arrows in order to cycle through the hits?
Save your eyes and use Command G instead! Note that you have to invoke Find with ‘Command F’ first.
5. ⌃A, ⌃E, ⌃K
That’s three separate shortcuts not one! In any text field, Control-A moves the cursor to the beginning of the line; Control-E moves it to the end, while ⌃K deletes everything from the cursor to the end of the line. Old-hands of the Terminal and other command-line utilities will know these well, but even they may be surprised that these work across a wide-range of GUI apps, too. Particularly useful in Web browser address bars, search fields, Spotlight and so on.
6. ⌘ L
Command-L in Safari and other browser-based apps (including Xcode’s Documentation viewer 😉 ) will immediately move the focus to the address bar/search bar and allow you to edit its contents. No clicking necessary!
7.⌘ ⌃ 1, ⌘ ⌃ 2, ⌘ ⌃ 3
Three commands new for Safari’s sidebar. Respectively, open the sidebar with Bookmarks showing, with Reading list showing, and with Shared links showing. Hitting the same shortcut again toggles the sidebar closed. Note that those are the numbers ‘1’, ‘2’ and ‘3’, not F1, F2, F3.
8. ⌘ ⇧ A, ⌘ ⇧ U, ⌘ ⇧ H, ⌘ ⇧ D
Again, that’s 4 different shortcuts, this time for the Finder. OK, hands up all those that switch to the Finder, hit Command-N to open a new window, and then click to their Desktop? Save yourself the grief (and the time), just activate Finder and hit Command-Shift-D. No need to open a window first, the shortcut will open a new window for you showing your Desktop folder. The others do the same thing but for different folders: Command-Shift-A opens your Applications folder, Command-Shift-U the Utilities folder, Command-Shift-H opens your Home folder.
9. ⌘ F1
This one’s for those of you with dual monitors or when your Mac is hooked up to a projector. Command-F1 toggles between mirroring and separate monitor views. Great for presentations.
10.(fn) ⌃ F8
Control-F8 activates the Status bar (top right row of icons on your screen). Use the left/right arrow keys to move along them, and the up/down arrows to select things in any menu. Hit ‘enter/return’ to perform the action. Note that on laptops you may need to invoke the fn ‘Function’ key, depending on your settings in Keyboard Preferences.
This is a great one for turning on/off Bluetooth, Wifi and checking Mavericks’ new Energy monitor in the Battery/Power icon. Mostly though, I use it for manually starting Time Machine backups or entering TM’s “star wars” interface (oh, and as a bonus…note that (fn) ⌃ F2 activates the other side of the menu bar (i.e., the menu). Enjoy!
Got your own favourite shortcuts? Tell us what you use most in the Comments! 🙂
There appears to be a bug in Apple’s Notes.app in OS X 10.8 that sometimes causes notes to disappear. If you need to recover the text of any note that’s gone missing on your mac, try pasting this into Terminal.app. It’s all one line, so copy and paste it as a whole, then hit ‘return’:
cd ~/Library/Containers/com.apple.Notes/Data/Library/Notes; strings NotesV1.storedata | grep body | open -f
The resulting TextEdit window can be searched or scrolled through for your missing text.
Due to this problem, a lot of people are giving up using Notes.app altogether and switching to alternatives like Evernote or nvALT. However, if for some reason you either want or need to keep using Notes, you might try saving that command above as its own little App that you can keep in the Dock (beats having to dig out or remember the Terminal code all the time).
To do so, we’re going to use the Applescript Editor. It lives in the /Applications/Utilities folder, but you can access it simply by clicking on Spotlight and typing
With the Editor open, paste this into the window (note, this is slightly different from the command above, as we have to tell AppleScript how to use the code):
do shell script "cd ~/Library/Containers/com.apple.Notes/Data/Library/Notes; strings NotesV1.storedata | grep body | open -f"
Press ‘Command-K’ to compile the script. If there are no errors, hit ‘Command-R’ to run it. If everything checks out fine, then hit ‘Command-S’ to bring up the ‘Save’ box. Change the ‘File Format:’ option at the bottom to ‘Application’, give the app a snappy name (‘findNotes’ ??), then choose your Applications folder as the destination before hitting ‘Save’.
The last thing to do is to go to your Applications folder and drag the icon to the Dock. Now, you can simply click the icon to produce the TextEdit window whenever a Note goes missing.
(Final tip: by the way, if you don’t like the Applescript icon in your Dock, you can always change it to an icon of your own! 🙂 )
getting to grips with AppleScript
sync notes between mac and android
FastTasks – download the free OS X utility from Applehelpwriter
Having an Android phone and Mac computer…not what intended…and syncing between them can be awkward. I use Dropbox on both and avoid iCloud for various reasons (some related to security), but I’ve always found syncing notes a problem.
My Android phone has a couple of notes apps , and my mac has, of course, the horribly-lemon yellow Notes.app, Stickies, and even Reminders can be used , not to mention TextEdit. The result was that I found I was keeping notes in several different places and then not knowing where to look for something, hunting around trying to remember which application I’d left some obscure, half-remembered note to myself in.
Then I discovered Notational Velocity. This is a great, free, open-source app built for OS X and it’s got all the simplicity I need. It has a combined search and enter field reminiscent of Spotlight, and the interface is clean, fast and efficient, requiring minimal keystrokes or trackpad movements.
On top of that, it can link to a web-based notes app called Simplenote.com (also a free service). Notational Velocity and Simplenote.com will sync to NV’s sister-app, Notational Acceleration, which is basically NV for the Android platform (and if you have an iOS device, there’s a Simplenote app for that too). Once you’ve got all of these set up, you have a notes app that you can use on your android phone, you mac, or log into from someone else’s computer, and the whole lot will keep in sync automatically!
Total cost: zero! 🙂
Click on the Trash can on the Dock, hold down the ‘option’ key and click the ‘Empty’ button over there on the left side of the window. If this doesn’t do it, the file may be in the .Trash folder of your Time Machine (TM) or some other disk.
To find out if that’s so, follow this procedure:
1. Open Terminal.app, copy and paste the following command into the Terminal window
defaults write com.apple.finder AppleShowAllFiles TRUE; killall Finder
Then press ‘Return’.
2. Open a Finder window. Navigate to the TM disk starting from its icon in the left hand column. You should see some greyed out folders called .Trash and .Trashes. Click on these and have a look for the boot.efi file that we’re hunting down.
To remove the boot.efi file from the hidden trash, try the following:
3. Go back to Terminal and copy and paste the following:
sudo rm -rf
Do NOT press ‘Return’. Instead, press the Spacebar once, then use your cursor to drag the boot.efi file from the hidden folder in Step 2 and drop it in the Terminal window. Now press ‘Return’. You will be asked for an administrator password and given a warning which you can ignore. Type in your password, but notice that your typing will be invisible, so type carefully.
4. If you typed your password incorrectly, repeat step 3. If you typed it correctly, hopefully, your Trash is empty. 🙂
5. The last thing is to hide all the hidden files again, so copy and paste this:
defaults write com.apple.finder AppleShowAllFiles FALSE; killall Finder
Then press ‘Return’.
You can close Terminal now.
Developing apps for iPhone, iPod, iPad, and Mac OS seems like the California gold-rush of the 21st century — the press are full of reports of the riches to be had in this amazing land, stories of “little people” making “big bucks”. Anyone can be an app developer, they say, but what’s the truth behind the hype, and how do you actually learn how to do it?
Last I heard, there were currently something like 600,000 apps on the Apple App store (for iPhone/iPad) and some 100,000 or so on the App store for Mac OS. Apple have paid out (i.e., passed on customer payments after taking their 30% cut) literally billions of dollars to developers. That’s a lot of cash! The question is, can you get a slice of it too?
In theory, there’s no reason why not. As I’ll detail below, the route to becoming an app developer is not particularly hard, nor is it particularly costly. But that doesn’t guarantee success. Anyone can write a book, but writing a killer book that’ll sell like Harry Potter is not so easy, and writing a killer app that will sell like Angry Birds is every bit as difficult.
The analogy holds for success in both cases: you need a great idea, you need to execute it well, and you need to market it properly. Did I mention those 600,000 apps on the App store? How, exactly, are you going to make your fortune if your app is buried in a pile like that? Well, I’ll save ideas and marketing for a future post. In this one, I want to focus on the things that we know we can achieve and only have to depend on ourselves for: developing the skills needed to turn that great idea into an actual piece of software that will run on Apple machines.
Learn the language
If you want to write a killer novel, the first thing you have to do is learn the language that you want to write the novel in, be it French, Chinese, or English. If you want to write a killer app, the same goes. Visual Basic? Visual C++? Java? Yes, that kind of thing except…if you’re developing for iOS (the iPhone/iPad operating system) or Mac OS (Mac computer operating system) you have to learn the Apple language, not any of those common ones associated with lesser machines!
So what is the Apple language? It’s called ‘Objective-C’, and it runs in a programming environment called ‘Cocoa’. You’ll need to learn ‘Cocoa’, but in order to learn that you’ll need to learn ‘Objective-C’, and to learn that, you’ll need to learn the basics of the standard (Ansi) C programming language. Oh my!
And once you’ve got a hold on all that, you’ll then need to learn Xcode, which isn’t a language or a programming environment at all, but a very sophisticated development tool (in fact, Xcode is itself an app!), in which you do all your Apple programming. You’re probably now thinking that it’d be easier to write that next Harry Potter novel and are already hunting around for the back of an envelope to start scratching down your ideas, but wait…
I know it sounds disheartening, but there is some good news. After all, it can’t be that hard if so many other people are doing it, right? (Well, actually, yeah it can, there’s a lot of dedicated programming geeks out there!). But look, I’ve been down this road too, and while I haven’t produced any killer apps (still waiting for that great idea…), I have gone from knowing next to nothing about programming to being able to put together an application that does what I tell it to and doesn’t crash my system.
(OK, not entirely true that I didn’t know anything about programming: in the 1980s, I once learned how to get a monochrome computer screen to print “Hello World” in BBC BASIC, which basically involved nothing other than typing >Print “Hello World”; it seemed so ridiculously pointless in 1982 that it turned me off programming for the next thirty years! Other than that, I’m a newbie 🙂 ).
And the good news gets better: most all of the documentation you need to learn how to be an app developer is available free from Apple. Truly, and I mean this with no trace of irony, it is hugely generous of Apple to put the amount of free material they have online for anyone to use. Want to be a Windows developer? Find your local bookstore and start shelling out one heck of a lot of $$$!! The cynical, of course, will say that Apple only do the giveaway to benefit themselves; others might say that giving away free training justifies their 30% cut.
I think of it as a symbiotic relationship: would-be developers who aren’t in computer science departments or big companies could never afford to buy all the material. Likewise, Apple could never have built an App store with such a huge number and wide variety of programs to Wow! their users if they had only had universities and commercial software developers to rely on. This way, both the little people, that’s me (and — I’m assuming — you), and Apple get to win.
I’ll tell you how to get started in a minute, but before I do let me point out that the ride is not entirely free. There’s probably a point at the beginning and certainly one at the end where you will need to lay out some of your hard-earned. So let’s deal with that now.
What you need
Right off, you’re going to need a Mac computer. Sorry, if you don’t already have one, you’re going to have to buy one; a low-range Macbook Air or Mac Mini will do, anything that can run OS X Lion. You can’t develop Mac apps on your iPhone or iPad, I’m afraid (but it does work the other way too: you don’t need an iPad or iPhone to develop apps for these devices. More on this below).
And what about if you have a good-spec PC? Yes, you could get a Mac emulator (VMmare) or mess around with OSx86, but frankly, these options are likely to cause you more grief than they’re worth; you could end up with apps that don’t build properly, and/or which breach Apple’s licensing conditions.
I’m not saying don’t do it, that’s your choice; I am saying your chances of successfully building an app, making it stable, and getting it accepted into the App store by Apple are significantly reduced if you go that route. Given the price of a basic Mac Mini on Ebay, you could well end up spending more money (as well as time) trying to avoid buying a Mac than just buying a cheap one.
The other expense you might need to lay out for is a basic ‘Intro to C’ book. There’s plenty of web offerings, but really a good ‘idiots’ book like the Dummies or Absolute Beginners should be enough and has the benefit of being reasonably likely to get you to the level of proficiency you need in the shortest amount of time. After that, you learn the rest for free (Objective-C, Cocoa, Xcode) from Apple. At the end of the process, when your app is built and you want to submit your app to the App store, you’ll have to register with Apple for a licence as an app developer and vendor; current cost $99.
Take the first step
“Sign me up”, you say, “where do I start?” The first thing to do is to sign up to Apple’s developer community: this is free (don’t confuse it with the Developer Program or Licensing, which costs $99 and which you don’t need till you’ve built an app you want to upload to the App store).
Once you’re in the Developer Community, download Xcode 4, Apple’s development environment (a different thing from a programming environment, but don’t worry, you’ll get the hang of all this terminology easily enough once you start reading the docs). This is a 4GB monster of a program – bigger than your average operating system, so make sure you have the space – and it is also free. Xcode comes with free iPhone and iPad simulators and in itself, this is a piece of software that’s probably worth a couple of thousand dollars. So smile: you’re already making a profit even though you bought that Mac Mini! This is also the reason why you don’t need, and in fact can’t use, your own iPad or iPhone to test your apps: everything has to be done in Xcode, and this monster app only runs on Mac OS X.
Once you’ve downloaded Xcode, you can play around with it if you want, but unless you’ve worked with an IDE (integrated development environment) before, it’s pretty complicated, so it’s best to wait till you work through the tutorials. It’s not the kind of software you can learn through serendipitous exploration.
Instead, go to the documentation resources and start with the tutorial Your First Mac Application.
By the time you get through this, you’re going to realise why you need to learn Ansi C, Objective-C and Cocoa. So put Xcode away for now, and start on the path of learning to speak Apple’s language. When you get there, just add 1 great idea + 1 great marketing strategy, and you’re on your way to California!
The short guide:
1. Get a Mac
2. Learn C, learn Objective-C, learn Cocoa, learn Xcode
3. Come up with an idea for a great app and plan it out carefully
4. Build and test your app
5. Pay the licensing fee and submit your app to Apple
6. Once it’s been through the review process and accepted, implement your marketing strategy
7. Watch the millions role in and retire.