Category Archives: 10.11
ConsoleSpy is a simple but powerful little app that offers a window into system.log and which can trap incoming messages meeting user-defined search criteria. It’s aimed at software testers, bug hunters, security researchers and anyone who needs to do analytical troubleshooting work on a mac.
Minimum system requirements: OS X 10.11. ConsoleSpy is currently free.
Here’s an intro to its features and how you can use ConsoleSpy to aid in analysing your mac and your software.
What does it do?
The best way to illustrate the use case for ConsoleSpy is to consider a couple of ‘based on a true story’ user problems I’ve encountered recently.
Case 1: In one case, a user was concerned that an attacker was logging into her computer remotely. Unaware of how that might be happening, the user searched the Console.app and found a number of suspicious remote login attempts. However, these always seemed to occur at times she wasn’t at the computer and sometimes weeks apart. It became a laborious job and anxious routine for her both to remember and to search through the Console logs every morning to see if anything suspicious had occurred.
Case 2: In the second case, a user realised that the Time Machine backups she’d been relying on had been silently failing to pass verification checks. There was no indication from Time Machine itself, and she only discovered the problem, weeks after it had began, by a fortuitous glance at the Console.app where she discovered multiple ‘Backup verification failed’ messages.
In both these cases, ConsoleSpy could have alerted the user to the problem as soon as it had occurred. ConsoleSpy allows you to set search terms to trap incoming messages. Both a Dock badge and a visual indicator in ConsoleSpy’s display indicate when a message has been trapped. By using the search term ‘sharing,’, our user worried about remote hacking would have instantly been able to see if a log in had been attempted and when. Our user with the failed backup problem would have likewise been alerted instantly the first time the problem occurred by using ‘backup,’ or ‘backup verification,’ (if she had only wanted to trap specific verification messages) as Alert strings.
ConsoleSpy becomes more useful the more accurately you know what you’re looking for. For bug hunters and software developers, simply setting an alert on your app or process id name will immediately funnel all incoming messages into ConsoleSpy’s ‘Alerts received’ box, allowing you to exercise your app in various conditions and immediately see the results. You can get as specific or as general as you want, but do see the help on Alert string syntax.
How do I use it?
After launching ConsoleSpy, you’ll be presented with an ‘always on top’ display of the most recent message into the Console. You can move the display around by clicking anywhere in the black part of the display and dragging. The four buttons on the right hand side offer you access to all of ConsoleSpy’s main functions, clockwise from top left:
i. Freeze the display: in the event that you see something interesting and want more time to read it before the next message comes in, you can lock the display by clicking the little padlock button. When locked, the text in the display changes colour and a padlock appears at the end of the text. Note that when the display is locked, the View buttons in the Preferences window (See below) will have no effect. Click the padlock again to unlock the display.
ii. Hide ConsoleSpy: click the orange button to hide ConsoleSpy. Often, you won’t want the display visible but you will want ConsoleSpy to keep watching for alerts. You can also hide the app with ‘Command-H’.
iii. Open Console.app: the little ‘eye’ button immediately opens Console and takes you to the most recent message in system.log.
iv. Preferences: this is a toggle button that opens or closes the Preferences drawer. We’ll get to that next.
The controls on the far left should be self-explanatory, but a couple of notes are in order.
View: As mentioned above the ‘View’ buttons are disabled when the display is locked, but otherwise they toggle the length of the display. The ‘Long’ view is particularly useful when reading multiple messages in the ‘Alerts received’ box.
Frequency: this controls the frequency at which ConsoleSpy updates the display. Note that ConsoleSpy continues to scan for messages that meet your Alert string criteria even between polls regardless of whether the app is visible or hidden, or the display is locked (see above). ConsoleSpy’s buffer can handle up to 40 messages between polls. If ConsoleSpy’s buffer is flooded with more than that, the display will show a ‘Flood’ warning. For more information see ‘The Hoary Gory’ section below.
Alert Strings: this is the most important field you’re going to want to manage. When you first launch ConsoleSpy, you’ll see some default search strings are already included by way of example. You can remove or add to them by clicking the ‘Edit’ button at the bottom left of the text box. Search string syntax is fairly basic, but allows you to be as specific or as general as you wish. Ensure that each term is comma-separated and the entire list is comma-terminated (i.e, there should be a comma after the last search term in the list, too). Click the ‘?’ button to go to the support page giving examples of search string syntax. Drop us a line in the Comments if you need help or contact Sqwarq support.
Alerts received: this is the main display for your results. You can select and copy all or parts of the message to search in Console.app if you want to see the message in context. Using the date string without the seconds is a particularly useful way to search for messages in Console if you want to see what else was happening around the same time.
You can clear the ‘Alerts received’ box (and the Dock badge and the display alert symbol) by clicking the ‘-‘ minus button at the bottom left of the text box. We suggest regularly and promptly removing messages from the Alerts received box once you’ve read them as the messages are already archived in Console.app.
The Hoary Gory
ConsoleSpy polls the system log every 1, 2 or 5 seconds according to the Frequency setting in the Preferences, and displays the most recent message. Unless the system log is being flooded with more than 40 messages since the last poll, ConsoleSpy won’t miss a thing and you’ll get an alert if any message meets your search criteria, even if it wasn’t displayed in ConsoleSpy’s display. If ConsoleSpy’s buffer is flooded, a small ‘flooding’ alert symbol shows in the display. The start and end flood times can be displayed in the Alerts Received box by setting an alert string for ‘flood,’.
If you experience a lot of flood warnings (entirely possible in scenarios where you are beta testing software or even the operating system itself), try using a faster frequency (i.e, 1 sec). While this may seem counterintuitive, it is a consequence of ConsoleSpy’s fixed buffer size. The buffer can hold up to 40 new messages since the last poll, so the amount of messages ConsoleSpy can search between each poll is 40/(frequency). As we develop the app, we plan to include a choice of larger buffer sizes. The current buffer size is a conservative choice designed to ensure the app is usable even on smaller, less powerful macs.
If you’re already using the fastest poll time of 1 sec and flood warnings are occurring constantly, this is a good sign that some software is not behaving as intended. Of course, when testing beta software, especially a beta OS, there may be so many deliberate logs to the system log that ConsoleSpy reports flooding almost all the time. This is not a problem for ConsoleSpy; indeed, having ConsoleSpy alert you of flooding is one of its intended functions, so that you can see just when and how often something is happening. The main thing to be aware of during times of repeated or constant flooding is that ConsoleSpy may not be able to search every single message received against your search terms. You can, of course, turn Alerts off during such times, but a better solution is to leave Alerts on (ConsoleSpy will still return most if not all search hits, depending on how severe the flooding) and simply use the Console.app itself to do an historical search to see if any crucial messages you would have expected but which did not get spotted by ConsoleSpy are in the log.
Note that while Alert string searches begin as soon as ConsoleSpy is launched, flood detection is not enabled until 30 seconds after launch. This is due to the fact that ConsoleSpy’s buffer needs to be full before it can determine the rate of incoming messages.
That about rounds up our introduction to ConsoleSpy. We hope you find it useful, and if you have any questions, drop us a comment or email us at Sqwarq support.
To help ameliorate that, I’ve produced a script that will abort a scheduled backup task using Carbon Copy Cloner if a user-defined percentage of changes have occurred in a designated ‘Canary’ folder.
Here’s how it works. In order to be successful, ransomware must change a large percentage, if not all, of your personal files in your Home folder by encrypting them. That means we can determine if a folder has been encrypted by looking for an unusual amount of changes or additions since the last backup.
A Canary folder is a folder that we use to warn us of precisely that. It should be a folder that contains some random dummy files (.doc, .png, .xls files etc), and/or a folder which you don’t make large changes to from one backup to the next. The script itself will change the folder slightly each time it runs, to ensure that the Canary folder does not look like it’s ‘stale’ (which might cause an attacking script to ignore it).
The key to the Canary is that the percentage of files changed or added on each scheduled backup is less than the threshold you set in the script. The default is set to no more than 10%. If the number of files changed or added is higher than that, then the backup aborts. You can of course change the default to a bit higher if you use a ‘real’ folder that you don’t change often, but remember we’re only talking changes between one scheduled backup and another, so it will also depend on how frequently your backups are scheduled.
For example, I have a 2-hour scheduled backup and I use my ‘Documents’ folder as the Canary. Since I only use that folder for long-term archives, it is actually rarely changed, and certainly never as much as 10% within 2 hours, and that makes it a perfect choice as a Canary. You can pick any real, rarely used folder or you can set up a complete dummy folder if you prefer.
If you do pick a real folder, keep in mind its size. The larger the folder, the longer it’s going to take the script to determine the differences between it and the last backup of it. A couple of thousand files is OK, but once you get into the tens of thousands you might find the script takes several minutes to complete. With only a few hundred files in my Documents folder, it takes literally a second or two.
Here’s a sample output from the log file the script produces in the ‘Canary’:
Destination /Volumes/Backup Disk/Users/phil/Documents has 360 files in the folder. There are 3 changes between it and the source /Users/phil/Documents. The threshold for aborting the task is 10 percent, or 30 changes. Result: task will run.
For our strategy to be successful, we need to ensure the attacking script doesn’t ignore the Canary and does try to encrypt the Canary before the next backup is scheduled. For that reason, if you opt for a complete dummy folder, you might like to give it a name so that it’s somewhere near the beginning (alphabetically) of your Home folder. Since the Canary folder will be slightly modified each time the script runs, it should get hit early if the attack is looking either for recently modified files or just starts trawling your home folder in ascending name order (and I know what you’re thinking: what about descending order? Sure, you could add another Canary at the end, and modify the script to check both ;)).
Note that this script is for use only with a regular, scheduled backup task, and only for use with Carbon Copy Cloner (version 4). We’ll be posting about Time Machine strategies later.
Another note of caution is that while this script should stop your scheduled CCC task from infecting a backup drive, it won’t stop an attacking script from attempting to encrypt any mounted drives it finds by itself. That really depends on the sophistication of the attack. To that end, we’ll soon be posting a general strategy for detecting a ransomware attack on your internal drive using multiple Canaries and a bit of Folder Action script magic. Stay tuned for that.
In the meantime, here’s the script. Due to the vagaries of WordPress.com formatting, I’ve hosted it over on my pastebin. Please read the extensive comments, which also explain how to set it up and how to use it. Any questions, drop a comment below. 🙂
Picture Credits: ‘Caged Egg’ by Marije Berting
If you’re not familiar with this junior partner in our troubleshooting suite of apps, App Fixer does a very specific job: it returns any app you select to its default preferences and settings with the click of a button.
It’s raison d’être is largely for those apps that get themselves stuck in some unresponsive state (looking at you anything-Adobe). It also does a neat trick rescuing Safari from Adware on the side. ;).
If you’ve been using App Fixer already and you’re currently running El Capitan, we’re afraid you won’t see an update notice (the blame for that lies with Amazon AWS, but that’s another story). Just go to the App Fixer home page and download directly. We’ll be introducing an in-app updater (Sparkle) in the next release to make future updates more convenient.
The standard method for doing this, either in Terminal or in code (via NSTask) has always been
defaults write com.apple.finder AppleShowAllFiles -bool true; killall Finder.
In every version of OS X from 10.6 thru to 10.10 this works as expected. In 10.11 I see a reproducible though not always consistent bug when using any GUI-wrapped app to toggle this Finder setting. The bug basically ends up with Finder showing the opposite of both what the app shows and what ‘defaults read’ shows (see image above; the value should be ‘1’ when invisible files are visible).
We believe a related bug is that Finder sometimes fails to show the new version of an app in the Finder preview after the app has been updated.
In both cases, there’s a couple of ways to deal with it (albeit temporary until Apple applies a proper cure):
i. log out then log in
ii. issue a ‘killall Finder’ on the command line
iii. purge the RAM then toggle the setting again in whatever app you’re using to toggle invisible files.
We just released version 2.04 of DetectX, but if you’re finding that you can’t update, here’s the reason (and the fix!).
Thanks to Apple’s new AppTransport security layer and Amazon S3 not supporting perfect forward secrecy protocol, users of DetectX on any version prior to 2.04 who are running El Capitan (OSX 10.11) will find that DetectX can’t be updated from within the app even though we use a secure https connection.
The simple, if inconvenient fix, is to download DetectX directly from sqwarq. We’ve fixed the bug so that once you’re on v2.04, the in-app update mechanism will function properly again (even on El Capitan !).