Author Topic: Ryanair Account Hacked €4.6Million  (Read 17035 times)

Offline Luther

  • Venus
  • **
  • Posts: 70
Re: Ryanair Account Hacked €4.6Million
« Reply #15 on: May 10, 2015, 07:42:55 PM »
Also, overwriting doesn't do what you think it does when the target is an SSD.

Not a situation I've had to face yet, but, since you brought it up :)  what does it do?  Also, does "overwriting" refer to one method or the other (that is, the "dd" method or the firmware method), or to both?

Offline ka9q

  • Neptune
  • ****
  • Posts: 3014
Re: Ryanair Account Hacked €4.6Million
« Reply #16 on: May 11, 2015, 02:34:42 AM »
The big difference between solid state flash memory and conventional magnetic disks is that magnetic disk sectors can be selectively rewritten; flash can't. That changes everything.

Flash memory is organized into pages, typically 4 kiB each, analogous to magnetic disk sectors. Flash and magnetic pages/sectors can be read randomly, but a flash page can only be written once. Flash pages can be reused only after being erased, a slow (milliseconds) operation that must be done on a large block of pages at once. Block sizes vary, but are typically 256 kiB or more.

SSDs have controllers to emulate the conventional magnetic disks they replace. They automatically map virtual disk sectors to physical flash pages and erase blocks (groups of pages) as needed to free up space for additional writes. If you write sector 10 on a brand new drive, the controller will assign a flash page to hold it that could be anywhere in the physical flash memory. If you then overwrite sector 10, the controller will assign a new flash page, write it with your new data, and map it so it will read back as sector 10.

Here's the crucial part: the controller will not immediately erase the flash page holding the previous version of sector 10; it will simply mark it as no longer in use. When the supply of free pages runs low, the controller will find a block of physical flash memory with the smallest number of active pages, copy those still-active pages to another block, and erase the block. (Remember, only an entire block can be erased; individual pages cannot be.)

You see where this is going? Writing zeroes onto the virtual disk sectors doesn't actually destroy the data in the flash memory, at least not right away. It simply assigns fresh pages to hold the zeroes you're writing and marks the old pages to be erased and reused at some future time.

One of the early big problems with SSDs is that the SSD controller had no way to know when you deleted a file; the only way it knew you no longer needed the contents of a given sector (page) was when you actually overwrote it with new data. This caused a lot of unnecessary copying of discarded data, which can really slow down writes; the problem is called write amplification.

This led to the TRIM command, which lets the file system tell the drive that the data in certain sectors is no longer needed. This greatly simplifies life for the controller when it decides to erase a block because it need not copy out TRIMmed pages. While the spec requires that a TRIMed virtual page immediately read back as zeroes, it does not require the immediate erasure of the physical page with the previous data. So even with TRIM, you can't be totally sure that you've actually destroyed anything. A sufficiently sophisticated forensic analysis that bypasses the controller and reads the physical flash pages directly might still find it.

And it gets still worse. Some controllers compress your data to use physical flash more efficiently and achieve faster I/O speeds. So writing binary zeroes, which compress very well, over the entire virtual drive might not even touch much of the data previously stored in physical flash.

The bottom line is that the best way to destroy everything on the SSD is to use the special secure erase command. It's also much faster than overwriting.

And even then you might not want to trust the drive firmware to do what it's supposed to do, e.g., because it's been hacked by some 3-letter government agency. So here's what to do if you're truly paranoid:

1. Issue the secure erase command. This is pretty fast on an SSD, taking typically only a few minutes.
2. Fill the entire drive with random data (not zeroes) to defeat any data compression.
3. Read back the random data to ensure it's exactly what you wrote. (Generate (pseudo)random data with a keyed cipher to make this easier.)
4. Issue the secure erase command a second time.

This will work unless the drive is at least twice as large as advertised to give malicious firmware room to save your previous disk image while still storing and recovering the random data you wrote in step #2. The SSD market is very competitive, so no manufacturer is going to label his drive as containing only a fraction of its true capacity.

Of course, there's a much simpler solution to all this: never write plaintext to the drive in the first place. Always use a disk encryption system like LUKS, Filevault or Truecrypt. Then all you have to do to make the stored data unrecoverable is to destroy all copies of the cipher keys.

Offline Luther

  • Venus
  • **
  • Posts: 70
Re: Ryanair Account Hacked €4.6Million
« Reply #17 on: May 12, 2015, 05:57:51 PM »
One of the early big problems with SSDs is that the SSD controller had no way to know when you deleted a file; the only way it knew you no longer needed the contents of a given sector (page) was when you actually overwrote it with new data. This caused a lot of unnecessary copying of discarded data, which can really slow down writes; the problem is called write amplification.

I guess this was a problem with any file system not specifically designed to deal with it.  If I understand these things correctly (a questionable proposition), then at the hardware level, there's no such thing as marking disk space unusued; there are only overwrite operations.  Knowing that a particular part of the disk was unused would require the disk to understand what particular file system had been written upon it; is that correct?

I further suppose that the introduction of the TRIM operation you describe does not require a revised filesystem, but rather an update to some of the OS routines, but works only if the various applications access the disk using file-based operations, rather than sector-based operations.  On the right track?

And it gets still worse. Some controllers compress your data to use physical flash more efficiently and achieve faster I/O speeds. So writing binary zeroes, which compress very well, over the entire virtual drive might not even touch much of the data previously stored in physical flash.

LMAO.  So you write a trillion zeroes to the disk, and it overwrites maybe 12 bytes :)

The bottom line is that the best way to destroy everything on the SSD is to use the special secure erase command. It's also much faster than overwriting.

And even then you might not want to trust the drive firmware to do what it's supposed to do, e.g., because it's been hacked by some 3-letter government agency. So here's what to do if you're truly paranoid:

1. Issue the secure erase command. This is pretty fast on an SSD, taking typically only a few minutes.
2. Fill the entire drive with random data (not zeroes) to defeat any data compression.
3. Read back the random data to ensure it's exactly what you wrote. (Generate (pseudo)random data with a keyed cipher to make this easier.)
4. Issue the secure erase command a second time.

This will work unless the drive is at least twice as large as advertised to give malicious firmware room to save your previous disk image while still storing and recovering the random data you wrote in step #2. The SSD market is very competitive, so no manufacturer is going to label his drive as containing only a fraction of its true capacity.

The solution I might move towards (which should be viable for me, but perhaps not for everyone) is just to keep old drives.  Maybe use them as bookends.

Of course, there's a much simpler solution to all this: never write plaintext to the drive in the first place. Always use a disk encryption system like LUKS, Filevault or Truecrypt. Then all you have to do to make the stored data unrecoverable is to destroy all copies of the cipher keys.

I've heard of these systems, and need to investigate further.  I am mainly OS X-based these days.  There is a "secure erase" function, but (i) it is painfully slow, and (ii) it only works on files that are deleted by manual action through the GUI.  Better than nothing, I guess, but a long way from perfect.

Offline Zakalwe

  • Uranus
  • ****
  • Posts: 1598
Re: Ryanair Account Hacked €4.6Million
« Reply #18 on: May 13, 2015, 03:17:21 AM »


The solution I might move towards (which should be viable for me, but perhaps not for everyone) is just to keep old drives.  Maybe use them as bookends.


A very large hammer will permanently erase ANY hard drive.
"The strain of anti-intellectualism has been a constant thread winding its way through our political and cultural life, nurtured by the false notion that democracy means that 'my ignorance is just as good as your knowledge.' " - Isaac Asimov

Offline ka9q

  • Neptune
  • ****
  • Posts: 3014
Re: Ryanair Account Hacked €4.6Million
« Reply #19 on: May 13, 2015, 05:07:07 AM »
at the hardware level, there's no such thing as marking disk space unusued; there are only overwrite operations.  Knowing that a particular part of the disk was unused would require the disk to understand what particular file system had been written upon it; is that correct?
Exactly right.
Quote
I further suppose that the introduction of the TRIM operation you describe does not require a revised filesystem, but rather an update to some of the OS routines, but works only if the various applications access the disk using file-based operations, rather than sector-based operations.  On the right track?
Yes, but it's rare to directly access a disk containing a filesystem except with tools for filesystem integrity checking and/or repair.

There are two approaches to using TRIM. The most obvious is to have the filesystem automatically issue it whenever a file is deleted. Copy-on-write file systems can issue TRIM commands for rewritten blocks, unless of course they're needed for a snapshot.

The problem is that on some SSDs TRIM is rather slow, and might block at an inconvenient time while it performs garbage collection (copying of remaining pages and erasure of blocks). TRIM only needs to be done often enough to maintain an adequate supply of erased pages for new writes. So some systems let you manually trim when you've got the time. Linux provides the fstrim command that works on XFS, among others. It just feeds the current file system free block list to the drive in a series of TRIM commands.
Quote
The solution I might move towards (which should be viable for me, but perhaps not for everyone) is just to keep old drives.  Maybe use them as bookends.
Nah. You still run the risk of having them lost or stolen. I do recommend the secure erase command; it's fast on SSDs and I've never heard of it actually being compromised by a three-letter-agency (NSA, FBI, CIA, BND, GCHQ -- okay, four). But all bets are off if you're the North Korean nuclear weapons development agency....and in that case I never told you any of this anyway.
Quote
I've heard of these systems, and need to investigate further.  I am mainly OS X-based these days.  There is a "secure erase" function, but (i) it is painfully slow, and (ii) it only works on files that are deleted by manual action through the GUI.
On OSX, just use Filevault. After Ed Snowden I'm not sure I fully trust Apple these days, but it's certainly easy to use.

The "secure erase" command on OSX was designed for magnetic disks, so it simply overwrites data before erasing it. That's why it's slow. Given all the caveats just discussed, this is strange since most newer Macbooks now come with SSDs. Supposedly, OSX automatically invokes TRIM on Apple-supplied SSDs, and that solves the problem. But I'd want to verify this before I really rely on it.

On the other hand, Filevault (if it works as advertised) is an even better solution. It saved my butt three years ago when my last Macbook was stolen. All the data was encrypted and backed up, and the machine itself was four years old and rapidly failing so basically I lost nothing but the value of the SSD I had retrofitted into it. Oh, and the accessories in my bag. I had a really nice 405 nm laser pointer in there.
« Last Edit: May 13, 2015, 05:11:54 AM by ka9q »

Offline ka9q

  • Neptune
  • ****
  • Posts: 3014
Re: Ryanair Account Hacked €4.6Million
« Reply #20 on: May 31, 2015, 04:20:43 AM »
Does anyone here believe Margie was considering an affair with Mike Yanagita?  Because I heard that suggested during discussion of the movie on a site I frequent when Fargo was Movie of the Week, and I'm not buying it.
I know this is a bit dated, but...

Absolutely not. She was simply being sympathetic, and politely backed off when she realized Yanagita was trying to take advantage of her.

Margie appeared almost naive at times, and that was another great aspect of her character. She reminded me of Detective Columbo, whose adversaries always underestimated to their great detriment. Yet she seemed honestly taken aback when she sees Jerry Lundegaard driving away from the dealership: "He's fleeing the interview! He's fleeing the interview!"
« Last Edit: May 31, 2015, 04:23:19 AM by ka9q »