nlited

The integration of the Crypt code went not as smoothly as I hoped and better than I feared. The original Crypt code was focused on using public/private keyfiles and did not use passwords at all. CryptDisk does not use public/private keys, instead using passwords to decrypt AES keys. So this required some fairly major surgery to Crypt. Fortunately, I was able to do almost all this work in the user-mode library. I then migrated just the minimum required into the kernel-mode CryptKM. The KM version doesn't need to deal with passwords, elliptical keys, or text -- it is given the pre-parsed key data and just needs to create the AES key and perform the bulk transcryption.

Instead of debugging the CryptKM code, I am once again struggling with VMware networking gremlins and WinDbg symbol buggery. For unknown reasons, Visual Studio can't see the remote debugger and refuses to deploy. Then WinDbg insists on downloading symbols from Redmond every time it runs, which kills the Visual Studio connection. And today, for unknown reasons, WinDbg keeps complaining that the pdb for my driver is mismatched.

I sorted through some problems with uploading the keys to the driver, and now the driver is able to generate the AES key, write the new volume header, and read it back.

The first read failed because I was expecting encrypted data when reading a blank sector from a newly created volume. I could build a (giant) bit table to tell me if a sector is encrypted, or (much simpler) treat a decryption error as a blank sector and return zeroes. I opt for Plan B.

On the first write, the CryptStreamEncode() returned 0 bytes. This was a migration bug: The UM code uses if(IsErr(...)) to check for errors while the KM code uses if(!NT_SUCCESS(...)), so I frequently get the '!' wrong.

On the second write, I ran into a problem with my "add random bytes" scheme. The random bytes need to be preserved in order to decrypt, then thrown away from the plaintext. This means I need to write an extra 16 bytes to the beginning of every sector. I can do this, but it will significantly reduce the usable space in the volume: 16/(1024+16)= 1.5% of the volume will be hidden random bytes. A 1GB volume would have 15MB that is unusable. It also means doing some funky math for the disk size and geometry queries. Or I could drop the random bytes feature for now. I am going to opt for Plan B.

BCryptEncrypt() bumps the output to the next block even if the input is a multiple of the block size. I am trying it without the PADDING flag, see if it will decrypt...

SUCCESS! I was able to create an encrypted disk, format it, write a file, read the file, and unmount the disk. The media file contained the plaintext volume header followed by lots of random data.

I was not able to remount the disk... The ZwCreateFile() is failing (C0000043: STATUS_SHARING_VIOLATION). I failed to close the file? Yes, I rebooted and then I was able to remount the volume -- but only after loading the key.

SUCCESS! CryptDisk is now mostly working! Debugging CryptKM went smoothly, about 4 hours from the first build to success. Not bad, especially considering I spent about half that time fighting network and system gremlins.

Sector Info

Basic encryption is working, now I want to extend it by adding a sector ~~prefix~~ info block to serve two purposes. The info block will begin with random data so that every sector will always be different when written to the media file -- even when the same sector is written with the same data. This prevents attacks where the attacker knows what a particular sector should contain.

NOTE: I should also scramble the sector index. File systems always store the master record in sector 0, which makes it a prime target to attack. If I scrambled the sector index through a two-way ~~hash~~ permutation, the master record would be stored in sector 1467 making it much harder to attack. The downside: This would spread file contents across the entire disk, making file I/O to the underlying media file less efficient (harder to cache). (See StackOverflow: Looking for Hash Function)

The second purpose of the sector info is to provide metadata about the sector, particularly the last write time. The timestamp can then be used when archiving or restoring a copy of the volume. I can skip copying sectors where the timestamp has not been changed. Copying my TrueCrypt volumes is a real pain when copying 200GB takes 35 minutes on even the fastest system (~100MB/s). This would be a fantastic improvement over TrueCrypt.

To make this actually more efficient, I need to avoid reading every sector just to read the timestamp. I can do this by storing all the sector info blocks in one area, separate from the sector data. This should make life simpler by keeping the size of the sectors unchanged. I can then read the sector info en masse during the bulk copy and never need to seek to or read the sector data for unchanged sectors. However, this means sector read and write operations will require two distinct media accesses. Reads will need to read the sector info, read the sector data, concatentate the data, decrypt the amalgamated sector, and return just the sector data. Writes will need to update (or simply create) the sector info, concatenate the sector data, encrypt the amalgamated sector, write the encrypted sector info, and finally write the encrypted sector data.

I simplified the DeviceDisk read/write functions by always breaking the operation into sectors for both plaintext and encrypted volumes.

Unfortunately, something was not right and CryptDisk could no longer format encrypted volumes. Plaintext volumes still worked. I needed an easier way to examine the sectors than WinDbg so I took a day to write Test Console into the app. Test Console is essentially a simple disk editor that can encrypt and decrypt sectors.

When the system tries to format the disk, it writes to sectors 0, 2038 (the last sector), 1, 2, 3, 4, 5, 6 before giving up.

Using Test Console, it appears the problem is in the encryption. The sectors written by the driver do not seem to contain valid timestamps in the sector info.

NOTE: When I read a blank sector (never been written) it always returns the same repeating 16-byte pattern:

0000: 4A 58 97 5E 0F 75 AF 97 2F BE 5E 8F BB 42 58 47 | JX.^.u../.^..BXG 0010: 4A 58 97 5E 0F 75 AF 97 2F BE 5E 8F BB 42 58 47 | JX.^.u../.^..BXG 0020: 4A 58 97 5E 0F 75 AF 97 2F BE 5E 8F BB 42 58 47 | JX.^.u../.^..BXG 0030: 4A 58 97 5E 0F 75 AF 97 2F BE 5E 8F BB 42 58 47 | JX.^.u../.^..BXG 0040: 4A 58 97 5E 0F 75 AF 97 2F BE 5E 8F BB 42 58 47 | JX.^.u../.^..BXG 0050: 4A 58 97 5E 0F 75 AF 97 2F BE 5E 8F BB 42 58 47 | JX.^.u../.^..BXG

I guess this makes sense, but was unexpected. To be reliable, decrypting the same inputs must always return the same output. If the input is always the same (ie a blank sector, all zeroes) and the IV (salt) is always the same, then I should expect to see the same output repeating every cipher block (16 bytes). So if I want to obfuscate blank sectors, I should detect them and return random data. I can detect blank sectors using the sector info timestamp.

Does this mean that the cipher from a sector that is written as all zeroes would be recognizable by a repeating pattern in the cipher? No, this is not the case. The output from encryption uses its own previous output to randomize its inputs -- the encryptor has an additional input that the decryptor does not have, which is its own previous state.

I need to take a close look at my EncryptSector() function. (UPDATE: There is a bug in there, can you find it?)

EncryptSector(): NTSTATUS DeviceDisk::EncryptSector(UINT64 nSector, BYTE *pSrc) { NTSTATUS Status= STATUS_SUCCESS; UINT BlockSz,ByteCt= Volume.SectorInfo+Volume.SectorSz; int CipherCt; if(!pPlain || PlainSz < ByteCt) { MemFree2(pPlain); if(!(pPlain= (BYTE*)MemAlloc("Plain",ByteCt))) return(Warn(STATUS_NO_MEMORY,"DeviceDisk:EncryptSector: NoMem(Plain:%d)",ByteCt)); PlainSz= ByteCt; } //Build the entire sector in pPlain struct SectorInfo_s *pInfo= (struct SectorInfo_s*)pPlain; CryptSetRandom(hCrypt,pInfo->Random,sizeof(pInfo->Random)); KeQuerySystemTime(&pInfo->WriteTime); RtlCopyMemory(&pPlain[Volume.SectorInfo],pSrc,Volume.SectorSz); //Encrypt as a single block. if(IsErr(Status= CryptStreamBegin(hCrypt,BlockSz,0,0,0))) return(Warn(Status,"DeviceDisk:EncryptSector: StreamBegin() failed.")); if(IsErr(CipherCt= CryptStreamEncode(hCrypt,pPlain,ByteCt,true,pCipher,CipherSz))) return(Warn(CipherCt,"DeviceDisk:EncryptSector: StreamEncode() failed.")); if(CipherCt!=ByteCt) Warn(STATUS_DATATYPE_MISALIGNMENT,"DeviceDisk:EncryptSector: StreamEncode() returned an odd size. %d,%d",CipherCt,ByteCt); //Write the SectorInfo from the pCipher[] stream. if(!NT_SUCCESS(Status= WriteSectorInfo(nSector,pCipher))) return(Warn(Status,"DeviceDisk:EncryptSector: WriteSectorInfo(%llu) failed.",nSector)); //Write the SectorData from the pCipher[] stream. IO_STATUS_BLOCK IoStatus; LARGE_INTEGER FilePos; Zero(IoStatus); FilePos.QuadPart= Volume.SectorOffset + nSector*Volume.SectorSz; BYTE *pData= &pCipher[Volume.SectorInfo]; UINT DataCt= Volume.SectorSz; if(!NT_SUCCESS(Status= ZwWriteFile(hMediaFile,0,0,0,&IoStatus,pData,DataCt,&FilePos,0))) return(Warn(Status,"DeviceDisk:EncryptSector: ZwWriteFile(%llu) failed.",FilePos.QuadPart)); return(Status); }

WriteSectorInfo(): // pSrc is already encrypted. NTSTATUS DeviceDisk::WriteSectorInfo(UINT64 nSector, BYTE *pSrc) { NTSTATUS Status= STATUS_SUCCESS; IO_STATUS_BLOCK IoStatus; LARGE_INTEGER FilePos; Zero(IoStatus); FilePos.QuadPart= Volume.MediaOffset + nSector*sizeof(Volume.SectorInfo); if(!NT_SUCCESS(Status= ZwWriteFile(hMediaFile,0,0,0,&IoStatus,pSrc,Volume.SectorInfo,&FilePos,0))) return(Warn(Status,"DeviceDisk:WriteSectorInfo(%llu): ZwWriteFile(%llu) failed.",nSector,FilePos.QuadPart)); return(Status); }

I first make sure my plaintext buffer (pPlain) is large enough to hold the sector info and data. (ByteCt= Volume.SectorInfo + Volume.SectorSz). I then build the info block inside pPlain[] and copy the source data after it. This lets me call CryptStreamEncode() once on the entire amalgamated sector. The sector info and data are then written as two separate calls to ZwWriteFile().

Contents of pPlain after the call to RtlCopyMemory():
ffff910e`71872000 57 de bd ac 58 ca 2c fa 7e 8f 64 29 a7 29 d3 01 W...X.,.~.d).).. ffff910e`71872010 30 c5 c2 05 00 00 00 00 50 e4 74 06 00 00 00 00 0.......P.t..... ffff910e`71872020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ ffff910e`71872030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ ffff910e`71872040 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ ffff910e`71872050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ The first 16 bytes are SectorInfo_s, 8 bytes of random and 8 bytes of FILETIME. Followed by the FAT volume header. Looks good.

The contents of pCipher[] after CryptStreamEncode():
ffff910e`7186d000 85 ee 64 ea 0b 1e 68 9f 9b b2 ee 3f 3b 22 d8 5f ..d...h....?;"._ ffff910e`7186d010 76 a4 f0 9c 1a 88 ee 6f 31 ba 9f 17 fd d6 01 b8 v......o1....... ffff910e`7186d020 f1 23 95 06 fb 09 4f fb 1f c7 bd b3 27 a2 0e d3 .#....O.....'... ffff910e`7186d030 13 11 97 43 08 2e 4c 71 51 d3 aa db bd 22 6f 6f ...C..LqQ...."oo ffff910e`7186d040 92 ca 0a 8e 2c 5b d4 32 3c 55 f9 4c 87 a6 b9 79 ....,[.2

In WriteSectorInfo(), FilePos=0x1000 (correct). ZwWriteFile() returns STATUS_SUCCESS and Information=0x10 (correct).

The sector data FilePos=0x9000 (correct), pData is d010 (correct), DataCt is 0x1000 (correct), ZwWriteFile() returns STATUS_SUCCESS and Information=0x1000 (correct). I return STATUS_SUCCESS and Information=0x1000 in the IRP. Everything looks good.

When I read (RAW) sector 0 using Test Console:
0000: 85 EE 64 EA 1D A5 F7 4C C6 BC 9D 3C CB EE 7F 09 | ..d....L...<... 0010: 76 A4 F0 9C 1A 88 EE 6F 31 BA 9F 17 FD D6 01 B8 | v......o1....... 0020: F1 23 95 06 FB 09 4F FB 1F C7 BD B3 27 A2 0E D3 | .#....O.....'... 0030: 13 11 97 43 08 2E 4C 71 51 D3 AA DB BD 22 6F 6F | ...C..LqQ...."oo 0040: 92 CA 0A 8E 2C 5B D4 32 3C 55 F9 4C 87 A6 B9 79 | ....,[.2 This matches what I saw in WinDbg, so there is no problem with the file writes. But Test Console decodes this as garbage:
Decode OK. SectorTime: E7FE,6030,66A4,95BF 0/00/00 65534:65535:65535.65535 Sector 0 Sector 0 MEDIA 0000: 89 5C F2 7C 1C DE DD AD BF 95 A4 66 30 60 FE E7 | .\.|.......f0`.. 0010: 30 C5 C2 05 16 BB 9F D3 0D EA 07 05 F0 CC A7 56 | 0..............V 0020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................ 0030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................ 0040: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................

The driver decrypts this as:
ffffdd0b`932ce000 89 5c f2 7c 1c de dd ad bf 95 a4 66 30 60 fe e7 .\.|.......f0`.. ffffdd0b`932ce010 30 c5 c2 05 16 bb 9f d3 0d ea 07 05 f0 cc a7 56 0..............V ffffdd0b`932ce020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ ffffdd0b`932ce030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ ffffdd0b`932ce040 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ Fortunately, this is the same garbage that I see in Test Console. This is a good thing, as it points the fickle finger of blame straight at CryptStreamEncode() and also tells me that CryptStreamDecode() is probably OK. It also tells me that the file read/write is OK. I also know, from working with Test Console, that my UM version of CryptStreamEncode() is OK. The fact that I could decode the stream to the same garbage in both Test Console and Driver tells me that the key and the code to create the keys is probably OK.

I noticed that in the CryptKM StreamBegin() I was not always allocating the pIV buffer. This may have resulted in using a random (but static?) IV. This change had no effect.

I need more information... I write the FILL and VOLUME WRITE commands. This will allow me to write a sector using the driver (volume write) and read it back from the media file (media read), and vice-versa. I also changed the file sharing in the driver so I can open the media file from the app while the driver has it open.

> sector fill count volume Sector 10 VOLUME 0000: 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F | ................ 0010: 10 11 12 13 14 15 16 17 18 19 1A 1B 1C 1D 1E 1F | ................ 0020: 20 21 22 23 24 25 26 27 28 29 2A 2B 2C 2D 2E 2F | !"#$%&'()*+,-./ 0030: 30 31 32 33 34 35 36 37 38 39 3A 3B 3C 3D 3E 3F | 0123456789:;<=>? 0040: 40 41 42 43 44 45 46 47 48 49 4A 4B 4C 4D 4E 4F | @ABCDEFGHIJKLMNO > volume write 10 > volume read 10 Volume sector 10 Sector 10 VOLUME 0000: 27 D7 C2 8F 94 1A 37 EA 97 24 03 83 E3 AC F0 12 | '.....7..$...... 0010: 10 11 12 13 14 15 16 17 18 19 1A 1B 1C 1D 1E 1F | ................ 0020: 20 21 22 23 24 25 26 27 28 29 2A 2B 2C 2D 2E 2F | !"#$%&'()*+,-./ 0030: 30 31 32 33 34 35 36 37 38 39 3A 3B 3C 3D 3E 3F | 0123456789:;<=>?

Aha! This tells me that the first 16 bytes of the sector data are being overwritten.

pCipher: ffffdd8f`9db76000 8b 3a c6 ee a9 be 03 06 c1 c9 e4 d4 0f bd f4 8e .:.............. ffffdd8f`9db76010 1f 4b 59 14 44 bc b0 c6 11 1f 41 78 f0 71 2e 50 .KY.D.....Ax.q.P ffffdd8f`9db76020 cf 9f db d1 eb 34 9f 66 7e 15 55 5a 6d 5a c2 eb .....4.f~.UZmZ.. ffffdd8f`9db76030 50 51 7f 88 e5 16 88 77 ae cf 15 16 4e 52 46 d3 PQ.....w....NRF. ffffdd8f`9db76040 1e 10 fd 82 d6 56 c2 1b d1 9d d2 11 12 95 1e 31 .....V.........1 ffffdd8f`9db76050 34 78 26 6c 65 a4 dc 54 d4 7f 59 eb fe c6 fc 49 4x&le..T..Y....I

WriteSectorInfo() bug: NTSTATUS DeviceDisk::WriteSectorInfo(UINT64 nSector, BYTE *pSrc) { NTSTATUS Status= STATUS_SUCCESS; IO_STATUS_BLOCK IoStatus; LARGE_INTEGER FilePos; Zero(IoStatus); FilePos.QuadPart= Volume.MediaOffset + nSector*sizeof(Volume.SectorInfo); if(!NT_SUCCESS(Status= ZwWriteFile(hMediaFile,0,0,0,&IoStatus,pSrc,Volume.SectorInfo,&FilePos,0))) return(Warn(Status,"DeviceDisk:WriteSectorInfo(%llu): ZwWriteFile(%llu) failed.",nSector,FilePos.QuadPart)); return(Status); }

Volume.SectorInfo is the size of the Info, not a struct. So I was miscalculating the file position of the info. ReadSectorInfo() did not contain this bug, so I was effectively reading garbage for the info, causing the decrypt to return garbage. (GIGO!)

This minor bug had me spinning around for over a day, although there other changes that may (or may not) have fixed other problems.

CryptKM: Debug

Debugging CryptKM

Sector Info