Return-Path: X-Original-To: apmail-poi-commits-archive@minotaur.apache.org Delivered-To: apmail-poi-commits-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 4624D5CA1 for ; Tue, 10 May 2011 10:38:48 +0000 (UTC) Received: (qmail 97827 invoked by uid 500); 10 May 2011 10:38:48 -0000 Delivered-To: apmail-poi-commits-archive@poi.apache.org Received: (qmail 97787 invoked by uid 500); 10 May 2011 10:38:48 -0000 Mailing-List: contact commits-help@poi.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@poi.apache.org Delivered-To: mailing list commits@poi.apache.org Received: (qmail 97780 invoked by uid 99); 10 May 2011 10:38:48 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 10 May 2011 10:38:47 +0000 X-ASF-Spam-Status: No, hits=-1998.0 required=5.0 tests=ALL_TRUSTED,FB_GET_MEDS X-Spam-Check-By: apache.org Received: from [140.211.11.4] (HELO eris.apache.org) (140.211.11.4) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 10 May 2011 10:38:40 +0000 Received: by eris.apache.org (Postfix, from userid 65534) id 60F16238890A; Tue, 10 May 2011 10:38:18 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r1101397 - in /poi/trunk: ./ src/documentation/content/xdocs/ src/java/org/apache/poi/poifs/crypt/ src/testcases/org/apache/poi/poifs/crypt/ test-data/poifs/ Date: Tue, 10 May 2011 10:38:18 -0000 To: commits@poi.apache.org From: maxcom@apache.org X-Mailer: svnmailer-1.0.8 Message-Id: <20110510103818.60F16238890A@eris.apache.org> X-Virus-Checked: Checked by ClamAV on apache.org Author: maxcom Date: Tue May 10 10:38:17 2011 New Revision: 1101397 URL: http://svn.apache.org/viewvc?rev=1101397&view=rev Log: bug#51165: Add support for OOXML Agile Encryption Added: poi/trunk/src/documentation/content/xdocs/encryption.xml poi/trunk/src/java/org/apache/poi/poifs/crypt/AgileDecryptor.java poi/trunk/src/java/org/apache/poi/poifs/crypt/EcmaDecryptor.java poi/trunk/src/testcases/org/apache/poi/poifs/crypt/AllPOIFSCryptoTests.java poi/trunk/src/testcases/org/apache/poi/poifs/crypt/TestDecryptor.java poi/trunk/src/testcases/org/apache/poi/poifs/crypt/TestEncryptionInfo.java poi/trunk/test-data/poifs/protected_agile.docx (with props) Removed: poi/trunk/src/testcases/org/apache/poi/poifs/crypt/DecryptorTest.java poi/trunk/src/testcases/org/apache/poi/poifs/crypt/EncryptionInfoTest.java Modified: poi/trunk/build.xml poi/trunk/src/java/org/apache/poi/poifs/crypt/Decryptor.java poi/trunk/src/java/org/apache/poi/poifs/crypt/EncryptionHeader.java poi/trunk/src/java/org/apache/poi/poifs/crypt/EncryptionInfo.java poi/trunk/src/java/org/apache/poi/poifs/crypt/EncryptionVerifier.java Modified: poi/trunk/build.xml URL: http://svn.apache.org/viewvc/poi/trunk/build.xml?rev=1101397&r1=1101396&r2=1101397&view=diff ============================================================================== --- poi/trunk/build.xml (original) +++ poi/trunk/build.xml Tue May 10 10:38:17 2011 @@ -122,6 +122,9 @@ under the License. + + @@ -166,6 +169,7 @@ under the License. + @@ -295,6 +299,7 @@ under the License. + @@ -312,6 +317,10 @@ under the License. + + + + Added: poi/trunk/src/documentation/content/xdocs/encryption.xml URL: http://svn.apache.org/viewvc/poi/trunk/src/documentation/content/xdocs/encryption.xml?rev=1101397&view=auto ============================================================================== --- poi/trunk/src/documentation/content/xdocs/encryption.xml (added) +++ poi/trunk/src/documentation/content/xdocs/encryption.xml Tue May 10 10:38:17 2011 @@ -0,0 +1,84 @@ + + + + + + +
+ Apache POI - Encryption support + + + +
+ + +
Overview +

Apache POI contains support for reading few variants of encrypted office files:

+
    +
  • XLS - RC4 Encryption
    • +
  • XML-based formats (XLSX, DOCX and etc) - AES Encryption
    • +
+ +

Some "write-protected" files are encrypted with build-in password, POI can read that files too.

+
+ +
XLS +

When HSSF receive encrypted file, it tries to decode it with MSOffice build-in password. + Use static method setCurrentUserPassword(String password) of org.apache.poi.hssf.record.crypto.Biff8EncryptionKey to + set password. It sets thread local variable. Do not forget to reset it to null after text extraction. +

+
+ +
XML-based formats +

XML-based formats are stored in OLE-package stream "EncryptedPackage". Use org.apache.poi.poifs.crypt.Decryptor + to decode file:

+ + +EncryptionInfo info = new EncryptionInfo(filesystem); +Decryptor d = new Decryptor(info); + +try { + if (!d.verifyPassword(password)) { + throw new RuntimeException("Unable to process: document is encrypted"); + } + + InputStream dataStream = d.getDataStream(filesystem); + + // parse dataStream + +} catch (GeneralSecurityException ex) { + throw new RuntimeException("Unable to process encrypted document", ex); +} + + +

If you want to read file encrypted with build-in password, use Decryptor.DEFAULT_PASSWORD.

+
+ + +
+ + Copyright (c) @year@ The Apache Software Foundation. All rights reserved. + +
+ + + + + Added: poi/trunk/src/java/org/apache/poi/poifs/crypt/AgileDecryptor.java URL: http://svn.apache.org/viewvc/poi/trunk/src/java/org/apache/poi/poifs/crypt/AgileDecryptor.java?rev=1101397&view=auto ============================================================================== --- poi/trunk/src/java/org/apache/poi/poifs/crypt/AgileDecryptor.java (added) +++ poi/trunk/src/java/org/apache/poi/poifs/crypt/AgileDecryptor.java Tue May 10 10:38:17 2011 @@ -0,0 +1,244 @@ +/* ==================================================================== + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +==================================================================== */ +package org.apache.poi.poifs.crypt; + +import java.util.Arrays; +import java.io.IOException; +import java.io.InputStream; +import java.io.FilterInputStream; +import java.io.ByteArrayInputStream; +import java.security.MessageDigest; +import java.security.GeneralSecurityException; +import java.security.NoSuchAlgorithmException; +import org.apache.poi.poifs.filesystem.POIFSFileSystem; +import org.apache.poi.poifs.filesystem.DirectoryNode; +import org.apache.poi.EncryptedDocumentException; + +import javax.crypto.Cipher; +import javax.crypto.CipherInputStream; +import javax.crypto.SecretKey; +import javax.crypto.spec.SecretKeySpec; +import javax.crypto.spec.IvParameterSpec; + +import org.apache.poi.poifs.filesystem.DirectoryNode; +import org.apache.poi.poifs.filesystem.DocumentInputStream; +import org.apache.poi.poifs.filesystem.POIFSFileSystem; +import org.apache.poi.util.LittleEndian; + +/** + * @author Gary King + */ +public class AgileDecryptor extends Decryptor { + + private final EncryptionInfo _info; + private SecretKey _secretKey; + + private static final byte[] kVerifierInputBlock; + private static final byte[] kHashedVerifierBlock; + private static final byte[] kCryptoKeyBlock; + + static { + kVerifierInputBlock = + new byte[] { (byte)0xfe, (byte)0xa7, (byte)0xd2, (byte)0x76, + (byte)0x3b, (byte)0x4b, (byte)0x9e, (byte)0x79 }; + kHashedVerifierBlock = + new byte[] { (byte)0xd7, (byte)0xaa, (byte)0x0f, (byte)0x6d, + (byte)0x30, (byte)0x61, (byte)0x34, (byte)0x4e }; + kCryptoKeyBlock = + new byte[] { (byte)0x14, (byte)0x6e, (byte)0x0b, (byte)0xe7, + (byte)0xab, (byte)0xac, (byte)0xd0, (byte)0xd6 }; + } + + public boolean verifyPassword(String password) throws GeneralSecurityException { + EncryptionVerifier verifier = _info.getVerifier(); + int algorithm = verifier.getAlgorithm(); + int mode = verifier.getCipherMode(); + + byte[] pwHash = hashPassword(_info, password); + byte[] iv = generateIv(algorithm, verifier.getSalt(), null); + + SecretKey skey; + skey = new SecretKeySpec(generateKey(pwHash, kVerifierInputBlock), "AES"); + Cipher cipher = getCipher(algorithm, mode, skey, iv); + byte[] verifierHashInput = cipher.doFinal(verifier.getVerifier()); + + MessageDigest sha1 = MessageDigest.getInstance("SHA-1"); + byte[] trimmed = new byte[verifier.getSalt().length]; + System.arraycopy(verifierHashInput, 0, trimmed, 0, trimmed.length); + byte[] hashedVerifier = sha1.digest(trimmed); + + skey = new SecretKeySpec(generateKey(pwHash, kHashedVerifierBlock), "AES"); + iv = generateIv(algorithm, verifier.getSalt(), null); + cipher = getCipher(algorithm, mode, skey, iv); + byte[] verifierHash = cipher.doFinal(verifier.getVerifierHash()); + trimmed = new byte[hashedVerifier.length]; + System.arraycopy(verifierHash, 0, trimmed, 0, trimmed.length); + + if (Arrays.equals(trimmed, hashedVerifier)) { + skey = new SecretKeySpec(generateKey(pwHash, kCryptoKeyBlock), "AES"); + iv = generateIv(algorithm, verifier.getSalt(), null); + cipher = getCipher(algorithm, mode, skey, iv); + byte[] inter = cipher.doFinal(verifier.getEncryptedKey()); + byte[] keyspec = new byte[_info.getHeader().getKeySize() / 8]; + System.arraycopy(inter, 0, keyspec, 0, keyspec.length); + _secretKey = new SecretKeySpec(keyspec, "AES"); + return true; + } else { + return false; + } + } + + public InputStream getDataStream(DirectoryNode dir) throws IOException, GeneralSecurityException { + DocumentInputStream dis = dir.createDocumentInputStream("EncryptedPackage"); + long size = dis.readLong(); + return new ChunkedCipherInputStream(dis, size); + } + + protected AgileDecryptor(EncryptionInfo info) { + _info = info; + } + + private class ChunkedCipherInputStream extends InputStream { + private int _lastIndex = 0; + private long _pos = 0; + private final long _size; + private final DocumentInputStream _stream; + private byte[] _chunk; + private Cipher _cipher; + + public ChunkedCipherInputStream(DocumentInputStream stream, long size) + throws GeneralSecurityException { + _size = size; + _stream = stream; + _cipher = getCipher(_info.getHeader().getAlgorithm(), + _info.getHeader().getCipherMode(), + _secretKey, _info.getHeader().getKeySalt()); + } + + public int read() throws IOException { + byte[] b = new byte[1]; + if (read(b) == 1) + return b[0]; + return -1; + } + + public int read(byte[] b) throws IOException { + return read(b, 0, b.length); + } + + public int read(byte[] b, int off, int len) throws IOException { + int total = 0; + + while (len > 0) { + if (_chunk == null) { + try { + _chunk = nextChunk(); + } catch (GeneralSecurityException e) { + throw new EncryptedDocumentException(e.getMessage()); + } + } + int count = (int)(4096L - (_pos & 0xfff)); + count = Math.min(available(), Math.min(count, len)); + System.arraycopy(_chunk, (int)(_pos & 0xfff), b, off, count); + off += count; + len -= count; + _pos += count; + if ((_pos & 0xfff) == 0) + _chunk = null; + total += count; + } + + return total; + } + + public long skip(long n) throws IOException { + long start = _pos; + long skip = Math.min(available(), n); + + if ((((_pos + skip) ^ start) & ~0xfff) != 0) + _chunk = null; + _pos += skip; + return skip; + } + + public int available() throws IOException { return (int)(_size - _pos); } + public void close() throws IOException { _stream.close(); } + public boolean markSupported() { return false; } + + private byte[] nextChunk() throws GeneralSecurityException, IOException { + int index = (int)(_pos >> 12); + byte[] blockKey = new byte[4]; + LittleEndian.putInt(blockKey, index); + byte[] iv = generateIv(_info.getHeader().getAlgorithm(), + _info.getHeader().getKeySalt(), blockKey); + _cipher.init(Cipher.DECRYPT_MODE, _secretKey, new IvParameterSpec(iv)); + if (_lastIndex != index) + _stream.skip((index - _lastIndex) << 12); + + byte[] block = new byte[Math.min(_stream.available(), 4096)]; + _stream.readFully(block); + _lastIndex = index + 1; + return _cipher.doFinal(block); + } + } + + private Cipher getCipher(int algorithm, int mode, SecretKey key, byte[] vec) + throws GeneralSecurityException { + String name = null; + String chain = null; + + if (algorithm == EncryptionHeader.ALGORITHM_AES_128 || + algorithm == EncryptionHeader.ALGORITHM_AES_192 || + algorithm == EncryptionHeader.ALGORITHM_AES_256) + name = "AES"; + + if (mode == EncryptionHeader.MODE_CBC) + chain = "CBC"; + else if (mode == EncryptionHeader.MODE_CFB) + chain = "CFB"; + + Cipher cipher = Cipher.getInstance(name + "/" + chain + "/NoPadding"); + IvParameterSpec iv = new IvParameterSpec(vec); + cipher.init(Cipher.DECRYPT_MODE, key, iv); + return cipher; + } + + private byte[] getBlock(int algorithm, byte[] hash) { + byte[] result = new byte[getBlockSize(algorithm)]; + Arrays.fill(result, (byte)0x36); + System.arraycopy(hash, 0, result, 0, Math.min(result.length, hash.length)); + return result; + } + + private byte[] generateKey(byte[] hash, byte[] blockKey) throws NoSuchAlgorithmException { + MessageDigest sha1 = MessageDigest.getInstance("SHA-1"); + sha1.update(hash); + return getBlock(_info.getVerifier().getAlgorithm(), sha1.digest(blockKey)); + } + + protected byte[] generateIv(int algorithm, byte[] salt, byte[] blockKey) + throws NoSuchAlgorithmException { + + + if (blockKey == null) + return getBlock(algorithm, salt); + + MessageDigest sha1 = MessageDigest.getInstance("SHA-1"); + sha1.update(salt); + return getBlock(algorithm, sha1.digest(blockKey)); + } +} \ No newline at end of file Modified: poi/trunk/src/java/org/apache/poi/poifs/crypt/Decryptor.java URL: http://svn.apache.org/viewvc/poi/trunk/src/java/org/apache/poi/poifs/crypt/Decryptor.java?rev=1101397&r1=1101396&r2=1101397&view=diff ============================================================================== --- poi/trunk/src/java/org/apache/poi/poifs/crypt/Decryptor.java (original) +++ poi/trunk/src/java/org/apache/poi/poifs/crypt/Decryptor.java Tue May 10 10:38:17 2011 @@ -19,150 +19,74 @@ package org.apache.poi.poifs.crypt; import java.io.IOException; import java.io.InputStream; import java.io.UnsupportedEncodingException; -import java.security.GeneralSecurityException; import java.security.MessageDigest; +import java.security.GeneralSecurityException; import java.security.NoSuchAlgorithmException; -import java.util.Arrays; - -import javax.crypto.Cipher; -import javax.crypto.CipherInputStream; -import javax.crypto.SecretKey; -import javax.crypto.spec.SecretKeySpec; - -import org.apache.poi.poifs.filesystem.DirectoryNode; -import org.apache.poi.poifs.filesystem.DocumentInputStream; import org.apache.poi.poifs.filesystem.NPOIFSFileSystem; import org.apache.poi.poifs.filesystem.POIFSFileSystem; +import org.apache.poi.poifs.filesystem.DirectoryNode; +import org.apache.poi.EncryptedDocumentException; import org.apache.poi.util.LittleEndian; -/** - * @author Maxim Valyanskiy - */ -public class Decryptor { +public abstract class Decryptor { public static final String DEFAULT_PASSWORD="VelvetSweatshop"; - private final EncryptionInfo info; - private byte[] passwordHash; + public abstract InputStream getDataStream(DirectoryNode dir) + throws IOException, GeneralSecurityException; + + public abstract boolean verifyPassword(String password) + throws GeneralSecurityException; + + public static Decryptor getInstance(EncryptionInfo info) { + int major = info.getVersionMajor(); + int minor = info.getVersionMinor(); + + if (major == 4 && minor == 4) + return new AgileDecryptor(info); + else if (minor == 2 && (major == 3 || major == 4)) + return new EcmaDecryptor(info); + else + throw new EncryptedDocumentException("Unsupported version"); + } + + public InputStream getDataStream(NPOIFSFileSystem fs) throws IOException, GeneralSecurityException { + return getDataStream(fs.getRoot()); + } + + public InputStream getDataStream(POIFSFileSystem fs) throws IOException, GeneralSecurityException { + return getDataStream(fs.getRoot()); + } - public Decryptor(EncryptionInfo info) { - this.info = info; + protected static int getBlockSize(int algorithm) { + switch (algorithm) { + case EncryptionHeader.ALGORITHM_AES_128: return 16; + case EncryptionHeader.ALGORITHM_AES_192: return 24; + case EncryptionHeader.ALGORITHM_AES_256: return 32; + } + throw new EncryptedDocumentException("Unknown block size"); } - private void generatePasswordHash(String password) throws NoSuchAlgorithmException { + protected byte[] hashPassword(EncryptionInfo info, + String password) throws NoSuchAlgorithmException { MessageDigest sha1 = MessageDigest.getInstance("SHA-1"); - - byte[] passwordBytes; + byte[] bytes; try { - passwordBytes = password.getBytes("UTF-16LE"); - } catch(UnsupportedEncodingException e) { - throw new RuntimeException("Your JVM is broken - UTF16 not found!"); + bytes = password.getBytes("UTF-16LE"); + } catch (UnsupportedEncodingException e) { + throw new EncryptedDocumentException("UTF16 not supported"); } sha1.update(info.getVerifier().getSalt()); - byte[] hash = sha1.digest(passwordBytes); - + byte[] hash = sha1.digest(bytes); byte[] iterator = new byte[4]; - for (int i = 0; i<50000; i++) { - sha1.reset(); + for (int i = 0; i < info.getVerifier().getSpinCount(); i++) { + sha1.reset(); LittleEndian.putInt(iterator, i); sha1.update(iterator); hash = sha1.digest(hash); } - passwordHash = hash; - } - - private byte[] generateKey(int block) throws NoSuchAlgorithmException { - MessageDigest sha1 = MessageDigest.getInstance("SHA-1"); - - sha1.update(passwordHash); - byte[] blockValue = new byte[4]; - LittleEndian.putInt(blockValue, block); - byte[] finalHash = sha1.digest(blockValue); - - int requiredKeyLength = info.getHeader().getKeySize()/8; - - byte[] buff = new byte[64]; - - Arrays.fill(buff, (byte) 0x36); - - for (int i=0; i source.length) { - for(int i=source.length; i source.length) { + for(int i=source.length; i0) { + zin.skip(zin.available()); + } + } + } +} Added: poi/trunk/src/testcases/org/apache/poi/poifs/crypt/TestEncryptionInfo.java URL: http://svn.apache.org/viewvc/poi/trunk/src/testcases/org/apache/poi/poifs/crypt/TestEncryptionInfo.java?rev=1101397&view=auto ============================================================================== --- poi/trunk/src/testcases/org/apache/poi/poifs/crypt/TestEncryptionInfo.java (added) +++ poi/trunk/src/testcases/org/apache/poi/poifs/crypt/TestEncryptionInfo.java Tue May 10 10:38:17 2011 @@ -0,0 +1,45 @@ +/* ==================================================================== + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +==================================================================== */ +package org.apache.poi.poifs.crypt; + +import junit.framework.TestCase; +import org.apache.poi.POIDataSamples; +import org.apache.poi.poifs.filesystem.POIFSFileSystem; + +import java.io.IOException; + +/** + * @author Maxim Valyanskiy + */ +public class TestEncryptionInfo extends TestCase { + public void testEncryptionInfo() throws IOException { + POIFSFileSystem fs = new POIFSFileSystem(POIDataSamples.getPOIFSInstance().openResourceAsStream("protect.xlsx")); + + EncryptionInfo info = new EncryptionInfo(fs); + + assertEquals(3, info.getVersionMajor()); + assertEquals(2, info.getVersionMinor()); + + assertEquals(EncryptionHeader.ALGORITHM_AES_128, info.getHeader().getAlgorithm()); + assertEquals(EncryptionHeader.HASH_SHA1, info.getHeader().getHashAlgorithm()); + assertEquals(128, info.getHeader().getKeySize()); + assertEquals(EncryptionHeader.PROVIDER_AES, info.getHeader().getProviderType()); + assertEquals("Microsoft Enhanced RSA and AES Cryptographic Provider", info.getHeader().getCspName()); + + assertEquals(32, info.getVerifier().getVerifierHash().length); + } +} Added: poi/trunk/test-data/poifs/protected_agile.docx URL: http://svn.apache.org/viewvc/poi/trunk/test-data/poifs/protected_agile.docx?rev=1101397&view=auto ============================================================================== Binary file - no diff available. Propchange: poi/trunk/test-data/poifs/protected_agile.docx ------------------------------------------------------------------------------ svn:mime-type = application/octet-stream --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscribe@poi.apache.org For additional commands, e-mail: commits-help@poi.apache.org