Format-preserving encryption – Wikipedia

In cryptography, format-preserving encryption ( FPE ), refers to encrypting in such a way that the output ( the ciphertext ) is in the lapp format as the input ( the plaintext ). The meaning of “ format ” varies. typically only finite sets of characters are used ; numeral, alphabetic or alphanumeric. For exemplar :

  • Encrypting a 16-digit credit card number so that the ciphertext is another 16-digit number.
  • Encrypting an English word so that the ciphertext is another English word.
  • Encrypting an n-bit number so that the ciphertext is another n-bit number (this is the definition of an n-bit block cipher).

For such finite domains, and for the purposes of the discussion below, the zero is equivalent to a permutation of N integers { 0, …, N −1 } where N is the size of the domain .

motivation [edit ]

Restricted field lengths or formats [edit ]

One motivation for using FPE comes from the problems associated with integrating encoding into existing applications, with chiseled data models. A distinctive exercise would be a credit card count, such as 1234567812345670 ( 16 bytes long, digits alone ). Adding encoding to such applications might be challenging if data models are to be changed, as it normally involves changing plain length limits or data types. For model, output from a typical auction block cipher would turn accredit poster number into a hexadecimal ( e.g. 0x96a45cbcf9c2a9425cde9e274948cb67, 34 bytes, hexadecimal digits ) or Base64 value ( e.g. lqRcvPnCqUJc3p4nSUjLZw==, 24 bytes, alphanumeric and special characters ), which will break any existing applications expecting the credit poster act to be a 16-digit number.

apart from simple formatting problems, using AES-128-CBC, this credit poster total might get encrypted to the hexadecimal respect 0xde015724b081ea7003de4593d792fd8b695b39e095c98f3a220ff43522a2df02. In accession to the problems caused by creating invalid characters and increasing the size of the data, data encrypted using the CBC mode of an encoding algorithm besides changes its value when it is decrypted and encrypted again. This happens because the random seed value that is used to initialize the encoding algorithm and is included as separate of the code value is different for each encoding operation. Because of this, it is impossible to use data that has been encrypted with the CBC modality as a unique key to identify a quarrel in a database. FPE attempts to simplify the conversion work by preserving the format and duration of the original data, allowing a drop-in substitution of plaintext values with their ciphertexts in bequest applications .

Comparison to truly random permutations [edit ]

Although a truly random permutation is the ideal FPE cipher, for large domains it is impracticable to pre-generate and remember a in truth random permutation. So the trouble of FPE is to generate a pseudorandom permutation from a secret key, in such a way that the calculation time for a single value is small ( ideally constant, but most importantly smaller than O(N) ) .

Comparison to block ciphers [edit ]

An n-bit block code technically is a FPE on the hardened { 0, …, 2n-1 }. If an FPE is needed on one of these standard sized sets ( for model, newton = 64 for DES and newton = 128 for AES ) a block code of the right size can be used. however, in typical usage, a block cipher is used in a mood of operation that allows it to encrypt randomly long messages, and with an low-level formatting vector as discussed above. In this modality, a block code is not an FPE .

definition of security [edit ]

In cryptanalytic literature ( see most of the references below ), the bill of a “ effective ” FPE is whether an attacker can distinguish the FPE from a rightfully random substitution. diverse types of attackers are postulated, depending on whether they have access to oracles or known ciphertext/plaintext pairs .

algorithm [edit ]

In most of the approaches listed here, a well-understood obstruct cipher ( such as AES ) is used as a archaic to take the place of an ideal random function. This has the advantage that incorporation of a secret winder into the algorithm is easy. Where AES is mentioned in the follow discussion, any other good block nothing would work ampere well .

The FPE constructions of Black and Rogaway [edit ]

Implementing FPE with security demonstrably related to that of the underlying stuff nothing was first undertaken in a paper by cryptographers John Black and Phillip Rogaway, [ 1 ] which described three ways to do this. They proved that each of these techniques is vitamin a impregnable as the forget calculate that is used to construct it. This means that if the AES algorithm is used to create an FPE algorithm, then the resulting FPE algorithm is angstrom dependable as AES because an adversary capable of defeating the FPE algorithm can besides defeat the AES algorithm. therefore, if AES is guarantee, then the FPE algorithm constructed from it are besides secure. In all of the pursuit, E denotes the AES encoding operation that is used to construct an FPE algorithm and F denotes the FPE encoding operation .

FPE from a prefix calculate [edit ]

One simple way to create an FPE algorithm on { 0, …, N-1 } is to assign a pseudorandom weight to each integer, then sort by weight. The weights are defined by applying an existing block cipher to each integer. Black and Rogaway call this proficiency a “ prefix calculate ” and showed it was probably vitamin a thoroughly as the block cipher used. frankincense, to create a FPE on the domain { 0,1,2,3 }, given a key K apply AES ( K ) to each integer, giving, for example ,

weight(0) = 0x56c644080098fc5570f2b329323dbf62
weight(1) = 0x08ee98c0d05e3dad3eb3d6236f23e7b7
weight(2) = 0x47d2e1bf72264fa01fb274465e56ba20
weight(3) = 0x077de40941c93774857961a8a772650d

Sorting [ 0,1,2,3 ] by weight gives [ 3,1,2,0 ], so the calculate is

F(0) = 3
F(1) = 1
F(2) = 2
F(3) = 0

This method acting is alone utilitarian for belittled values of N. For larger values, the size of the search table and the necessitate number of encryptions to initialize the table gets excessively big to be practical .

FPE from cycle walking [edit ]

If there is a adjust M of allowed values within the knowledge domain of a pseudorandom permutation P ( for exemplar P can be a forget cipher like AES ), an FPE algorithm can be created from the jam cipher by repeatedly applying the forget cipher until the result is one of the allowed values ( within M ) .

CycleWalkingFPE(x) {
    if P(x) is an element of M then
        return P(x)
    else
        return CycleWalkingFPE(P(x))
}

The recursion is guaranteed to terminate. ( Because P is one-to-one and the domain is finite, recur application of P forms a cycle, so starting with a point in M the cycle will finally terminate in M. ) This has the advantage that the elements of M do not have to be mapped to a straight sequence { 0, …, N -1 } of integers. It has the disadvantage, when M is a lot smaller than P ‘s knowledge domain, that excessively many iterations might be required for each operation. If P is a block code of a fixed size, such as AES, this is a severe restriction on the sizes of M for which this method is effective. For example, an application may want to encrypt 100-bit values with AES in a way that creates another 100-bit respect. With this technique, AES-128-ECB encoding can be applied until it reaches a value which has all of its 28 highest bits set to 0, which will take an average of 228 iterations to happen .

FPE from a Feistel network

[edit ]

It is besides possible to make a FPE algorithm using a Feistel network. A Feistel net needs a source of pseudo-random values for the sub-keys for each orotund, and the output of the AES algorithm can be used as these pseudo-random values. When this is done, the resulting Feistel construction is estimable if enough rounds are used. [ 2 ] One way to implement an FPE algorithm using AES and a Feistel network is to use as many bits of AES output as are needed to equal the length of the left or right halves of the Feistel network. If a 24-bit rate is needed as a sub-key, for example, it is possible to use the lowest 24 bits of the output signal of AES for this value. This may not result in the output of the Feistel net preserving the format of the input signal, but it is possible to iterate the Feistel network in the same way that the cycle-walking technique does to ensure that format can be preserved. Because it is possible to adjust the size of the inputs to a Feistel network, it is possible to make it very likely that this iteration ends very quickly on average. In the case of credit card numbers, for example, there are 1015 potential 16-digit credit card numbers ( accounting for the pleonastic check digit ), and because the 1015 ≈ 249.8, using a 50-bit wide Feistel net along with motorbike walking will create an FPE algorithm that encrypts reasonably quickly on average .

The Thorp shamble [edit ]

A Thorp shuffle is like an idealized card-shuffle, or equivalently a maximally-unbalanced Feistel cipher where one side is a single moment. It is easier to prove security for unbalanced Feistel ciphers than for balance ones. [ 3 ]

VIL modality [edit ]

For domain sizes that are a power of two, and an existing block cipher with a smaller block size, a newfangled calculate may be created using VIL mode as described by Bellare, Rogaway. [ 4 ]

Hasty Pudding Cipher [edit ]

The Hasty Pudding Cipher uses custom constructions ( not depending on existing auction block ciphers as primitives ) to encrypt arbitrary finite minor domains .

The FFSEM/FFX mood of AES [edit ]

The FFSEM manner of AES ( stipulation [ 5 ] ) that has been accepted for consideration by NIST uses the Feistel network construction of Black and Rogaway described above, with AES for the round off function, with one little modification : a individual keystone is used and is tweaked slenderly for each round. As of February 2010, FFSEM has been superseded by the FFX mode written by Mihir Bellare, Phillip Rogaway, and Terence Spies. ( specification, [ 6 ] [ 7 ] NIST Block Cipher Modes Development, 2010 ) .

FPE for JPEG 2000 encoding [edit ]

In JPEG 2000 standard, the marker codes ( in the range 0xFF90 through 0xFFFF ) should not appear in the plaintext and ciphertext. The elementary modular-0xFF90 proficiency can not be applied to solve the JPEG 2000 encoding trouble. For exercise, the ciphertext words 0x23FF and 0x9832 are valid, but their combination 0x23FF9832 becomes invalid since it introduces the marker code 0xFF98. similarly, the elementary cycle-walking technique can not be applied to solve the JPEG2000 encoding problem since two valid ciphertext blocks may give invalid ciphertext when they get combined. For case, if the first ciphertext block ends with bytes “ … 30FF ” and the second ciphertext freeze starts with bytes “ 9832 … ”, then the marker code “ 0xFF98 ” would appear in the ciphertext. Two mechanisms for format-preserving encoding of JPEG 2000 were given in the newspaper “ effective and secure Encryption Schemes for JPEG2000 ” [ 8 ] by Hongjun Wu and Di Ma. To perform format-preserving encoding of JPEG 2000, the technique is to exclude the byte “ 0xFF ” in the encoding and decoding. then a JPEG 2000 encoding mechanism performs modulo-n summation with stream code ; another JPEG 2000 encoding mechanism performs the cycle-walking technique with pulley code .

other FPE constructions [edit ]

several FPE constructs are based on adding the output of a standard nothing, modulo normality, to the data to be encrypted, with diverse methods of unbiasing the leave. The modulo-n addition shared by many of the constructs is the immediately obvious solution to the FPE problem ( therefore its use in a count of cases ), with the chief differences being the unbiasing mechanisms used. section 8 of the FIPS 74, Federal Information Processing Standards Publication 1981 Guidelines for Implementing and Using the NBS Data Encryption Standard, [ 9 ] describes a way to use the DES encoding algorithm in a manner that preserves the format of the data via modulo-n addition followed by an unbiasing operation. This standard was withdrawn on May 19, 2005, so the technique should be considered disused in terms of being a formal standard. Another early mechanism for format-preserving encoding was Peter Gutmann ‘s “ Encrypting datum with a qualify range of values ” [ 10 ] which again performs modulo-n summation on any zero with some adjustments to make the solution uniform, with the resulting encoding being angstrom potent as the underlying encoding algorithm on which it is based. The wallpaper “ use Datatype-Preserving encoding to Enhance Data Warehouse Security ” [ 11 ] by Michael Brightwell and Harry Smith describes a way to use the DES encoding algorithm in a direction that preserves the format of the plaintext. This proficiency does n’t appear to apply an unbiasing mistreat as do the other modulo-n techniques referenced here. The wallpaper “ Format-Preserving encoding ” [ 12 ] by Mihir Bellare and Thomas Ristenpart describes using “ closely balanced ” Feistel networks to create secure FPE algorithm. The newspaper “ Format Controlling Encryption Using Datatype Preserving Encryption ” [ 13 ] by Ulf Mattsson describes other ways to create FPE algorithm. An example of FPE algorithm is FNR ( Flexible Naor and Reingold ). [ 14 ]

adoption of FPE algorithm by standards authorities [edit ]

NIST Special Publication 800-38G, “ recommendation for Block Cipher Modes of Operation : Methods for Format-Preserving Encryption ” [ 15 ] specifies two methods : FF1 and FF3. Details on the proposals submitted for each can be found at the NIST Block Cipher Modes Development site, [ 16 ] including patent and test vector information. sample values are available for both FF1 and FF3. [ 17 ]

  • FF1 is FFX[Radix] “Format-preserving Feistel-based Encryption Mode” which is also in standards processes under ANSI X9 as X9.119 and X9.124. It was submitted to NIST by Mihir Bellare of University of California, San Diego, Phillip Rogaway of University of California, Davis, and Terence Spies of Voltage Security Inc. Test vectors are supplied and parts of it are patented. (DRAFT SP 800-38G Rev 1) [18] requires the minimum domain size of the data being encrypted to be 1 million (previously 100).
  • FF3 is BPS named after the authors. It was submitted to NIST by Eric Brier, Thomas Peyrin and Jacques Stern of Ingenico, France. Authors declared to NIST that their algorithm is not patented.[19] The CyberRes Voltage product, although claims to own patents also for BPS mode.[20][21] On 12 April 2017, NIST concluded that FF3 is “no longer suitable as a general-purpose FPE method” because researchers found a vulnerability.[22]
  • FF3-1 (DRAFT SP 800-38G Rev 1) [18] replaces FF3 and requires the minimum domain size of the data being encrypted to be 1 million (previously 100).

Another mood was included in the draft NIST guidance but was removed before final publication .

  • FF2 is VAES3 scheme for FFX: An addendum to “The FFX Mode of Operation for Preserving Encryption”: A parameter collection for encipher strings of arbitrary radix with subkey operation to lengthen life of the enciphering key. It was submitted to NIST by Joachim Vance of VeriFone Systems Inc. Test vectors are not supplied separately from FF1 and parts of it are patented. Authors have submitted a modified algorithm as DFF[23] which is under active consideration by NIST.

Korea has besides developed a FPE standard, FEA-1 and FEA-2 .

Implementations [edit ]

Open Source implementations of FF1 and FF3 are publicly available in C language, Go speech, Java, Node.js, Python, C # /.Net and Rust

References [edit ]

Leave a Reply

Your email address will not be published.