Flat ORAM: A Simplified Write-Only Oblivious RAM Construction for Secure Processors

Oblivious RAM (ORAM) is a cryptographic primitive which obfuscates the access patterns to a storage thereby preventing privacy leakage. So far in the current literature, only `fully functional' ORAMs are widely studied which can protect, at a cost of considerable performance penalty, against the strong adversaries who can monitor all read and write operations. However, recent research has shown that information can still be leaked even if only the write access pattern (not reads) is visible to the adversary. For such weaker adversaries, a fully functional ORAM turns out to be an overkill causing unnecessary overheads. Instead, a simple `write-only' ORAM is sufficient, and, more interestingly, is preferred as it can offer far more performance and energy efficiency than a fully functional ORAM. In this work, we present Flat ORAM: an efficient write-only ORAM scheme which outperforms the closest existing write-only ORAM called HIVE. HIVE suffers from performance bottlenecks while managing the memory occupancy information vital for correctness of the protocol. Flat ORAM resolves these bottlenecks by introducing a simple idea of Occupancy Map (OccMap) which efficiently manages the memory occupancy information resulting in far better performance. Our simulation results show that, on average, Flat ORAM only incurs a moderate slowdown of $3\times$ over the insecure DRAM for memory intensive benchmarks among Splash2 and $1.6\times$ for SPEC06. Compared to HIVE, Flat ORAM offers $50\%$ performance gain on average and up to $80\%$ energy savings.


INTRODUCTION
User's data privacy concerns in computation outsourcing to cloud servers have gained serious a ention over the past couple of decades. To address this challenge, various trusted-hardware based secure processor architectures have been proposed including TPM [1,34,42], Intel's TPM+TXT [16], eXecute Only Memory (XOM) [21][22][23], Aegis [39,40], Ascend [11], Phantom [25] and Intel's SGX [26]. A trusted-hardware platform receives user's encrypted data, which is decrypted and computed upon inside the trusted boundary, and nally the encrypted results of the computation are sent to the user.
Although all the data stored outside the secure processor's trusted boundary, e.g. in the main memory, can be encrypted, information can still be leaked to an adversary through the access pa erns to the stored data [50]. In order to prevent such leakage, Oblivious RAM (ORAM) is a well established technique, rst proposed by Goldreich and Ostrovsky [12]. ORAM obfuscates the memory access pa ern by introducing several randomized 'redundant' accesses, thereby preventing privacy leakage at a cost of signi cant performance penalty. Intense research over the last few decades has resulted in more and more e cient and secure ORAM schemes [6, 8, 9, 13-15, 29, 30, 35-37, 43, 48, 49].
A key observation regarding the adversarial model assumed by the current renowned ORAM techniques is that the adversary is capable of learning ne-grained information of all the accesses made to the memory. is includes the information about which location is accessed, the type of operations (read/write), and the time of the access. It is an extremely strong adversarial model which, in most cases, requires direct physical access to the memory address bus in order to monitor both read and write accesses, e.g. the case of a curious cloud server.
On the other hand, for purely remote adversaries (where the cloud server itself is trusted), direct physical access to the memory address bus is not possible thereby preventing them from directly monitoring read/write access pa erns. Such remote adversaries, although weaker than adversaries having physical access, can still "learn" the application's write access pa erns. Interestingly, privacy leakage is still possible even if the adversary is able to infer just the write access pa erns of an application.
John et al. demonstrated such an a ack [27] on the famous Montgomery's ladder technique [19] commonly used for modular exponentiation in public key cryptography. In this a ack, a 512-bit secret key is correctly inferred in just 3.5 minutes by only monitoring the application's write access pa ern via a compromised Direct Memory Access (DMA 1 ) [3][4][5]24, 31] device on the system. e adversary collects the snapshots of the application's memory via the compromised DMA. Clearly, any two memory snapshots only di er in the locations where the data has been modi ed in the la er snapshot. In other words, comparing the snapshots not only reveals the fact that write accesses (if any) have been made to the memory, but it also reveals the exact locations of the accesses which leads to a precise access pa ern of memory writes resulting in privacy leakage. 1 DMA is a standard performance feature which grants full access of the main memory to certain peripheral buses, e.g. FireWire, underbolt etc.

arXiv:1611.01571v4 [cs.AR] 10 Sep 2017
Recent work [38] demonstrated that DMA a acks can also be launched remotely by injecting malware to the dedicated hardware devices, such as graphic processors and network interface cards, a ached to the host platform. is allows even a remote adversary to learn the application's write access pa ern. Intel's TXT has also been a victim of DMA based a acks where a malicious OS directed a network card to access data in the protected VM [44,45].
One approach to prevent such a acks, as adapted by TXT, could be to block certain DMA accesses through modi cations in DRAM controller. However, this requires the DRAM controller to be included in the trusted computing base (TCB) of the system which is undesirable. Nevertheless, there could potentially be many scenarios other than DMA based a acks where write access pa erns can be learned by the adversary.
Current so-called fully functional ORAM schemes, which obfuscate both read and write access pa erns, also o er a solution to such weaker adversaries. However, the added protection (obfuscation of reads) o ered by fully functional ORAMs is unnecessary and is practically an overkill in this scenario which results in signi cant performance penalty. Path ORAM [37], the most e cient and practical fully functional ORAM system for secure processors so far, still incurs about 2 − 10× performance degradation [11,32] over an insecure DRAM system. A far more e cient solution to this problem is a write-only ORAM scheme, which only obfuscates the write accesses made to the memory. Since read accesses leave no trace in the memory snapshot and hence do not need to be obfuscated in this model, a write-only ORAM can o er signi cant performance advantage over a fully functional ORAM.
A recent work, HIVE [2], has proposed a write-only ORAM scheme for implementing hidden volumes in hard disk drives. e key idea is similar to Path ORAM, i.e., each data block is wri en to a new random location, along with some dummy blocks, every time it is accessed. However, a fundamental challenge that arises in this scheme is to avoid collisions. I.e., to determine whether a randomly chosen physical location contains real or dummy data, in order to avoid overwriting the existing useful data block. HIVE proposes a complex inverse position map approach for collision avoidance. Essentially, it maintains a forward position mapping (logical to physical blocks) and a backward/inverse position mapping (physical to logical blocks) for the whole memory. Before each write, both forward and backward mappings are looked up to determine whether the corresponding physical location is vacant or occupied.
is approach, however, turns out to be a storage and performance bo leneck because of the size of inverse position map and dual lookups of the position mappings.
A simplistic and obvious solution to this long-standing problem, from a computer architect's perspective, would be to use a bit-mask to mark each physical block as vacant or occupied. Based on this intuition, we propose a simpli ed write-only ORAM scheme called Flat ORAM 2 . At the core of the algorithm, Flat ORAM introduces a new data structure called Occupancy Map (OccMap): a bit-mask which o ers an e cient collision avoidance mechanism.
e OccMap records the availability information (occupied/vacant) 2 Flat ORAM replaces the binary tree structure of Path ORAM with a at array of data blocks; hence termed as 'Flat' ORAM.
of every physical location in a highly compressed form (i.e., just 1bit per cache line). For typical parameter se ings, OccMap is about 25× compact compared to HIVE's inverse position map structure. is dense structure allows the OccMap blocks to exploit higher locality, resulting in considerable performance gain. While naively storing the OccMap breaks the ORAM's security, we present how to securely store and manage this structure in a real system. In particular, the paper makes the following contributions: (1) We are the rst ones to implement an existing write-only ORAM scheme, HIVE, in the context of secure processors with all state-of-the-art ORAM optimizations, and analyze its performance bo lenecks. (2) A simple write-only ORAM, named Flat ORAM, having an e cient collision avoidance approach is proposed. e micro-architecture of the scheme is discussed in detail and the design space is comprehensively explored. It has also been shown to seamlessly adopt various performance optimizations of its predecessor: Path ORAM. e rest of the paper is organized as follows: Section 2 describes our adversarial model in detail along with a practical example from the current literature. Section 3 provides the necessary background of fully functional ORAMs and write-only ORAMs. e proposed Flat ORAM scheme along with its security analysis is presented in Section 4, and the detailed construction of its occupancy map structure is shown in Section 5. A few additional optimizations from literature implemented in Flat ORAM are discussed in Section 6. Section 7 evaluates Flat ORAM's performance, and we conclude the paper in Section 8.

ADVERSARIAL MODEL
We assume a relaxed form of the adversarial model considered in several prior works related to fully functional oblivious RAMs in secure processor se ings [9,32,48].
In our model, a user's computation job is outsourced to a cloud, where a trusted processor performs the computation on user's private data. e user's private data is stored (in encrypted form) in the untrusted memory external to the trusted processor, i.e. DRAM. In order to compute on user's private data, the trusted processor interacts with DRAM. e cloud service itself is not considered as an adversary, i.e. it does not try to infer any information from the memory access pa erns of the user's program. However, since the cloud serves several users at the same time, sharing of critical resources such as DRAM among various applications from di erent users is inevitable. Among these users being served by the cloud service, we assume a malicious user who is able to monitor remotely (and potentially learn the secret information from) the data write sequences of other users' applications to the DRAM, e.g., by taking frequent snapshots of the victim application's memory via a compromised DMA. Moreover, he may also tamper with the DRAM contents or play replay a acks in order to manipulate other users' applications and/or learn more about their secret data.

, ,
To protect the system from such an adversary, we add to the processor chip a Write-Only ORAM controller: an additional trusted hardware module. Now all the o -chip tra c goes to DRAM through the ORAM controller. In order to formally de ne the security properties satis ed by our ORAM controller, we adapt the write-only ORAM privacy de nition from [17] as follows: In other words, memory snapshots only reveal to the adversary the timing of write accesses made to the memory (i.e. leakage over ORAM Timing Channel) instead of their precise access pattern, whereas no trace of any read accesses made to the memory is revealed to the adversary. An important aspect of De nition 2.1 to note is that it completely isolates the problem of leakage over ORAM Termination Channel from ORAM's originally targeted problem (which is also targeted in this paper) i.e., preventing leakage over memory address channel. Notice that the original de nition of ORAM [12] does not protect against timing a acks, i.e. it does not obfuscate when an access is made to the memory (ORAM Timing Channel) or how long it takes for the application to nish (ORAM Termination Channel). e write-only ORAM security de nition followed by HIVE [2] also allows leakage over ORAM termination channel, as two memory access sequences generated by the ORAM for two same-length logical access sequences can have different lengths [17]. erefore, in order to de ne precise security guarantees o ered by our ORAM, we follow De nition 2.1.
Periodic ORAMs [11] deterministically make ORAM accesses always at regular prede ned (publicly known) intervals, thereby preventing leakage over ORAM timing channel and shi ing it to the ORAM termination channel. We present a periodic variant of our write-only ORAM to protect leakage over ORAM timing channel. Following the prior works [25] [9], (a) we assume that the timing of individual DRAM accesses made during an ORAM access does not reveal any information; (b) we do not protect the leakage over ORAM termination channel (i.e. total number of ORAM or DRAM accesses). e problem of leakage over ORAM termination channel has been addressed in the existing literature [10] where only log 2 (n) bits are leaked for a total of n accesses. A similar approach can easily be applied to the current scheme.
In order to detect malicious tampering of the memory by the adversary, we follow the standard de nition of data integrity and freshness [9]: De nition 2.2. (Write-Only ORAM Integrity) From the processor's perspective, the ORAM behaves like a valid memory with overwhelming probability, and detects any violation to data authenticity and/or freshness.

Practicality of the Adversarial Model
Modular exponentiation algorithms, such as RSA algorithm, are widely used in public-key cryptography. In general, these algorithms perform computations of the form = k mod n, where the a acker's goal is to nd the secret k. Algorithm 1 shows the Montgomery Ladder scheme [19] which performs exponentiation ( k ) through simple square-and-multiply operations. For a given input and a secret key k, the algorithm performs multiplication and squaring operations on two local variables R 0 and R 1 for each bit of k starting from the most signi cant bit down to the least signi cant bit. is algorithm prevents leakage over power side-channel since, regardless of the value of bit k j , the same number of operations are performed in the same order, hence producing the same power footprint for k j = 0 and k j = 1.
Notice, however, that the speci c order in which R 0 and R 1 are updated in time depends upon the value of k j . E.g., for k j = 0, R 1 is wri en rst and then R 0 is updated; whereas for k j = 1 the updates are done in the reverse order. is sequence of write access to R 0 and R 1 reveals to the adversary the exact bit values of the secret key k. A recent work [27] demonstrated such an a ack where frequent memory snapshots of victim application's data (particularly R 0 and R 1 ) from the physical memory are taken via a compromised DMA. ese snapshots are then correlated in time to determine the sequence of write access to R 0 , R 1 , which in turn reveals the secret key. e reported time taken by the a ack is 3.5 minutes.
One might argue that under write-back caches, the updates to application's data will only be visible in DRAM once the data is evicted from the LLC. is will de nitely introduce noise to the precise write-access sequence discussed earlier, hence making the a acker's job di cult. However, he can still collect several 'noisy' sequences of memory snapshots and then run correlation analysis on them to nd the secret key k. Furthermore, if the adversary is also a user of the same computer system, he can ush the system caches frequently to reduce the noise in write-access sequence even further.

BACKGROUND OF OBLIVIOUS RAMS
A fully functional Oblivious RAM [12], or more commonly known as ORAM, is a primitive that obfuscates the user's (i.e. Processor's) access pa erns to a storage (i.e. DRAM) such that by monitoring the memory access pa erns, an adversary is not able to learn anything about the data being accessed. e ORAM interface transforms the user's access sequence of program addresses into a sequence of ORAM accesses to random looking physical addresses. Since the physical locations being accessed are revealed to the adversary, the ORAM interface guarantees that the physical access pa ern is independent of the logical access pa ern hence user's potentially data dependent access pa erns are not revealed. Furthermore, the data stored in ORAMs should be encrypted using probabilistic encryption to conceal the data content as well as the fact whether or not the content has been updated.

Path ORAM
Path ORAM [37] is currently the most e cient and well studied ORAM implementation for secure processors. It has two main hardware components: the binary tree storage and the ORAM controller. Binary tree stores the data content of the ORAM and is implemented on DRAM. Each node in the tree can hold up to Z useful data blocks, and any empty slots are lled with dummy blocks. All blocks, real or dummy, are probabilistically encrypted and cannot be distinguished. e path from the root node to the leaf s is dened as path s. ORAM controller is a piece of trusted hardware that controls the tree structure. Besides necessary logic circuits, the ORAM controller contains two main structures, a position map and a stash. e position map is a lookup table that associates the program address a of a data block with a path in the ORAM tree (path s). e stash is a bu er that stores up to a small number of data blocks at a time.
Each data block a in Path ORAM is mapped (randomly) to some path s via the position map, i.e. at any time, the data block a must be stored either on path s, or in the stash. Path ORAM follows the following steps when a request on block a is issued by the processor: (1) e path (leaf) number s of the logical address a is looked up in the position map. (2) All the blocks on path s are read and decrypted, and all real blocks added to the stash. (3) Block a is returned to the processor. (4) e position map of a is updated to a new random leaf s . (5) As many blocks from stash as possible are encrypted and wri en on path s, where empty spots are lled with dummy blocks.
Step (4) guarantees that when block a is accessed later, a random path will be accessed which is independent of any previously accessed paths (unlinkability). As a result, each ORAM access is random and unlinkable regardless of the request pa ern.
Path ORAM incurs signi cant energy and performance penalties compared to insecure DRAM. Under typical se ings for secure processors (gigabytes of memory and 64-to 128-byte blocks), Path ORAM has a 20-30 level binary tree where each node typically stores 3 or 4 data blocks [32,36].
is means that each ORAM access reads and writes 60-120 blocks, in contrast to a single read or write operation in an insecure storage system.

Write-Only ORAMs
In contrast to fully functional ORAMs, a write-only ORAM only obfuscates the pa erns of write accesses made to a storage. Write-only ORAM is preferred for performance reasons over fully functional ORAMs in situations where the adversary is not able to monitor the read access pa erns.
ere has been very limited research work done so far to explore write-only ORAMs, and to best of our knowledge, write-only ORAMs for secure processors have not been explored at all. Li and Da a [20] present a write-only ORAM scheme to be used with Private Information Retrieval (PIR) in order to preserve the privacy of data outsourced to a data center. Although this scheme achieves an amortized write cost of O(B log N ), it incurs a read cost of O(B.N ) for a storage of N blocks each of size B. For e cient reads, it requires the client side storage (i.e. the on-chip position map) to be polynomial in N . In a secure processor se ing, DRAM reads are usually the major performance bo leneck, and introducing a complexity polynomial in N on this critical path is highly unwanted.
A recent work, HIVE [2], has proposed a write-only ORAM scheme for hidden volumes in hard disk drives. Although HIVE write-only ORAM presented-as-is [2] targets a totally di erent application, we believe that its parameter se ings can be tweaked to be used in the secure processor se ing. is paper is the rst one to implement HIVE in the secure processor context, and we consider this implementation as the baseline write-only ORAM to be compared with our proposed Flat ORAM.
Roche et al. [33] have also proposed an e cient write-only ORAM scheme for hard disk storages.
is scheme along with its complex optimizations has been implemented in so ware which is completely feasible at the DRAM-Disk boundary. However, in this work, we target the Processor-DRAM boundary for our proposed ORAM which needs to be implemented in hardware, and hence our focus is only towards simpli ed algorithms and optimizations which can easily be synthesized in hardware without substantial area overhead.

FLAT ORAM SCHEME
In this section, we rst present the core algorithm of Flat ORAM, then we discuss its architectural details and various optimizations for a practical implementation.

Fundamental Data Structures
Position Map (PosMap): It is a standard Path ORAM structure that maintains randomized mappings of logical blocks to physical locations. However there is one subtle di erence between PosMap of Path ORAM and Flat ORAM. In Path ORAM, PosMap stores a path number for each logical block and the block can reside anywhere on that path. In contrast, PosMap in Flat ORAM stores the exact physical address where a logical block is stored.
Occupancy Map (OccMap): OccMap is a newly introduced structure in Flat ORAM. It is essentially a large bit-mask where each bit corresponds to a physical location (i.e., a cache line). e binary value of each bit represents whether the corresponding physical block contains real or outdated/dummy data. OccMap is of crucial importance to avoid data collisions, and hence for the correctness of Flat ORAM. A collision happens when a physical location, which is randomly chosen to store a logical block, already contains useful data which cannot be overwri en. Managing the OccMap securely and e ciently is a major challenge which we address in section 5 in detail.
Stash: Stash, also adapted from Path ORAM, is a small bu er in the trusted memory to temporarily hold data blocks evicted from the processor's last level cache (LLC). A slight but crucial for j ∈ {1, · · · , N } do 5:

Basic Algorithm
Let N be the total number of logical data blocks that we want to securely store in our ORAM, which is implemented on top of a DRAM; and let each data block be of size B bytes. Let P be the number of physical blocks that our DRAM can physically store, i.e. the DRAM capacity (where P ≥ N ).
Initial Setup: Algorithm 2 shows the setup phase of our scheme. Two null-initialized arrays PosMap and OccMap, corresponding to position and occupancy map of size N and P entries respectively, are allocated. For now, we assume that both PosMap and OccMap reside on-chip in the trusted memory to which the adversary has no access. However, since these arrays can be quite large and the trusted memory is quite constrained, we later on show how this problem is solved. Initially, since all physical blocks are empty, each OccMap entry is set to 0. Now, each logical block is mapped to a uniformly random physical block, i.e. PosMap is initialized, while avoiding any collisions using OccMap.
e OccMap is updated along the way in order to mark those physical locations which have been assigned to a logical block as 'occupied'. Notice that in order to minimize the probability of collision, P should be su ciently larger than N , e.g. P ≈ 2N gives a 50% collision probability.
Reads: e procedures to read/write a data block corresponding to the virtual address a from/to the ORAM are shown in Algorithm 3. A read operation is straightforward as it does not need to be obfuscated. e PosMap entry for the logical block a is looked up, and the encrypted data is read through normal DRAM read. e data is decrypted and returned to the LLC along with its current physical position s. e location s is stored in the tag array of the LLC, and proves to be useful upon eviction of the data from the LLC.
Writes: Since write operations should be non-blocking in a secure processor se ing, the ORAM writes are performed in two 3 Blocks with modi ed data. return (s, data) Position is also returned. 5 steps. Whenever a data block is evicted from the LLC, it is rst simply added to the stash queue, without incurring any latency. While the processor moves on to computing on other data in its registers/caches, the ORAM controller then works in the background to evict the block from stash to the DRAM. A block a to be wri en is picked from the stash, and a new uniformly random physical position s new is chosen for this block. e OccMap is looked up to determine whether the location s new is vacant. If so, the write operation proceeds by simply recording the new position s new for block a in PosMap, updating the OccMap entries for s new and s old accordingly, and nally writing encrypted data at location s new . Otherwise if the location s new is already occupied by some useful data block, the probability of which is N /P, the existing data block is read, decrypted, re-encrypted 4 under probabilistic encryption and wri en back. A new random position is then chosen for the block a and the above mentioned process is repeated until a vacant location is found to evict the block. Notice that storing s old along with the data upon reads will save extra ORAM accesses to lookup s old from the recursive PosMap (cf. Section 4.5).

Avoiding Redundant Memory Accesses
e fact that the adversary cannot see read accesses allows Flat ORAM to avoid almost all the redundancy incurred by a fully functional ORAM (e.g. Path ORAM). Instead of reading/writing a whole path for each read/write access as done in Path ORAM, Flat ORAM simply reads/writes only the desired block directly given its physical location from the PosMap. is is fundamentally where the write-only ORAMs (i.e. HIVE [2], Flat ORAM) get the performance edge over the fully functional ORAMs. However, the question arises whether Flat ORAM is still secure a er eliminating the redundant accesses.

Security
Privacy: Consider any two logical write-access sequences O 0 and O 1 of the same length. In E S procedure (cf. Algorithm 3), a physical block chosen uniformly at random out of P blocks is always modi ed regardless of it being vacant or occupied. erefore, the write accesses generated by Flat ORAM while executing either of the two logical access sequences O 0 and O 1 will update memory locations uniformly at random throughout the whole physical memory space. As a result, an adversary monitoring these updates cannot distinguish between real vs. dummy blocks, and in turn the two sequences O 0 and O 1 seem computationally indistinguishable.
Furthermore, notice that in Path ORAM the purpose of accessing a whole path instead of just one block upon each read and write access is to prevent linkability between a write and a following read access to the same logical block. In Flat ORAM's model, however, since the adversary cannot see the read accesses at all, therefore the linkability problem would never arise as long as each logical data block is wri en to a new random location every time it is evicted (which is guaranteed by Flat ORAM algorithm). Although HIVE proposes a constant k-factor redundancy upon each data write, we argue that it is unnecessary for the desired security as explained above, and can be avoided to gain performance.
Hence, the basic algorithm of Flat ORAM presented above guarantees the desired privacy property of our write-only ORAM (cf. De nition 2.1).
Integrity: Next, we move on to making the basic Flat ORAM practical for a real system. e main challenge is to get rid of the huge on-chip memory requirements imposed by PosMap and OccMap. While addressing the PosMap management problem, we discuss an existing e cient memory integrity veri cation technique from Path ORAM domain called PMMAC [9] (cf. Section 6.2) which satis es our integrity de nition (cf. De nition 2.2).
Stash Management: Another critical missing piece is to prevent the unlikely event of stash over ow for a small constant sized stash, as such an event could break the privacy guarantees o ered by Flat ORAM. We completely eliminate the possibility of a stash over ow event by using a proven technique called Background Eviction [32]. We present a detailed discussion about the stash size under Background Eviction technique in Section 5.4.

Recursive Position Map & PLB
In order to squeeze the on-chip PosMap size, a standard recursive construction of position map [35] is used. In a 2-level recursive position map, for instance, the original PosMap structure is stored in another set of data blocks which we call a hierarchy of position map, and the PosMap of the rst hierarchy is stored in the trusted memory on-chip (Figure 1). e above trick can be repeated, i.e., adding more hierarchies of position map to further reduce the nal position map size at the expense of increased latency. Notice that all the position map hierarchies (except for the nal position map) are stored in the untrusted DRAM along with the actual data blocks, and can be treated as regular data blocks; this technique is called Uni ed ORAM [9].
Uni ed ORAM scheme reduces the performance penalty of recursion by caching position map ORAM blocks in a Position map Lookaside Bu er (PLB) to exploit locality (similar to the TLB exploiting locality in page tables). To hide whether a position map access hits or misses in the cache, Uni ed ORAM stores both data and position map blocks in the same binary tree. Further compression of PosMap structure is done by using Compressed Position Map technique discussed in section 6.1.

Background Eviction
Stash (cf. Section 4.1) is a small bu er to temporarily hold the dirty data blocks evicted from the LLC/PLB. If at any time, the rate of blocks being added to the stash becomes higher than the rate of evictions from the stash, the blocks may accumulate in stash causing a stash over ow. Background eviction [32] is a proven and secure technique proposed for Path ORAM to prevent stash over ow. e key idea of background evictions is to temporarily stop serving the real requests (which increase stash occupancy) and issue background evictions or so-called dummy accesses (which decrease stash occupancy) when the stash is full.
We use background eviction technique to eliminate the possibility of a stash over ow event. When the stash is full, the ORAM controller suspends the read requests which consequently stops any write-back requests preventing the stash occupancy to increase further. en it simply chooses random locations and, if vacant, evicts the blocks from the stash until the stash occupancy is reduced to a safe threshold. e probability of a successful eviction in each a empt is determined by the DRAM utilization parameter, i.e. the ratio of occupied blocks to the total blocks in DRAM. In our experiments, we choose a utilization of ≈ 1/2, therefore each eviction a empt has ≈ 50% probability of success. Note that background evictions essentially push the problem of stash over ow to the program's termination channel. Con guring the DRAM utilization to be less than 1 guarantees the termination, and we demonstrate good performance for a utilization of 1/2 in Section 7. Although it is true that background eviction may have di erent e ect on the total runtime for di erent applications, or even for di erent inputs for the same application which leak the information about data locality etc. However, the same argument applies to Path ORAM based systems, and also for other system components as well, such as enabling vs. disabling branch prediction or the L3 cache etc. Protecting any leakage through the program's termination time is out of scope of this paper (cf. Section 2).

Periodic ORAM
As mentioned in our adversarial model, the core de nition of ORAM [12] do not leakage over ORAM timing or termination channel (cf. Section 2). Likewise, the fundamental algorithm of Flat ORAM (Algorithm 3) does not target to prevent these leakages. erefore in order to protect the ORAM timing channel, we adapt the Flat ORAM algorithm to issue periodic ORAM accesses, while maintaining its security guarantees.
In the literature, periodic variants of Path ORAM have been presented [11] which simply always issue ORAM requests at regular periodic intervals. However, under write-only ORAMs, such a straightforward periodic approach would break the security as explained below. Since in write-only ORAMs, the read requests do not leave a trace, therefore for a logical access sequence of (Write, Read, Write), the adversary will only see two writes occurring at times 0 and 2T for T being the interval between two ORAM accesses. e access at time T will be omi ed which reveals to the adversary that a read request was made at this time.
To x this problem, we modify the Flat ORAM algorithm as follows. Among the periodic access, for every real read request to physical block s, another randomly chosen physical block s is also read. Block s is consumed by the processor, whereas block s is re-encrypted (under probabilistic encryption) and wri en back to the same location from where it was read. is would always result in update(s) to the memory a er each and every time period T .
Security: A few things should be noted: First, it does not ma er whether the location s contains real or dummy data, because the plain-text data content is never modi ed but just re-encrypted. Second, writing back s is indistinguishable from a real write request as this location is chosen uniformly at random. ird, this write to s does not reveal any trace of the actual read of s as the two locations are totally independent.
We present our simulation results for periodic Flat ORAM in the evaluation section.

EFFICIENT COLLISION AVOIDANCE
In sections 4.3 and section 4.4, we discuss how Flat ORAM outperforms Path ORAM by avoiding redundant memory accesses. However, an immediate consequence of this is the problem of collisions which now becomes the main performance bo leneck. A collision refers to a scenario when a physical location s, which is randomly chosen to write a logical block a, already contains useful data which cannot be overwri en. e overall e ciency of such a write-only ORAM scheme boils down to its collision avoidance mechanism. In the following subsections, we discuss the occupancy map based collision avoidance mechanism of Flat ORAM in detail and compare it with HIVE's inverse position map based collision avoidance scheme.

Inverse Position Map Approach
Since a read access must not leave its trace in the memory in order to avoid the linkability problem, a naive approach of marking the physical location as 'vacant' by writing 'dummy' data to it upon each read is not possible. HIVE [2] proposes an inverse position map structure that maps each physical location to a logical address. Before each write operation to a physical location s, a potential collision check is performed which involves two steps. First the logical address a linked to s is looked up via the inverse position map. en the regular position map is looked up to nd out the most recent physical location s linked to the logical address a. If s = s then since the two mappings are synchronized, it shows that s contains useful data, hence a collision has occurred. Otherwise, if s s then this means that the entry for s in the inverse position map is outdated, and block a has now moved to a new location s . erefore, the current location can be overwri en, hence no collision.
HIVE stores the encrypted inverse position map structure in the untrusted storage at a xed location. With each physical block being updated, the corresponding inverse position map entry is also updated. Since write-only ORAMs do not hide the physical block ID of the updated block, therefore revealing the position of the corresponding inverse position map entry does not leak any secret information.
We demonstrate in our evaluations that the large size of inverse position map approach introduces storage as well as performance overheads. For a system with a block size of B bytes and total N logical blocks, inverse position map requires log 2 (N ) additional bits space for each of the P physical blocks. Crucially, this large size of a single inverse position map entry restricts the total number of entries per block to a small constant, which leads to less locality within a block. is results in performance degradation.

Occupancy Map Approach
In the simpli ed OccMap based approach, each of the P physical blocks requires just one additional bit to store the occupancy information (vacant/occupied). In terms of storage, this gives log 2 (N ) times improvement over HIVE.

Insecurely Managing the Occupancy
Map. e OccMap array bits are rst sliced into chunks equal to the ORAM block size (B bytes). We call these chunks the OccMap Blocks. Notice that each OccMap block contains occupancy information of 8B physical locations (i.e. 8 bits per byte; 1-bit per location). Now the challenge is to e ciently store these blocks somewhere o -chip. A naive approach would be to encrypt OccMap blocks under probabilistic encryption, and store them contiguously in a dedicated xed area in DRAM. However under Flat ORAM algorithm, this approach would lead to a serious security aw which is explained below.
If the OccMap blocks are stored contiguously at a xed location in DRAM, an adversary can easily identify the corresponding OccMap block for a given a physical address; and for a given OccMap block, he can identify the contiguous range of corresponding 8B physical locations. With that in mind, when a data block a (previously read from s old ) is evicted from the stash and wri en to location s new (cf. Algorithm 3), the old OccMap entry is marked as 'vacant' and the new OccMap entry is marked as 'occupied'; i.e. two OccMap blocks O old and O new are updated. Furthermore, the s new location, which falls in the contiguous range covered by one of the two updated OccMap blocks, is also updated with the actual data -thereby revealing the identity of O new . is reveals to the adversary that a logical data block was previously read from some location within the small contiguous range covered by O old , and it is now wri en to location s new (i.e. coarse grained linkability). Recording several such instances of read-write access pairs and linking them together  in a chain reveals the precise pa ern of movement of logical block a across the whole memory.

Securely Managing the Occupancy Map.
To avoid this problem, we treat the OccMap blocks as regular data blocks, i.e., OccMap blocks are encrypted and also randomly distributed throughout the whole DRAM, and tracked by the regular PosMap. Figure 1 shows the logical organization of data, PosMap and OccMap blocks. e OccMap blocks are added as 'additional' data blocks at the data hierarchy (Hierarchy 0). en the recursive position map hierarchies are constructed on top of the complete data set (data blocks and OccMap blocks). Every time an OccMap block is updated, it is mapped to a new random location and hence avoids the linkability problem. We realize that this approach results in overall more position map blocks, however for practical parameters se ings (e.g. 128B block size, 8GB DRAM, 4GB working set), it does not result in an additional PosMap hierarchy. erefore the recursive position map lookup latency is una ected.

Performance Related Optimizations
Now that we have discussed how to securely store the occupancy map, we move on to discuss some performance related optimizations implemented in Flat ORAM.

Locality in OccMap
Blocks. For realistic parameters, e.g. 128 bytes block size, each OccMap block contains occupancy information of 1024 physical locations. is factor is termed as OccMap scaling factor. e dense structure of OccMap blocks o ers an opportunity to exploit spatial locality within a block. In other words, for a large scaling factor it is more likely that two randomly chosen physical locations will be covered by the same OccMap block, as compared to a small scaling factor. In order to also bene t from such locality, we cache the OccMap blocks as well in PLB along with the PosMap blocks. For a fair comparison with HIVE in our experiments, we also model the caching of HIVE's inverse position map blocks in PLB. Our experiments con rm that the OccMap blocks show a higher PLB hit rate as compared to HIVE's inverse position map blocks cached in PLB in the same manner. e reason is that, for the same parameters, the scaling factor of inverse position map approach is just about 40 which results in a larger size of the data structure and hence more capacity-misses from the PLB.

Dirty vs. Clean Evictions from LLC & PLB.
An eviction of a block from the LLC where the data content of the block has not been modi ed is called a clean eviction, whereas an eviction where the data has been modi ed is called dirty eviction. In Path ORAM, since all the read operations also need to be obfuscated, therefore following a read operation, when a block gets evicted from the LLC, it must be re-wri en to a new random location even if its data is unmodi ed, i.e. a clean eviction. is is crucial for Path ORAM's security as it guarantees that successive reads to the same logical block result in random paths being accessed. is notion is termed as read-erase, which assumes that the data will be erased from the memory once it is read.
In write-only ORAMs, however, since the read access pa erns are not revealed therefore the notion of read-erase is not necessary. A data block can be read from the same location as o en as needed as long as it's contents are not modi ed. We implement this relaxed model in Flat ORAM which greatly improves performance. Essentially, upon a clean eviction from the LLC, the block can simply be discarded since one useful copy of the data is still stored in the DRAM. Same reasoning applies to the clean evictions from the PLB. Only the dirty evictions are added to the stash to be wri en back at a random location in the memory.

Implications on PLB & Stash Size
Each dirty eviction requires not only the corresponding data block to be updated but also the two related OccMAP blocks which store the new and old occupancy information. In order to relocate these blocks to new random positions, the 'd' hierarchies of corresponding PosMAP blocks will need to be updated and this in turn implies updating their related OccMAP blocks, and so on. If not prevented, this avalanche e ect will repeatedly ll the stash implying background evictions which stop serving real requests and increase the termination time. For a large enough PLB with respect to a benchmark's locality, most of the required OccMAP blocks during the benchmark's execution will be in the PLB. is prevents the avalanche e ect most of the time (as our evaluation shows) since the OccMAP blocks in PLB can be directly updated.
Even if all necessary OccMAP blocks are in PLB, a dirty eviction still requires the data block with its d PosMAP blocks to be updated. Each of these d + 1 blocks is successfully evicted from the stash with probability 1/2, determined by the DRAM utilization, on each a empt. e probability that exactly m a empts are needed to evict all d + 1 blocks is equal to m−1 d /2 m . is probability becomes very small for m equal to a small constant c times d log d. If the dirty eviction rate (per DRAM access) is at most 1/c, then the stash size will mostly be contained to a small size (e.g., 100 blocks for d = 4) so that additional background eviction which stops serving real requests is not needed.
Notice that the presented write-only ORAM is not asymptotically e cient: In order to show at most a constant factor increase in termination time with overwhelming probability, a proper argument needs to show a small probability of requiring background eviction which halts normal execution. An argument based on M/D/1 queuing theory or 1-D random walks needs the OccMap to be always within the PLB and this means that the e ective stash size as compared to Path ORAM's stash de nition includes this PLB which scales linearly with N and is not O(lo N ).

ADOPTING MORE EXISTING TRICKS
Here we discuss a few more architectural optimizations from the Path ORAM paradigm which can be awlessly incorporated and are implemented in Flat ORAM for further improvements and features. To achieve this goal, Compressed Position Map [9] has been proposed, which results in less PosMap hierarchies than uncompressed PosMap. e basic idea is to store a monotonically increasing counter in the PosMap entry for each logical data block. is counter along with the block's logical address is used as a 'seed' to a keyed pseudo-random function in order to compute a random position for the block. Every time a block is to be wri en, its PosMap counter is rst incremented so that a new random position is generated by the pseudo-random function for the block. To compress these counters to a feasible size, [9] presents an optimization using a big group counter and several small individual counters per PosMap block. We refer the readers who might be interested in more details to the above citation.
We tweak the compressed PosMap technique for Flat ORAM. e key modi cation is that the counter for any block to be evicted is incremented even upon unsuccessful eviction a empts, i.e. even if a collision is detected. It is important because otherwise the pseudo-random function will generate the same random location over and over which is already occupied, and hence the block will never be evicted.

Integrity Veri cation (PMMAC)
Flat ORAM also implements an e cient memory integrity verication technique termed as PosMap MAC (PMMAC) [9]. PMMAC leverages the per-block counters of compressed PosMap to perform MAC 5 checks on the data upon reads. Suppose a logical block a has a counter c, then upon writes, the ORAM controller computes a MAC h = MAC K (a || c || data) using the secret key K and writes the tuple (h, data) to the DRAM. Upon reads, the potentially tampered data tuple (h * , data * ) is read. e ORAM controller recomputes h = MAC K (a || c || data * ) and checks whether h = h * . If so, the data integrity is veri ed. Also, since the counter is incremented upon every write, the freshness of the data is also veri ed, i.e. integrity check guarantees that the most recently wri en data has been read.

EXPERIMENTAL EVALUATION 7.1 Methodology
We use Graphite [28] to model di erent ORAM schemes in all our experiments. Graphite simulates a tiled multi-core chip. e hardware con gurations are listed in Table 1. We assume there is only one memory controller on the chip, and all ORAM accesses are serialized. e DRAM in Graphite is simply modeled by a at latency. e 16 GB/s is calculated assuming a 1 GHz chip with 128 pins and pins are the bo leneck of the data transfer.
We use Splash-2 [46], SPEC06 [18] and two OLTP database management system (DBMS) [47] workloads namely YCSB [7] and 5 Message Authentication Code (MAC), e.g. a keyed cryptographic hash, is a small piece of information to verify the authenticity of a message/data.

Performance Comparison
Although all ORAM schemes incur performance slowdown over DRAM, however it is important to note that this slowdown is proportional to the memory intensiveness of the application. Memory bound applications su er from higher performance degradation than compute bound applications. Splash2 and SPEC06 benchmarks are sorted in ascending order of slowdowns over DRAM from le to right. We consider all the benchmarks with less than 2× overhead as Computation Intensive benchmarks (plo ed over green background) and all those with more than 2× overhead as Memory Intensive benchmarks (plo ed over red background).
Clearly, Path ORAM incurs the highest overhead, as expected, among all three ORAM schemes because it is a fully functional ORAM which provides higher security. However, the point of presenting this comparison is to convince the readers that using Path ORAM for only write-access protection is indeed an overkill when be er alternatives (e.g. HIVE, Flat ORAM) exist. On average, Path ORAM incurs about 8.6× slowdown for Splash2 and 5.3× for SPEC06 memory intensive benchmarks (mem avg). TPCC and YCSB incur 7.2× and 9.3× slowdowns respectively.
HIVE also shows signi cant performance degradation compared to Flat ORAM for memory intensive benchmarks (ocean contiguous, ocean non contiguous, mcf, tpcc, ycsb). e average slowdown of HIVE adaptation for memory bound Splash2 and SPEC06 workloads even a er several additional optimizations   is performance gap is primarily because the inverse position map approach of HIVE results in signi cantly increased number of additional DRAM accesses. Figure 2 also shows normalized total number of DRAM accesses w.r.t. insecure DRAM system (shown by red markers) for HIVE and Flat ORAM. ese numbers include both the DRAM accesses issued to serve regular ORAM requests and also the ones caused by background evictions (cf. Section 4.6). e normalized access count for Path ORAM is around 200 on average, and is not shown on the plots. It can be seen that HIVE issues 8.2 and 4.2 DRAM accesses as opposed to Flat ORAM's 4.8 and 2.3 accesses on average for each request issued by the processor for memory intensive Splash2 and SPEC06 workloads respectively. e reason for higher number of DRAM accesses from HIVE can be found in Figure 3 which shows the overall PLB hit rate of both HIVE and Flat ORAM. e large memory footprint of the HIVE's inverse position map structure results in overall more data being inserted into the PLB and hence translates into higher number of evictions from PLB. Consequently the ORAM controller experiences higher number of PLB misses and issues relatively higher number of DRAM accesses. Whereas the dense structure of OccMap o ers a smaller memory footprint, thus causing less PLB evictions and exhibiting be er locality (cf. Section 5.3) which directly translates into performance gain. e normalized number of DRAM accesses is proportional to the energy consumption of the memory subsystem. I.e., a higher number of DRAM accesses would result in more energy consumption. On average, Flat ORAM saves up to 80% energy over HIVE for various workloads.

Sensitivity Study
In this section, we will study how di erent parameters in the system a ect the performance of write-only ORAMs.
DRAM Utilization: When a block needs to be wri en to the DRAM, a random position is chosen and if that location is vacant, the block is wri en at that location (cf. Section 4.2). e probability that a randomly chosen location is 'vacant' is determined by the DRAM utilization, i.e. the ratio of occupied blocks to total blocks in DRAM. In order to study the e ect of DRAM utilization, we show the results of various physical DRAM sizes (8, 16, 32GB) for a constant working set of 4GB in Figure 4. e resulting DRAM utilizations are 50%, 25% and 12.5% respectively.  Going from 50% to 25% utilization, memory intensive benchmarks (ycsb) gain performance, as the collisions during write operations are reduced by half. However, the jump from 25% to 12.5% utilization yields li le gain because the collision probability of 25% at 16GB mark is already too low to be a major performance bottleneck. Notice that HIVE bene ts more compared to Flat ORAM from the reduced collisions since it has a much higher collisionpenalty. Since less memory intensive benchmarks (sjeng) are not constrained by write operations anyway, lower utilizations do not help much.
Stash Size: As discussed in Section 4.6, when the stash occupancy increases than a particular threshold, the ORAM starts performing 'background evictions'. Since background evictions cause the real requests to be suspended temporarily, frequent background evictions cause performance degradation. A larger stash is less likely to become full and thus reduces background eviction rate and improves performance.
In Figure 5, the stash size is swept for two di erent benchmarks, one is highly memory intensive (ocean non contiguous) and the other one is signi cantly less memory bound (sjeng). e memory intensive benchmark bene ts from a large stash, as it experiences high background evictions rate at lower stash sizes. e less memory intensive benchmark does not bene t much from increased stash sizes, as it already has a low background evictions rate. In general, Flat ORAM shows signi cant performance gain over HIVE even at small stash sizes.

DRAM Latency & Bandwidth:
In Path ORAM, each ORAM access results in about 200 DRAM accesses on average under typical parameter se ings. Most of these accesses can be issued in a burst without waiting for the rst data block to arrive, since the addresses are known a priori, e.g. accessing a full path. erefore, the DRAM bandwidth becomes the main bo leneck in Path ORAM, whereas the DRAM latency plays less signi cant role as it is incurred less o en. However, the write-only ORAMs under consideration typically only issue less than 10 DRAM accesses per ORAM access (cf. Section 7.2). Furthermore, there could be interdependencies within these 10 accesses, e.g., reading an OccMap block to nd out if a position is vacant, and then issuing further writes in case a vacant position is found. In such cases, DRAM latency is incurred more o en and hence plays more prominent role in the overall performance than the DRAM bandwidth.
is phenomenon is shown in DRAM latency and bandwidth sweep studies in Figure 6 and Figure 7 respectively. Memory intensive benchmarks (ocean contiguous) are more sensitive to DRAM latency and experience more performance degradation at higher latencies. On the other hand, compute bound benchmarks (sjeng) are less sensitive to the DRAM latency. Increasing the DRAM bandwidth seems to help only a li le as expected and explained in the discussion above.
Periodic ORAM:. Figure 8 shows the experimental results of periodic write-only ORAM schemes. e results are normalized to insecure DRAM. e period in terms of number of cycles between two consecutive ORAM accesses is chosen to be 100 cycles. In general, adding periodicity to our ORAM scheme does not signi cantly hurt performance.

CONCLUSION
We propose an e cient and practical write-only Oblivious RAM scheme called Flat ORAM for secure processor architectures. It is the rst write-only ORAM with a concrete implementation in secure processors domain. e implementation details are discussed and the design space is comprehensively explored. On memory intensive Splash-2 and SPEC06 benchmarks, Flat ORAM only incurs on average 3× and 1.6× slowdown respectively. Compared to a closest related work in the literature, Flat ORAM o ers up to 75% higher performance and 80% energy savings. ACKNOWLEDGMENTS e work is partially supported by NSF grant CNS-1413996 for MACS: A Modular Approach to Cloud Security.