Download Hashing Algorithms: Dictionaries & Universally Hashing Functions - CS357, UT Austin 2007 and more Study notes Health sciences in PDF only on Docsity!
The University of Texas at Austin Dictionaries; Hashing Department of Computer Sciences Professor Vijaya Ramachandran Lecture 20 CS357: ALGORITHMS, Spring 2007
1 Dictionaries
A dictionary is a data structure that supports the operations of Search, Insert, Delete.
- Search(S, k) is given the set S and a key value k. It returns a pointer to an element x in S such that key(x) = k. If there is no element in S with key value k it returns NIL.
- Insert(S, x) is given the set S and a pointer to an element x (which comes with a key value supplied to it). It forms the set S ∪ {x}.
- Delete(S, x) is given the set S and a pointer to an element x in S. It forms the set S − {x}, i.e., it removes element x from the set S.
There are two important variants of a dictionary.
- A static dictionary is a restricted version that supports only the Search operation. The goal here is to come up with a compact representation of the set while supporting very fast look-ups.
- When the elements of the set come from a totally ordered set, a dic- tionary on a total order supports Minimum, Maximum, Successor, Predecessor operations in addition to the three standard dictionary operations.
We will study two widely-used approaches to maintaining a dictionary effi- ciently: hashing and balanced search trees.
2 Hashing
Hashing is a widely-used class of data structures that support the operations of insert, delete and search on a dynamic set of keys S. The keys are drawn from a universe U (|U| >> |S|), which for our purposes will be a set of positive integers. These keys are mapped into a hash table using a hash function; the size of the hash table is usually within a constant factor of the number of elements in the current set S.
There are two main methods used to implement hashing: hashing with chain- ing and hashing with open addressing. We will study hashing with chaining here.
2.1 Hashing with Chaining
In hashing with chaining, the elements in S are stored in a hash table T [0 .. m− 1] of size m, where m is somewhat larger than n, the size of S. The hash table is said to have m slots. Associated with the hashing scheme is a hash function h which is a mapping from U to { 0 , · · · , m − 1 }. Each key k ∈ S is stored in location T [h(k)], and we say that key k is hashed into slot h(k). If more than one key in S hashes into the same slot, then we have a collision. In such a case, all keys that hash into the same slot are placed in a linked list associated with that slot; this linked list is called the chain at that slot.
The load factor of a hash table is defined to be α = n/m; it represents the average number of keys per slot. We typically operate in the range m = Θ(n) so α is usually a constant (usually α < 1). If a large number of insertions or deletions destroys this property, a more suitable value of m is chosen and the keys are re-hashed into a new table with a new hash function.
In simple uniform hashing we assume that we have a ‘good’ hash function h such that any element in U is equally likely to hash into any of the m slots in T , independent of the other keys in the table. We also assume that h(k) can be computed in constant time.
In hashing with chaining, inserts and deletes can be performed in constant time with a suitable linked list representation for the chains. In the worst case, a search operation can take as long as n steps if all keys hash into the same slot. However, if we assume simple uniform hashing then the expected time for a search operation is easily shown to be Θ(α), which is a constant under the normal operation condition of m = Θ(n). The main drawback with this method is the requirement that the scheme implements simple uniform hashing.
2.2 Universal Hashing
Let H = {h 1 , h 2 , · · · , hr} be a finite family of hash functions, where each hi is a mapping from the universe U to { 0 , 1 , · · · , m − 1 }. This family is universal if, for each pair of distinct keys x, y ∈ U, the number of hash functions in H for which hi(x) = hi(y) is at most r/m.
Observation. If hi is chosen uniformly at random from H, then
P r(hi(x) = hi(y)) ≤
r
r m
m
for any distinct pair of keys x, y
Assume that a hash function h that is chosen uniformly from a universal family H is used to hash n keys into a table T [0..m − 1]. Then, for any given key k, using the above Observation we can bound the expected length of the chain at slot h(k) as follows:
For each key l in T , l 6 = k, let Xkl be the i.r.v. of the event that h(k) = h(l).
Let Y denote the number of keys in T other than k that hash into h(k). Then, Y =
∑ l∈T,l 6 =k Xkl.
Case 1. Key k is not in the table. Then, Y is equal to the length of the chain at slot h(k).
By the above Observation, we have
E[Y ] =
∑
l∈T,l 6 =k
E[Xkl] ≤
m
· n = α
Case 2. Key k is in the table. Then, the length of the chain at slot h(k) is Y + 1, since the occurrence of key k is not counted in Y. In this case we have E[Y ] ≤ (n − 1)/m < α, hence the expected length of the chain at h(k) is less than α + 1.
Thus, by choosing m = Θ(n), and by choosing a random hash function from an universal family of hash functions we can achieve constant expected time for the dictionary operations of search, insert and delete, while using only linear space.
2.3 Constructing a Universal Family of Hash Func-
tions
As before, let U be the universe, m the size of the hash table and n the number of keys in the hash table.
Let p be a prime larger than |U|.
Let Zp = { 0 , 1 , · · · , p − 1 } (the additive group mod p).
(Similarly Zm = { 0 , 1 , · · · , m − 1 } is the additive group mod m).
Let Z p∗ = { 1 , · · · , p − 1 } (the multiplicative group mod p).
For all a ∈ Z p∗ , b ∈ Zp, define ha,b : U → { 0 , 1 , · · · m − 1 } by
ha,b(k) = ( (a · k + b) mod p) mod m
Finally, define the family Hp,m = {ha,b : a ∈ Z p∗ , b ∈ Zp}.
Note that |Hp,m| = p(p − 1). (Since p > |U|, this is a very large family.)
Theorem. Hp,m is a universal family of hash functions.
Proof. Omitted.
Claim 2. Let the ni be as defined in step 1 of the perfect hashing algorithm. Then,
E[
n∑− 1
i=
n^2 i ] < 2 n and P r(
n∑− 1
i=
n^2 i > 4 n) < 1 / 2
Proof. Let us use C(n, k) to denote (^) (n−nk!)!k!. We will use the following two facts:
- A useful identity: a^2 = a + 2 · C(a, 2) for any real value a.
- If a set of n elements is hashed on to a table of size m using a random hash function drawn from a universal family, then the expected number of collisions ≤ C(n, 2) · 1 /m by definition of a universal family.
So we have
E[
n∑− 1
i=
n^2 i ] = E[
n∑− 1
i=
(ni + 2 · C(ni, 2))] using the identity for each ni
= E[
n∑− 1
i=
ni] + 2E[
n∑− 1
i=
C(ni, 2)] (by linearity of expectations)
≤ n + 2C(n, 2) · 1 /n (since second term is the expected no. of collisions) = n + 2(n − 1)/ 2 < 2 n
The second part of the Claim follows from Markov’s inequality as in Claim
- []
With this scheme, any key can be searched using just two probes, one into the primary table to obtain the secondary hash function, and the second into the appropriate secondary table.
The method we have just described is the first scheme efficient proposed for perfect hashing. Since then there have been other methods proposed for perfect hashing that work with just a constant number of hash functions.
2.4 Bloom Filters
A Bloom filter represents a set S of n elements with an array T of m bits. (Typically the elements themselves are stored in slower memory, and the Bloom filter is stored in main memory, and is used as an initial ‘filter’ to determine if it is necessary to access the slower memory for a given element.)
Initially all bits in the array T are 0.
We use k independent hash functions h 1 , · · · , hk that implement simple uni- form hashing, each mapping from U to { 0 , · · · , m − 1 }.
Each x ∈ S is inserted by setting bit hi(x) to 1, for 1 ≤ i ≤ k. Note that a given bit may be set by many different elements of S.
Search(S, y) checks if hi(y) = 1 for all i, 1 ≤ i ≤ k. If so, it reports that y is in set S, otherwise it reports ‘no’.
If Search returns ‘no’ we know that the element y is not in the set S. How- ever, sometimes, when y is not in S, the Search command may incorrectly report that y is in S. This is called a false positive. This could happen if all of the bit positions hi(y) were set to 1 during the insertion of elements of S.
What is the probability of a false positive, i.e., what is the false positive rate?
Let p be the probability that a given bit position in table T remains 0 after all elements in S are inserted. Since each hash function implements simple uniform hashing and the hash functions are independent, we have
p = (1 −
m
)kn^ ≈ e−kn/m
A false positive occurs if all k positions examined during the search for an element that is not in S return a 1. Hence, since the hi are independent,
False positive rate F = (1 − p)k^ ≈ (1 − e−kn/m)k^ (1)
Suppose we wish to optimize for k in terms of minimizing the false positive rate, given n and m. There are two competing forces at play:
- the larger k is, the more chances we have to find a 0 when we search for an element that is not in S, so this helps in reducing the false positive rate; but
- the larger k is, the smaller the probability that we will find a 0 at any given position
