Docsity
Docsity
Prepare for your exams
Prepare for your exams

Study with the several resources on Docsity

Earn points to download
Earn points to download

Earn points by helping other students or get them with a premium plan

Guidelines and tips
Guidelines and tips
Sell on Docsity
Sell on Docsity
Docsity AI
Docsity AI
Log inSign up

Prepare for your exams

Study with the several resources on Docsity

Find documents

Prepare for your exams with the study notes shared by other students like you on Docsity

Search for your university

Find the specific documents for your university's exams

Docsity AINEW

Summarize your documents, ask them questions, convert them into quizzes and concept maps

Explore questions

Clear up your doubts by reading the answers to questions asked by your fellow students

Earn points to download

Earn points by helping other students or get them with a premium plan

Share documents
download point icon20 Points

For each uploaded document

Answer questions
download point icon5 Points

For each given answer (max 1 per day)

All the ways to get free points
Get points immediately

Choose a premium plan with all the points you need

Study Opportunities

Choose your next study program

Get in touch with the best universities in the world. Search through thousands of universities and official partners

Community

Ask the community

Ask the community for help and clear up your study doubts

Free resources

Our save-the-student-ebooks!

Download our free guides on studying techniques, anxiety management strategies, and thesis advice from Docsity tutors

Bloom Filters - Advanced Data Structures - Lecture Slides, Slides of Computer Science

Alagappa UniversityComputer Science

These are the Lecture Slides of Advanced Data Structures which includes Split Algorithm, Unbalanced Binary Search Trees, Forward Pass, Forward Pass Example, Backward Cleanup Pass, Retrace Path, Current Nodes, Roots of Respective Tries, Branch Nodes etc. Key important points are: Bloom Filters, Differential Files, Simple Large Database, File of Records, Naive Mode of Operation, Index and File Change, Changes to Master File, Differential File Operation, Recovery Time

Typology: Slides

2012/2013

Uploaded on 03/19/2013

dharmakeerti
dharmakeerti 🇮🇳

4.2

(27)

89 documents

1 / 11

Toggle sidebar

This page cannot be seen from the preview

Don't miss anything!

bg1
Bloom Filters
•Differential Files
•Simple large database.
File of records residing on disk.
Single key.
Index to records.
•Operations.
Retrieve.
Update.
•Insert a new record.
•Make changes to an existing record.
•Delete a record.
Naïve Mode Of Operation
•Problems.
Index and File change with time.
Sooner or later, system will crash.
Recovery =>
•Copy Master File (MF) from backup.
•Copy Master Index (MI) from backup.
•Process all transactions since last backup.
Recovery time depends on MF & MI
size + #transactions since last backup.
Key
Index
File
Ans.
Docsity.com
pf3
pf4
pf5
pf8
pf9
pfa

Partial preview of the text

Download Bloom Filters - Advanced Data Structures - Lecture Slides and more Slides Computer Science in PDF only on Docsity!

Bloom Filters

  • Differential Files
  • Simple large database. ƒ File of records residing on disk. ƒ Single key. ƒ Index to records.
  • Operations.

ƒ Retrieve. ƒ Update.

  • Insert a new record.
  • Make changes to an existing record.
  • Delete a record.

Naïve Mode Of Operation

  • Problems. ƒ Index and File change with time. ƒ Sooner or later, system will crash. ƒ Recovery => - Copy Master File (MF) from backup. - Copy Master Index (MI) from backup. - Process all transactions since last backup. ƒ Recovery time depends on MF & MI size + #transactions since last backup.

Key

Index

File

Ans.

Differential File

  • Make no changes to master file.
  • Alter index and write updated record to a new file called differential file.

Differential File Operation

Key

Index

File

Ans.

DF

  • Advantage. ƒ DF is smaller than File and so may be backed up more frequently. ƒ Index needs to be backed up whenever DF is. So, index should small as well. ƒ Recovery time is reduced.

Differential File & Index Operation

  • Performance hit. ƒ Most queries search both DI and Index. ƒ Increase in # of disk accesses/query.
  • Use a filter to decide whether or not DI should be searched.

Key

Index

File

Ans.Ans.

DF

DI N Y

Y

Ideal Filter

Key

Index

File

Ans.Ans.

DF

Filter N Y

Y

DI

Y

  • Y => this key is in the DI.
  • N => this key is not in the DI.
  • Functionality of ideal filter is same as that of DI.
  • So, a filter that eliminates performance hit of DI doesn’t exist.

Bloom Filter (BF)

  • N => this key is not in the DI.
  • M (maybe) => this key may be in the DI.
  • Filter error. ƒ BF says Maybe. ƒ DI says No.

Key

Index

File

Ans.Ans.

DF

BF

Y

N

Y

DI

M

N

Bloom Filter (BF)

  • Filter error. ƒ BF says Maybe. ƒ DI says No.

Key

Index

File

Ans.Ans.

DF

BF

Y

N

Y

DI

M

N

  • BF resides in memory.
  • Performance hit paid only when there is a filter error.

Bloom Filter Design

  • Use m bits of memory for the BF.
  • Larger m => fewer filter errors.
  • When DI empty, all m bits = 0.
  • Use h > 0 hash functions: f 1 (), f 2 (), …, fh().
  • When key k inserted into DI, set bits f 1 (k), f 2 (k), …, and fh(k) to 1.
  • f 1 (k), f 2 (k), …, fh(k) is the signature of key k.

Example

  • m = 11 (normally, m would be much much larger).
  • h = 2 ( 2 hash functions).
  • f 1 (k) = k mod m.
  • f 2 (k) = (2k) mod m.
  • k = 15.

0 0 0 0 0 0 0 0 0 0 0 1 2 3 4 5 6 7 8 9

1 1 1 0

  • k = 17.

1

Example

  • DI has k = 15 and k = 17.
  • Search for k. ƒ f 1 (k) = 0 or f 2 (k) = 0 => k not in DI. ƒ f 1 (k) = 1 and f 2 (k) = 1 => k may be in DI.
  • k = 6 => filter error.

0 0 0 0 0 0 0 0 0 0 0 1 2 3 4 5 6 7 8 9

1 1 1 1 0

Bloom Filter Design

  • Choose m (filter size in bits). ƒ Use as much memory as is available.
  • Pick h (number of hash functions).

ƒ h too small => probability of different keys having same signature is high. ƒ h too large => filter becomes filled with ones too soon.

  • Select the h hash functions. ƒ Hash functions should be relatively independent.

A = p(request for unmodified record)

  • p(update j is for record i) = 1/n.
  • p(record i not modified by update j) = 1 – 1/n.
  • p(record i not modified by any of the u updates)

= (1 – 1/n)u = A.

B = p(filter bits are all 1 for this

request)

  • Consider an update with key K.
  • p(fj(K) != i) = 1 – 1/m.
  • p(fj(K) != i for all j) = (1 – 1/m)h.
  • p(bit i = 0 after one update) = (1 – 1/m)h.
  • p(bit i = 0 after u updates) = (1 – 1/m)uh.
  • p(bit i = 1 after u updates) = 1 – (1 – 1/m)uh.
  • p(signature of K is 1 after u updates) = [1 – (1 – 1/m)uh]h = B.

Probability Of Filter Error

  • p(u) = A * B = (1 – 1/n)u^ * [1 – (1 – 1/m)uh]h
  • (1 – 1/x)q^ ~ e–q/x^ when x is large.
  • p(u) ~ e–u/n(1 – e–uh/m^ )h
  • d p(u)/dh = 0 => h = (ln 2)m/u ~ 0.693m/u.

Optimal h

  • h ~ 0.693m/u.
  • m = 10^6 , u = 10^6 / ƒ h ~ 1. ƒ Use h = 1 or h = 2. - m = 2*10^6 , u = 10^6 / ƒ h ~ 2. ƒ Use h = 2 or h = 3.

h

p(u)

hopt

p(u) ~ e–u/n(1 – e–uh/m^ )h