Data Compression Two - Data Structures - Lecture Slides, Slides of Data Structures and Algorithms

Download Data Compression Two - Data Structures - Lecture Slides and more Slides Data Structures and Algorithms in PDF only on Docsity!

1

Data Compression

2

Introduction

 Compression is used to reduce the

volume of information to be stored into

storages or to reduce the

communication bandwidth required for

its transmission over the networks

4

Compression Principles

 Entropy Encoding  Run-length encoding  Lossless & Independent of the type of source information  Used when the source information comprises long substrings of the same character or binary digit (string or bit pattern, # of occurrences), as FAX e.g) 000000011111111110000011…… ⇒ 0,7 1, 10, 0,5 1,2…… ⇒ 7,10,5,2……

5

Compression Principles

 Entropy Encoding  Statistical encoding  Based on the probability of occurrence of a pattern  The more probable, the shorter codeword  “Prefix property”: a shorter codeword must not form the start of a longer codeword

7  E.g) symbols M(10), F(11), Y(010), N(011), 0(000), 1(001) with probabilities 0.25, 0.25, 0.125, 0.125, 0.125, 0.  H’ = Σ i = 6 N i P i = (2(2×0.25) + 4(3×0.125)) = 2. bits/codeword  H = -Σ i = 6 P i log 2 P i = - (2(0.25log 2 0.25) + 4(0.125log 2 0.125)) = 2.  E = H/H’ =100 %  3-bit/codeword if we use fixed-length codewords for six symbols

8

Huffman Algorithm

Method of construction for an encoding tree

• Full Binary Tree Representation
• Each edge of the tree has a value,

(0 is the left child, 1 is the right child)

• Data is at the leaves, not internal nodes
• Result: encoding tree
• “Variable-Length Encoding”

10

• Huffman coding
  • variable length code whose length is inversely proportional to that character’s frequency
  • must satisfy nonprefix property to be uniquely decodable
  • two pass algorithm
    • first pass accumulates the character frequency and generate codebook
    • second pass does compression with the codebook

11

• create codes by constructing a binary tree
  1. consider all characters as free nodes
  2. assign two free nodes with lowest frequency to a parent nodes with weights equal to sum of their frequencies
  3. remove the two free nodes and add the newly created parent node to the list of free nodes
  4. repeat step2 and 3 until there is one free node left. It becomes the root of tree Huffman coding

13

• Example(64 data)

• R K K K K K K K

• K K K R R K K K

• K K R R R R G G

• K K B C C C R R

• G G G M C B R R

• B B B M Y B B R

• G G G G G G G R

• G R R R R G R R

14

• Color frequency Huffman code
• =================================
• R 19 00
• K 17 01
• G 14 10
• B 7 110
• C 4 1110
• M 2 11110
• Y 1 11111

16

Static Huffman Coding

 Huffman (Code) Tree  Given : a number of symbols (or characters) and their relative probabilities in prior  Must hold “ prefix property ” among codes Symbol Occurrence A 4/ B 2/ C 1/ D 1/ Symbol Code A 1 B 01 C 001 D 000 4 ×1 + 2×2 + 1×3 + 1 ×3 = 14 bits are required to transmit “AAAABBCD” 0 1 D A B C 0 1 0 8 1 4 2 Leaf node Root node Branch node Prefix Property!

17

The end