Compression and Huffman Codes for Object Oriented Programming II | CMSC 132, Study notes of Computer Science

Download Compression and Huffman Codes for Object Oriented Programming II | CMSC 132 and more Study notes Computer Science in PDF only on Docsity!

Compression & Huffman Codes

Nelson Padua-Perez

William Pugh

Department of Computer Science

University of Maryland, College Park

Compression

Definition

Reduce size of data

(number of bits needed to represent data)

Benefits

Reduce storage needed

Reduce transmission cost / latency / bandwidth

Sources of Compressibility

Redundancy

Recognize repeating patterns

Exploit using

Dictionary

Variable length encoding

Human perception

Less sensitive to some information

Can discard less important data

Types of Compression

Lossless

Preserves all information

Exploits redundancy in data

Applied to general data

Lossy

May lose some information

Exploits redundancy & human perception

Applied to audio, image, video

Effectiveness of Compression

Depends on data

Random data ⇒ hard

Example: 1001110100 ⇒?

Organized data ⇒ easy

Example: 1111111111 ⇒ 1 × 10

Corollary

No universally best compression algorithm

Effectiveness of Compression

Lossless Compression is not guaranteed

Pigeonhole principle

Reduce size 1 bit ⇒ can only store ½ of data

Example

000, 001, 010, 011, 100, 101, 110, 111 ⇒ 00, 01, 10, 11

If compression is always possible (alternative view)

Compress file (reduce size by 1 bit)

Recompress output

Repeat (until we can store data with 0 bits)

Huffman Code

Approach

Variable length encoding of symbols

Exploit statistical frequency of symbols

Efficient when symbol probabilities vary widely

Principle

Use fewer bits to represent frequent symbols

Use more bits to represent infrequent symbols

A A B A

A A B A

Huffman Code Example

Expected size

Original ⇒ 1/8×2 + 1/4×2 + 1/2×2 + 1/8×2 = 2 bits / symbol

Huffman ⇒ 1/8×3 + 1/4×2 + 1/2×1 + 1/8×3 = 1.75 bits / symbol

Symbol

3 bits 2 bits 1 bit 3 bits

Huffman 110 10 0 111

Encoding

2 bits

Bird

Frequency 1/8 1/4 1/

2 bits 2 bits 2 bits

Original

Encoding

Dog Cat Fish

Huffman Code Algorithm Overview

Encoding

Calculate frequency of symbols in file

Create binary tree representing “best” encoding

Use binary tree to encode compressed file

For each symbol, output path from root to leaf

Size of encoding = length of path

Save binary tree

Huffman Code – Creating Tree

Algorithm

Place each symbol in leaf

Weight of leaf = symbol frequency

Select two trees L and R (initially leafs)

Such that L, R have lowest frequencies in tree

Create new (internal) node

Left child ⇒ L

Right child ⇒ R

New frequency ⇒ frequency( L ) + frequency( R )

Repeat until all nodes merged into one tree

Huffman Tree Construction 2

A

C E

H

I

Huffman Tree Construction 3

A

C

E

H

I

Huffman Tree Construction 5

A

C E

H

I

E = 01

I = 00

C = 10

A = 111

H = 110

Huffman Coding Example

Huffman code

Input

ACE

Output

E = 01

I = 00

C = 10

A = 111

H = 110