Data Structures - Data Structures and Algorithm - Lecture Slides, Slides of Data Structures and Algorithms

Download Data Structures - Data Structures and Algorithm - Lecture Slides and more Slides Data Structures and Algorithms in PDF only on Docsity!

Data Structures

& Algorithm Analysis

Problem Solving: Main Steps

1. Problem definition
2. Algorithm design / Algorithm specification
3. Algorithm analysis
4. Implementation
5. Testing
6. [Maintenance]

2. Algorithm Design / Specifications

• Algorithm: Finite set of instructions that, if followed, accomplishes a particular task.
• Describe: in natural language / pseudo-code / diagrams / etc.
• Criteria to follow:
  • Input: Zero or more quantities (externally produced)
  • Output: One or more quantities
  • Definiteness: Clarity, precision of each instruction
  • Finiteness: The algorithm has to stop after a finite (may be very large) number of steps
  • Effectiveness: Each instruction has to be basic enough and feasible
    • Understand speech
    • Translate to Chinese

4,5,6: Implementation, Testing,

Maintainance

• Implementation
  • Decide on the programming language to use
    • C, C++, Lisp, Java, Perl, Prolog, assembly, etc. , etc.
  • Write clean, well documented code
• Test, test, test
• Integrate feedback from users, fix bugs, ensure compatibility across different versions  Maintenance

Space Complexity

• Space complexity = The amount of memory required by an algorithm to run to completion - [Core dumps = the most often encountered cause is “memory leaks” – the amount of memory required larger than the memory available on a given system]
• Some algorithms may be more efficient if data completely loaded into memory - Need to look also at system limitations - E.g. Classify 2GB of text in various categories [politics, tourism, sport, natural disasters, etc.] – can I afford to load the entire collection?

Space Complexity (cont’d)

1. Fixed part: The size required to store certain data/variables, that is independent of the size of the problem:
   • e.g. name of the data collection
   • same size for classifying 2GB or 1MB of texts
2. Variable part: Space needed by variables, whose size is dependent on the size of the problem:
   • e.g. actual text
   • load 2GB of text VS. load 1MB of text

Time Complexity

• Often more important than space complexity
  • space available (for computer programs!) tends to be larger and larger
  • time is still a problem for all of us
• 3-4GHz processors on the market
  • still …
  • researchers estimate that the computation of various transformations for 1 single DNA chain for one single protein on 1 TerraHZ computer would take about 1 year to run to completion
• Algorithms running time is an important issue

Running Time

• Problem: prefix averages
  • Given an array X
  • Compute the array A such that A[i] is the average of elements X[0] … X[i], for i=0..n-
• Sol 1
  • At each step i, compute the element X[i] by traversing the array A and determining the sum of its elements, respectively the average
• Sol 2
  • At each step i update a sum of the elements in the array A
  • Compute the element X[i] as sum/I

Big question: Which solution to choose?

Experimental Approach

• Write a program that implements the algorithm
• Run the program with data sets of varying size.
• Determine the actual running time using a system call to measure time (e.g. system (date) );
• Problems?

Experimental Approach

• It is necessary to implement and test the algorithm in order to determine its running time.
• Experiments can be done only on a limited set of inputs, and may not be indicative of the running time for other inputs.
• The same hardware and software should be used in order to compare two algorithms. – condition very hard to achieve!

Algorithm Description

• How to describe algorithms independent of a programming language
• Pseudo-Code = a description of an algorithm that is
  • more structured than usual prose but
  • less formal than a programming language
• (Or diagrams)
• Example: find the maximum element of an array. Algorithm arrayMax(A, n ): Input: An array A storing n integers. Output: The maximum element in A. currentMax ← A[0] for i ← 1 to n -1 do if currentMax < A[i] then currentMax ← A[i] return currentMax

Pseudo Code

• Expressions: use standard mathematical symbols
  • use ← for assignment (? in C/C++)
  • use = for the equality relationship (? in C/C++)
• Method Declarations: - Algorithm name( param1, param2 )
• Programming Constructs:
  • decision structures: if ... then ... [else ..]
  • while-loops while ... do
  • repeat-loops: repeat ... until ...
  • for-loop: for ... do
  • array indexing: A[i]
• Methods
  • calls: object method(args)
  • returns: return value
• Use comments
• Instructions have to be basic enough and feasible!

Example

Algorithm arrayMax(A, n): Input: An array A storing n integers. Output : The maximum element in A. currentMax A[0] for i  1 to n -1 do if currentMax < A[i] then currentMax ← A[i] return currentMax

How many operations?

Asymptotic Notation

• Need to abstract further
• Give an “idea” of how the algorithm performs
• n steps vs. n+5 steps
• n steps vs. n 2 steps