Programming Language Design and Performance, Exams of Programming Languages

Download Programming Language Design and Performance and more Exams Programming Languages in PDF only on Docsity!

Programming Language Design and

Performance

Jonathan Aldrich

17-396: Language Design and Prototyping Spring 2020

Opening discussion

• What features/tools make a language fast?

2

Dynamic optimization

4

Safety/Abstraction/Dynamism

Performance

C

Java

JavaScript

Dynamic optimization techniques Brings Java to ~2x of C run time, JavaScript ~3x (depending on benchmark) Origins in Self VMs

Source: https://benchmarksgame-team.pages.debian.net/benchmarksgame/

Parallelizing Compilers

5

Safety/Abstraction/Dynamism

Performance

C, Fortran

Java

JavaScript

Parallelizing Compilers

• Can parallelize C, Fortran
• Requires substantial platform-specific tweaking
• One study: hand-optimized Fortran code ~10x larger (and ~2x faster) than unoptimized Fortran

Source: https://queue.acm.org/detail.cfm?id=

The Importance of Libraries

• Python: widely used for scientific computing

• Perceived to be easy to use
• Slow (but see PyPy, which is a lot like Truffle/Graal)
  • Dynamic, interpreted language
  • Boxed numbers (everything is a number allocated on the heap)

• Python packages for scientific computing

• Numpy: multidimensional arrays
  • Fixed size, homogeneous, packed data (like C arrays)
  • Vectorized operations: c = a+b // adds arrays elementwise
• SciPy: mathematical/scientific libraries
  • Wraps BLAS, LAPACK and others
  • Uses Numpy in interface

7

Julia: Performance + Usability

• Dynamic language, like Python/JavaScript

• Excellent libraries for scientific computing

• Like Fortran, Python

• Unique performance strategy

• Uses multiple dispatch to choose appropriate algorithms
  • e.g. sparse vs. full matrix multiplication; special cases for tridiagonal matrices
• Aggressive specialization to overcome cost of abstraction
  • Reduces dispatch overhead, enables inlining
• Optional static type annotations
  • Annotations on variables, parameters, fields enforced dynamically
  • Make specialization more effective

8

Multiple Dispatch

• Ordinary dispatch: choose method based on receiver

x.multiply(y) // selects implementation based on class of x

• Note: overloading changes this slightly, but relies on static type

rather than run-time type

• Multiple dispatch: choose method based on both types

10

Works for Matrices too

11

Specialization/Inlining in Julia

• s

13

Resulting assembly same as C

Source: Bezanson et al., Julia: Dynamism and Performance Reconciled by Design. PACMPL(OOPSLA) 2018.

Type Inference

14

Interprocedural

Source: Bezanson et al., Julia: Dynamism and Performance Reconciled by Design. PACMPL(OOPSLA) 2018.

Why Good Performance in Julia

…despite so little effort on the implementation?

• Dispatch & specialization

• Chooses right algorithm based on run-time types
• Specialize implementation for actual run-time types encountered
  • Allows inlining, unboxing, further optimization

• Programmer discipline

• Type annotations on fields
  • Allows compiler to infer types read from the heap
  • It knows types of arguments from dispatch/specialization
• Type stability
  • Code is written so that knowing the concrete types of arguments allows the compiler to infer concrete types for all variables in the function - Thus specialized code becomes monomorphic: no dispatch, no polymorphism
  • Maintained by programmer discipline

16

Zero Cost Abstraction: From C to C++

• Starts with C, but adds abstraction facilities (and a little

dynamism)

• Motto: “Zero-cost abstraction”

• C++ can perform similarly to C, but is (somewhat) higher-level

• Generic programming: Static specialization with templates

• Templated code is parameterized by one or more types T
• A copy is generated for each instantiation with a concrete type
• Can get genericity with static dispatch instead of dynamic
  • Same benefits as Julia, no GC overhead (unless you choose to add it)
  • More language complexity, and little more safety than C

17

Adding Safety in C++

• Memory issues one of the big problems in C, early C++

• Modern solution: smart pointers

• The pointer itself is an abstraction
• Method calls are passed on to the object

shared_ptr p(new Obj());

shared_ptr q = p; // reference count increments

q->foo(); // OK

p->foo(); // OK

// deallocate memory automatically when both p and q go out of scope

19

Modern C++ programming is completely different from when I taught the language circa 2001 due to smart pointers

Rust: Ownership Types for Memory Safety

• Rust keeps “close to the metal” like C,

provides abstraction like C++

• Safety achieved via ownership types

• Like in Obsidian, every block of memory has an owner
• Adds power using regions
  • A region is a group of objects with the same lifetime
  • Allocated/freed in LIFO (stack) order
  • Younger objects can point to older ones
  • Type system tracks region of each object, which regions are younger
• Fast and powerful—but (anecdotally) hard to learn
  • Nevertheless anyone in this class could do it!

• Unsafe blocks allow bending the rules

• But clients see a safe interface

20