Buffer Overflows: Understanding and Preventing Common Cybersecurity Threats - Prof. John H, Study notes of Cryptography and System Security

An in-depth analysis of various types of buffer overflows, including trojan horses, login spoofing, worms, logic bombs, trap doors, and buffer overflow attacks. It covers topics such as operating system security, weak c functions, and stack smashing. Students will learn about the consequences of poor programming and the importance of security principles.

Typology: Study notes

Pre 2010

Uploaded on 08/18/2009

koofers-user-grh-2
koofers-user-grh-2 🇺🇸

8 documents

1 / 28

Toggle sidebar

This page cannot be seen from the preview

Don't miss anything!

bg1
COMP 6370 – Buffer Overflows – Lecture 8
1
Operating System Security
Trojan Horses
Does NOT self-replicate
Free program made available to unsuspecting user
Actually contains code to do harm
Place altered version of utility program on victim's computer
trick user into running that program
la
/usr/mal/ls
Rootkits
Remote Access Tools
PCAnywhere
Laplink
Back Orifice
pf3
pf4
pf5
pf8
pf9
pfa
pfd
pfe
pff
pf12
pf13
pf14
pf15
pf16
pf17
pf18
pf19
pf1a
pf1b
pf1c

Partial preview of the text

Download Buffer Overflows: Understanding and Preventing Common Cybersecurity Threats - Prof. John H and more Study notes Cryptography and System Security in PDF only on Docsity!

COMP 6370 – Buffer Overflows – Lecture 8

Operating System Security

Trojan Horses

  • Does NOT self-replicate
  • Free program made available to unsuspecting user
    • Actually contains code to do harm
  • Place altered version of utility program on victim's computer
    • trick user into running that program
    • la
    • /usr/mal/ls
  • Rootkits
  • Remote Access Tools
    • PCAnywhere
    • Laplink
    • Back Orifice

COMP 6370 – Buffer Overflows – Lecture 8

Login Spoofing

(a) Correct login screen

(b) Phony login screen

COMP 6370 – Buffer Overflows – Lecture 8

Logic Bombs

• Company programmer writes program

  • potential to do harm
  • OK as long as he/she enters password daily
  • ff programmer fired, no password and bomb explodes

COMP 6370 – Buffer Overflows – Lecture 8

Trap Doors

(a) Normal code.

(b) Code with a trapdoor inserted

COMP 6370 – Buffer Overflows – Lecture 8

Generic Security Attacks

Typical attacks

  • Request memory, disk space, tapes and just read
  • Try illegal system calls
  • Start a login and hit DEL, RUBOUT, or BREAK
  • Try modifying complex OS structures
  • Try to do specified DO NOTs
  • Convince a system programmer to add a trap door
  • Beg admin's sec’y to help a poor user who forgot password

COMP 6370 – Buffer Overflows – Lecture 8

Design Principles for Security

1. System design should be public

2. Default should be no access

3. Check for current authority

4. Give each process least privilege possible

5. Protection mechanism should be

  • simple
  • uniform
  • in lowest layers of system

6. Scheme should be psychologically acceptable

And … keep it simple

COMP 6370 – Buffer Overflows – Lecture 8

RC5 Stats

  • Ignoring artificially high numbers resulting from network difficulties, we completed 86,950,894 workunits on our best day. - This is 0.12% of the total keyspace meaning that at our peak rate we could expect to exhaust the keyspace in 790 days.
  • Our peak rate of 270,147,024 kkeys/sec is equivalent to 32, 800MHz Apple PowerBook G4 laptops or 45,998 2GHz AMD Athlon XP machines or (to use some rc5-56 numbers) nearly a half million Pentium Pro 200s.
  • Over the course of the RC5-64 project, 331,252 individuals participated. We tested 15,769,938,165,961,326,592 keys.
  • Sources: Charles Iser
    • http://www.distributed.net/pressroom/news-20020926.html
    • http://www.rsasecurity.com/news/releases/pr.asp?doc_id=

COMP 6370 – Buffer Overflows – Lecture 8

Review: Preparing a Program for

Execution

  • Use an editor to enter the source code into main memory and save it on a disk as a source file.
  • Use a compiler program to translate the source program into machine languages. - In Ada, you may need to make corrections to your code to get the code to compile, C will usually compile the first time and the errors become apparent at run-time.
  • When the source program is error-free, the compiler saves its machine-language translation as an object file.
  • Call the linker or binder to combine your object program with additional object files needed for your program to execute. - Generally, the linker saves the final result as an executable program on disk.

COMP 6370 – Buffer Overflows – Lecture 8

C, an average programming language

• C is inherently unsafe – programs may overflow

buffers at will.

• No runtime checks that prevent writing past the

end of a buffer.

• Reading or writing past the end of a buffer can

cause a number of diverse behaviors

  • Programs may act in strange ways
  • Programs may fail completely
  • Programs may proceed without any noticeable difference in execution.

COMP 6370 – Buffer Overflows – Lecture 8

Buffer Overrun Side Effects

• Depend on:

  • How much data are written past the buffer bounds
  • What data (if any) are overwritten when the buffer gets full and spills over
  • Whether the program attempts to read data that are overwritten during the overflow
  • What data end up replacing the memory that gets overwritten

COMP 6370 – Buffer Overflows – Lecture 8

Heap Overflow versus Stack Overflow

• Common goal of overflow attacks are root shells

• Attacks are typically against a particular

architecture (OS/machine combination)

• One common technique is to find a buffer

overflow in a suid program

  • ex. lpr, xterm and eject to name a few

• Heaps are harder to exploit because they are

dynamic, not static.

  • programming strategy is to new or malloc() everything
  • main protection is that fewer people know how to exploit heap overflows
  • Generally takes longer to set up a heap overflow attack

COMP 6370 – Buffer Overflows – Lecture 8

Weak C Functions

• strcpy()

• strcat()

• sprint()

• scanf()

• sscanf()

• fscanf()

• vfscanf()

• vscanf()

• vsscanf()

• streadd()

• strecpy()

• strtrns()

COMP 6370 – Buffer Overflows – Lecture 8

realpath()

  • realpath(3C) C Library Functions realpath(3C) NAME realpath - returns the real file name SYNOPSIS #include <stdlib.h> #include <sys/param.h> char *realpath(char *file_name, char *resolved_name); DESCRIPTION realpath() resolves all links and references to .'' and..'' in file_name and stores it in resolved_name. It can handle both relative and absolute path names. For absolute path names and the relative names whose resolved name cannot be expressed relatively (for example, ../../rel-dir), it returns the resolved absolute name. For the other relative path names, it returns the resolved relative name. resolved_name must be big enough (MAXPATHLEN) to contain the fully resolved path name.

COMP 6370 – Buffer Overflows – Lecture 8

Bounds Checking: A Good Thing

• create your own security problem

char buf[1024]; int i = 0; char ch; while ((ch = getchar()) != ‘\n’) { if (ch == -1) break; buf[i++] = ch; }

• Almost any C function that can read in a

character is a candidate for an overflow.