CS155: Homework #1

Spring 2009

Due: Tuesday, Apr. 28

Problem 1: overflows

A proposal for preventing stack buffer overflow attacks is based on making a backup copy of the return address when a function starts. The backup copy is written to a shadow stack located at some location L on the heap. When the function terminates the backup copy is copied to the appropriate place in the stack and then the function executes the return instruction.

(a) Explain why this mechanism can make it harder to mount a stack buffer overflow attack.

(b) Give sample code that is vulnerable to a stack buffer overflow even if this mechanism is used. Hint: first, try to expose L, then use L to mount the exploit.

Problem 2: overflows

(a) In class we described how LibSafe defends against stack buffer overflows. Give a short sample C code that is vulnerable to buffer overflows even though LibSafe is used.

(b) Can you adapt the LibSafe mechanism to work in the heap? If so explain how and what additional functionality you would need from the memory manager. If not, explain why not.
Hint: it will help to review the inner workings of the heap memory manager used in malloc and free.

(c) Suppose the OS marks the stack memory pages as non-execute. Can a stack overflow be used to get a root shell on the machine? If so, briefly explain how. If not, explain why not.

Problem 3: race-conditions

Consider the following code snippet:

  if (!stat("./file.dat", buf)) return;   // Abort if file exists.
  sleep(10);                              // Sleep for 10 seconds.
  fp = fopen("./file.dat", "w" );
  fprintf(fp, "Hello world" );
  close(fp);

(a) Suppose this code is running as a setuid root program. Give an example of how this code can lead to unexpected behavior that could cause a security problem. (try using symbolic links.)

(b) Suppose the sleep(10) is removed from the code above. Could the problem you identified in part (a) still occur? Please explain.

Problem 4

In the first lecture of the course, we talked about an iPhone vulnerability that was discovered shortly after the phone was released.

Based on slides 15-19 of lecture 1, what kind of vulnerability do you think was found? Explain how you came to this conclusion.
Many security experts caution against "security by obscurity." Was open source useful to the attackers? You may want to look at slides 35-39 of Charlie Miller's slides on Real World Fuzzing.
For each of the following security measures, explain whether Apple's use of the measure could have made it harder to find the iPhone vulnerability, or harder exploit it to take control of the iPhone. If the measure is good security practice, but not related to the iPhone example, say so.
1. Address randomization
2. Non-executable heaps
3. Run applications as an unprivileged user
4. chroot applications to prevent access to unrelated data

Problem 5: covert channels

Recall that modern processors have an on chip data cache (called an L2 cache) used to reduce the number of accesses to main memory. Reading from the L2 cache is about 10 times faster than reading from main memory. Explain how the L2 processor cache can result in a covert channel on the local system. Give a sample short (pseudo) code to exploit this channel.

Assume that processes have separate address spaces (no shared memory), although they share the underlying hardware. You don't need to worry about making the covert channel fast. You may make any reasonable assumption about the L2 cache, as long as you state your assumptions.

CS155: Computer and Network Security