CS410, Summer 1998 Lecture 14 Outline Dan Grossman Goals: * Reviewing the Prelim * Amortized Analysis * The prelim results are just fine! Sample solution is on the web site. We went over all the questions in class. There is not point in repeating the sample solution here. Mean was 60. This is because the mean on question 4 was 11 out of 32. The question was too hard, but except for that, everything was fine. The histogram of grades looks like this: 40s 9 50s 7 60s 9 70s 6 80-82 3 * Amortized Analysis Often we want to bound the worst-case time per operation. But often what we really care about is the worst-case time for _all_ operations put together. One conservative upper bound is: Worst-case total = (worst-case per operation)*(number of operations) But this is weak if the worst-case per op happens rarely. Instead we can try to analyze the worst-case total over many operations directly. Then we can define: Amortized (worst-case) time = (worst-case total)/(number of operations) Individual operations may take longer than the amortized time, but no sequence of operations starting from an "empty" data structure will. Since no such sequence can take more, this means we can never "fall behind". That is, the average must always be less than or equal to the amortized time. Example: Stacks with multi-pop. That is, let push() be as usual, but let pop take an integer i and have it return the i things off the top of the stack (and an error if there aren't i things). If we have n operations, then the stack could have as many as n things on it. Then pop could take an argument that is of order n. So the worst-case time for such an operation is O(n). Since we have n operations, we can claim the total time is O(n^2). But this seems awfully weak. Intuitively, you cannot pop more things than you push and you can't push more than O(n) things. So the total work cannot exceed 2*(number of pushes). That's easy enough just to hand-wave about, but let's use this easy example to show a framework that works for harder examples. What's really going on is we can only do an expensive multi-pop if we have done enough cheap pushes. So let's devise an accounting scheme where we keep careful track of what we mean by "enough cheap pushes". We will have "chips". A chip can pay for O(1) work. So in earlier operations we can pay for later work by buying chips and keeping them around. In order for the accounting to work, we imagine storing the chips in our data structure. Then we can prove we always have enough. In our example, on each push, let's do the real O(1) work and contribute an additional 1 chip of savings. We put this chip on the element we inserted. Now we clearly see that every element in the stack has a chip on it. So when we pop an element, we can spend the chip. So the pop is already paid for! In effect, we have transferred the cost to the push using creative accounting. By making each push cost "2" we have have made pops "free" and proven that the total work over n operations is O(n). So the amortized time is O(1). THESE CHIPS ARE AN ANALYSIS TECHNIQUE ONLY. WE DO NOT ACTUALLY CODE THEM UP. THE IMPLEMENTATION DOES NOT CHANGE IN THE SLIGHTEST. Example: Binary incrementer. Suppose we have a binary incrementer. It starts at 0 and has one operation called increment(). After successive calls, the value of the data structure is 1, 10, 11, 100, 101, etc. We assume all we can do in O(1) time is change the value of a single bit. In the worst case, a single increment might cause w bits to change where w is the number of bits currently in the counter. Since w could be as much as log n after n increments, a weak bound on the total time for n increments would be O(n log n). Amortization can prove a ighter bound... Rather than pay the O(1) to change any bit, let's break the operations into two categories: * Changing a 0 to a 1 * Changing a 1 to a 0 For the former, let's pay the one chip to do the flip and put another chip on the resulting 1. So every 1 in the data structure has a chip on it. For the latter, let's use the saved up chip from earlier. Now we just notice that when incrementing, we always have exactly one operation of the former kind. This is the only one that isn't already paid for, so the operation is O(1) amortized time! Example: Splay Trees. We will not go through the hairy math and analysis necessary to prove the O(log n) amortized time for splay tree operations. Notice though, that this example does proves an O(log n) amortized time, not O(1). Amortization can be used for any bound; it just happens that all our examples today are O(1). Example: Table re-sizing. I have said a hundred times that if you make an array bigger, increase its size by a factor of 2. Now we can prove why -- it gives us an amortized cost of O(1) for insert/delete. As a first attempt, we might try putting one chip on every item when we insert it into the array. Then when we resize, we use the chips to copy the elements over. This doesn't work though -- the second time an item needs to be copied it doesn't have a chip on it any more! What does work is to put _two_ chips on every item when we insert it. Now since we doubled, when the array is full, at least half the elements have 2 chips on them. So we have enough chips to copy over all the elements! What if we do some deletes before the table fills up? Then we're actually doing more cheap operations than we need. Nothing wrong with that. Just forget about any extra chips lying around. (As I said in class, we can give them to the United Way.) We might also want to contract (make smaller) our table if it gets too empty. We can't do so if it becomes half empty, else we would be in bad shape if we did insert, delete, insert, delete, ... right when the table was half-full. What does work is to make the table half as big once it becomes three-quarters empty. We can extend our amortized analysis by also putting 1 chip on an array slot that was full but had its element deleted. It is then easy to see that when the table is three-quarters empty, at least all of the slots from the one-quarter point to the half-way point have chips on them. So we have saved up enough to copy the one-quarter full places in O(1) amortized time.