3/06: 3. naive & better solutions to max subsequence sum
_______________________________________________________________________________

3.0 general problems with simple solutions to max subsequence sum

+ let $s$ be the input sequence

+ let $n=length(s)$ be the length of the input sequence

+ storing the entire input sequence and then processing it
  would be big and slow :(

+ storing the entire input sequence would require $O(n)$ storage versus
  $O(1)$ for our sophisticated solution

+ computing all subsequence sums would require $O(n^3)$ or $O(n^2)$ steps
  versus $O(n)$ for our sophisticated solution
_______________________________________________________________________________

3.1 naive solution:

  sofar = 0;
  for j = 1:length(s)
      for i = 1:j
          % compute $i2j = sum(s(i:j))$
              i2j = 0;
              for k = s(i:j)
                  i2j = i2j + k;
              end
          if i2j > sofar
              sofar = i2j;
          end
      end
  end
_______________________________________________________________________________

3.2 (3/08 preview) performance of naive solution

space complexity of naive solution: $O(n)$ to store $s$

time complexity:
+ loop $k$ is executed at most about $n$ times ($n+1$, $n-1$ who cares?)
+ loop $i$ is executed at most about $n$ times ($n+1$, $n-1$ who cares?)
+ loop $j$ is executed at most about $n$ times ($n+1$, $n-1$ who cares?)
+ every line that is not a $for$-loop does bounded work
+ total time is about at most $constant * n * n * n = O(n^3)$.

remark: a careful analysis yields that line $i2j = i2j + k;$ is executed
        about $n^3/6$ times, which matches our answer
_______________________________________________________________________________

3.3 better solution

+ observe: if $i<=j<=k$, then $sum(s(i:k)) = sum(s(i:j-1)) + sum(s(j:k))$

  words: the sum from positions $i$ to $k$
         = the sum from positions $i$ to $j-1$
         + the sum from positions $j$ to $k$

  picture:    i     j-1 j       k
              |       | |       |
              V       V V       V
    +--------+-+-----+-+-+-----+-+--------+
  s |  ...   | | ... | | | ... | |   ...  +
    +--------+-+-----+-+-+-----+-+--------+

             +-+-----+-+-+-----+-+
             | | ... | | | ... | |  sum(s(i:k))
             +-+-----+-+-+-----+-+
                                       =
            +-+-----+-+ +-+-----+-+
            | | ... | | | | ... | | sum(s(i:j-1)) + sum(s(j:k))
            +-+-----+-+ +-+-----+-+
             
+ plug in $i=1$: $sum(s(1:k)) = sum(s(1:j-1)) + sum(s(j:k))$.

+ rearrange terms: $sum(s(j:k)) = sum(s(1:k)) - sum(s(1:j-1))$.

  words: the sum from positions $j$ to $k$
         = (the cumulative sum to position $k$)
         - (the cumulative sum to position $j-1$).

  picture:

     1              j-1 j       k
     |                | |       |
     V                V V       V
    +----------------+-+-+-----+-+--------+
  s |      ...       | | | ... | |   ...  +
    +----------------+-+-+-----+-+--------+

                       +-+-----+-+
                       | | ... | |   sum(s(j:k))
                       +-+-----+-+
                                        =
    +- - - - - - - - + +-+-----+-+
  s :     ...        : | | ... | |   sum(s(1:k)) - sum(s(1:j-1))
    +- - - - - - - - + +-+-----+-+


+ let $S = [0 cumsum(s)]$, i.e. $S(i) = sum(s(1:i-1))$.

+ the maximum subsequence sum of $s$ can be rewritten as

     max    sum(s(i:j)) =     max (S(j+1)-S(i)) =      max    (S(j)-S(i))
  1<=i<=j+1,              1<=i<=j+1,              1<=i<=j<=n+1
     j<=n                    j<=n

  note: we need $i<=j+1$ (or $i<=j$) because as we saw earlier,
        $sum(s(i:j))=0$ when $i>j$, but we are guaranteed 
        $(S(j+1)-S(i))=0$ only when $i=j+1$ but not in general when $i>j$.


using the ideas above gives us the following unvectorized code:

  % compute $S = [0 cumsum(s)]$, i.e. $S(i) = sum(s(1:i-1))$
      S = 0;
      for x = s
          S = [S S(end)+x];
      end
  % compute $sofar = max subsequence sum of s$
      sofar = 0;
      for j = 1:length(s)+1
          for i = 1:j-1
              i2j = S(j) - S(i); % $sum(s(i:j-1))$
              if i2j > sofar
                  sofar = i2j;
              end
          end
      end

note: $i$ goes only up to $j-1$ because when $j=i$ we get $S(j)-S(i)=0$,
      which is already taken care of by initializing $sofar$ to 0.
_______________________________________________________________________________

3.4 (3/08 preview) performance of better solution

space complexity: $O(2n+1) = O(n)$ to store $s$ and $S$.

time complexity: $O(n^2)$

+ $x$ loop: $O(n^2$)
  + $x$ loop is executed about $n$ times
  + HOWEVER, body $S = [S S(end)+s];$: $O(n^2)$ does NOT do bounded work!
    + the $i$-th time takes
      + about $i$ steps to copy each the previous $i$ or so values of $S$
      + about 1 step to compute $S(end)+s$
  + total time is at most about $constant * n * n = O(n^2)$.

  + remark: the total number of elements copied for $S$ by this loop is about
            1 + 2 + 3 + ... + n = about n(n+1)/2

+ nested $i,j$ loops: $O(n^2)$
  + body of $j$: bounded work
  + $j$ loop: executed at most about $n$ times
  + $i$ loop: executed at most about $n$ times
  + total time is at most about $constant * n * n = O(n^2)$.
  
  + remark: the total number of times $i2j = S(j) - S(i);$ is executed is
            about 1 + 2 + 3 + ... + n = about n(n+1)/2

+ total time: $O(n^2) + O(n^2) = O(n^2)$

+ remark: the total number of elements copied for $S$ plus
          number of assignments to $i2j$ is about 2*n(n+1)/2 = n^2+n,
          which is $O(n^2+n) = O(n^2)$ -- same result as above.

+ so this is better than the first, naive algorithm, but is still
  much worse than $O(n)$.