Dali: Streams -- C API

Header Files

#include <dvmbasic.h>

Type Definitions

BitStream

A BitStream is a memory buffer containing raw binary data. It is most often used as an I/O buffer.

    typedef struct BitStream {
        unsigned char *buffer;
        unsigned char *endDataPtr;
        unsigned char *endBufPtr;
        unsigned char isVirtual;
        int size;
    } BitStream;

buffer

pointer to the first byte in the BitStream memory buffer.

endBufPtr

pointer to the last allocated byte in the BitStream memory buffer.

endDataPtr

pointer to the last data byte in the memory buffer. (If we have a 100 byte buffer, but store only 80 bytes of data inside, endDataPtr will point to the 80-th byte while endBufPtr will point to the 100-th byte.)

isVirtual

1 iff the BitStream is created by casting from another type.
0 otherwise.

size

the size of the BitStream, equivalent to endBufPtr - buffer.

BitParser

A BitParser maintains a cursor into a BitStream. Reading and writing to and from a BitStream are usually done via a BitParser.

    typedef struct BitParser {
        BitStream *bs;
        unsigned char *offsetPtr;
        int currentBits;
        int bitCount;
    } BitParser;

bs

pointer to the BitStream the BitParser is attached to.

offsetPtr

pointer to the next unconsumed byte inside the BitStream.

currentBits

32-bits data in the BitStream just before offsetPtr.

bitCount

number of unconsumed bits in the currentBits.

FilterEntry

A FilterEntry is a pair (offset, length) identifying a segment of a BitStream.

    typedef struct FilterEntry {
        unsigned int offset;
        unsigned int length;
    } FilterEntry;

offset

offset of a segment from the beginning of the input.

length

length of a segment in the input.

BitStreamFilter

A BitStreamFilter consists of an array of FilterEntrys which defines a sequence of segments in a BitStream (i.e., it's a scatter/gather vector). It also maintains enough state to perform a scan over these segments using multiple function calls. This is an essential feature for large BitStreams.
For example, a BitStreamFilter can be used to define the video stream within an MPEG system stream.

    typedef struct BitStreamFilter {
        int maxEntry;
        int lastEntry;
        int currEntry;
        int currOffset;
        int currLength;
        FilterEntry *table;
    } BitStreamFilter;

maxEntry

The number of entries allocated in the filter (i.e., the size of table).

lastEntry

The index (in table) of the last entry added to the filter.

table

The array of filter entries.

currEntry
The next three slots are needed when a filter is used to copy data from a source over multiple calls. This can get tricky because only part of a segment may get copied during one call. These 3 slots maintain sufficient state so that the next call works correctly.
The following simplified pseudo-code shows how a filter is used to read N bytes from source (assuming the source is a file):
         while (currLength < N) {
             Seek to position currOffset within source;
             read currLength bytes from source into a BitStream;
             N = N - currLength;
             currEntry++;
             currLength = table[currEntry].length;
             currOffset = table[currEntry].offset;
         }
         if (N != 0) {
             Seek to position currOffset within source;
             read N bytes from source into a BitStream;
             currLength -= N;
             currOffset += N;
         }
         
CurrLength bytes are copied from the source, starting at offset currOffset from the beginning of the source. For example, if currLength is 32,currOffset is 2500, and the source is a file, the next call will first copy 32 bytes, starting at 2500 bytes into the file.
CurrEntry is the index (in table) of the entry that will be first accessed during the next call to copy data. If part of that entry was accessed during the previous call, the rest of it will be read during the next call.
currOffset

If the last call using the filter resulted in a partial read, currOffset stores the offset from the beginning of the source where data should be taken from.

currLength

If the last call using the filter resulted in a partial read, currLength stores the amount of data (in bytes) that should be copied from the source during the next read to complete this segment.

Constants

Return Codes

DVM_STREAMS_OK indicates that a bitstream operation is successful. DVM_STREAM_FILTER_FULL is returned if we are trying to add entries into a full BitStreamFilter.

    #define DVM_STREAMS_OK 0
    #define DVM_STREAMS_FILTER_FULL -1

Operators

BitStream

BitStream *BitStreamNew(int size)

Allocate a new BitStream of size bytes, and return a pointer to the BitStream. If size=0, only the header, not the buffer, is allocated. This form is useful in conjunction with BitStreamShare() (see below).

BitStream *BitStreamMmapReadNew (char *fileName)

Allocate and return a new BitStream whose contents are a memory map of the file fileName. Returns NULL if the memory map fails. This function is only available if the source is compiled with the flag HAVE_MMAP=1.

void BitStreamMmapReadFree (BitStream *bs)

Free the memory associated with bs and clear the memory map of the file. Bs must have been allocated with BitStreamMmapReadNew().

void BitStreamFree(BitStream *bs)

Deallocate the BitStream pointed to by bs. Use BitStreamMmapReadFree to free memory mapped bitstreams

void BitStreamShift(BitStream *bs, int offset)

Moves data in bs, from offset to the end of data in the BitStream, to the beginning of buffer.
For example, if we have 80 bytes of data in a BitStream, calling BitStreamShift(bs, 59) will move bytes 59..79 to positions 0..19.)

void BitStreamResize(BitStream *bs, int size)

Reallocate the BitStream to size number of bytes. If the BitStream size is increased, the contents of the buffer are unchanged. If it is decreased, the data will be truncated.

void BitStreamShareBuffer (BitStream *src, BitStream *dest)

Share the buffer of src in dest. That is, dest's buffer will be set to src's buffer. Warning: dest's buffer is not free'd, so memory leaks can occur if not used carefully.
A common way to use this function is in conjunction with BitStreamNew(0).

int BitStreamBytesLeft(BitStream *bs, int off)

Return the number of bytes of data in bs from postion off to the end.
For example, if bs has 80 bytes, BitStreamBytesLeft(bs, 60) will return 20.

int BitStreamDump (BitStream *inbs, int inOff, BitStream *outbs, int outOff, int len)

Copy len bytes from inbs+inOff to outbs+outOff. Returns the total number of bytes copied.

int BitStreamDumpSegments (BitStream *inbs, int inOff, BitStream *outbs, int outOff, int len, int skip, int count)
This function is useful for copying regularly spaced segments from one BitStream to another. It works as follows. Starting with inbs's at offset inOff, repeat the following steps count times:

Copy len bytes to outbs, starting with offset outoff.
Skip over skip bytes in inbs.
Returns the total number of bytes copied.
For example, the following fragment copies bytes 100-119 and 130-149 of bs1 to bytes 10-49 of bs2:
         BitStreamDumpSegments (bs1, 100, bs2, 10, 20, 10, 2 )
      

BitParser

BitParser *BitParserNew ()

Allocate and return a pointer to a new BitParser.

void BitParserFree (BitParser *bp)

Free the memory allocated for bp.

void BitParserWrap (BitParser *bp, BitStream *bs)

Attach bp to bs. The cursor of bp will be set to the first byte in bs. All future reads from and writes to bs will modify the data in bp. All future writes using bp will modify data in bs.

int BitParserTell(BitParser *bp)

Return the position of the cursor of bp. This is the offset, in bytes, from the beginning of the BitStream to which bp is attached.

void BitParserSeek(BitParser *bp, int off)

Move the cursor of bp to byte offset off.

BitStreamFilter

BitStreamFilter *BitStreamFilterNew(int size)

Create a new BitStreamFilter with size entries.

void BitStreamFilterResize(BitStreamFilter *filter, unsigned int size)

Change the size of BitStreamFilter filter to size entries, existing entries in the filter will remain unchanged.

void BitStreamFilterFree(BitStreamFilter *filter)

Deallocate the BitStreamFilter filter and all entries in it.

int BitStreamFilterAdd(BitStreamFilter *filter, unsigned int offset, unsigned int length)

Add a new entry (offset, length) to the BitStreamFilter filter. Return DVM_STREAMS_FILTER_FULL if there are no more entries left in the BitStreamFilter filter, return DVM_STREAMS_OK otherwise.

void BitStreamFilterRead(FILE *file, BitStreamFilter *filter) void BitStreamFilterWrite(FILE *file, BitStreamFilter *filter)

Allocate and return a new BitStreamFilter whose contents are a memory map of the file fileName. Returns NULL if the memory map fails. This function is only available if the source is compiled with the flag HAVE_MMAP=1.

void BitStreamFilterStartScan(BitStreamFilter *filter)

Reset filter so that the next time it's used it will retrieve data from the beginning of the source.

int BitStreamDumpUsingFilter(BitStream *src, int srcOff, BitStream *dest, int destOff, int length, BitStreamFilter *index)
Copy a subset of data from the BitStream src (starting at srcOffset) to the BitStream dest (starting at destOffset). The subset is specified by the scatter/gather data in the BitStreamFilter filter. At most length bytes are copied. Filter maintains sufficient state so that the next call to BitStreamDumpUsingFilter() will read the data correctly (provided src is not modified between calls) . You can use the function BitStreamFilterStartScan to reset this state (e.g., if you rewind to the beginning of the file).
The number of bytes copied is returned. For example, the following function uses BitStreamDumpUsingFilter to make sure that outbs has at least minsize bytes in it. If less than minsize bytes are available in outbs, data from inbs is copied into outbs using the BitStreamFilter filter. The number of bytes added to outbs is returned.
      int FillBS (Bitstream *inbs, Bitstream *outbs, BitParser *outbp,
                  BiStreamFilter *index, int minsize) {
          pos = BitParserTell (outbp);
          remain = BitStreamBytesLeft (outbs, pos);
          if (remain < minsize) {
              BitStreamShift (outbs, pos);
              BitParserSeek (outbp, 0);
              return BitStreamDumpUsingFilter (inbs, 0, outbs, remain, 65536, index);
          } else {
              return 0
          }
      }
      

Input/Output

int BitStreamFileRead ( BitStream *bs, FILE *f, int offset)

Fill up the BitStream bs buffer with bits from FILE f starting at byte offset position offset. Return the number of bytes read into the buffer.

int BitStreamFileReadSegment ( BitStream *bs, FILE *f, int offset, int length)

Reads length bytes into the BitStream bs from file f, starting at byte offset position offset of the bitstream. Return the number of bytes read into the buffer (may be less than length if there is not enough data from the bs).

int BitStreamFileReadSegments ( BitStream *bs, FILE *f, int offset, int size, int skip, int times)
Starting with bitstream bs's offset offset, do the following for times number of loops :

read size bytes into the bitstream buffer from file f,
skip skip number of bytes in f,
Return the total number of bytes read into the buffer.
For example, the following fragment reads bytes 100-119 and 130-149 of file1 into bs1
         BitStreamFileReadSegments (bs1, file1, 100, 20, 10, 2)
      
int BitStreamFileFilterIn ( BitStream *bs, FILE *f, int offset, BitStreamFilter *filter)

Starting with BitStream bs's offset offset fill up the BitStream with data from file f, using the filter : For each pair (offset, length) in filter, or until the buffer is full, do

seek to offset in the channel
read length bytes (or less if the BitStream is full) from the channel into the buffer
The BitStreamFilter maintains the state of the file and will be changed after this function is called, so that the next call to this primitive will read the data correctly.

int BitStreamFileWrite (BitStream *bs, FILE *f, int offset)

Write all data from starting from byte offset offset from the BitStream bs out into file f, Return the number of bytes actually been written into channel.

int BitStreamFileWriteSegment (BitStream *bs, FILE*f, int offset, int size)

Write size bytes starting from byte offset offset from the BitStream bs out into file f, Return the number of bytes written into f.

int BitStreamFileWriteSegments ( BitStream *bs, FILE *f, int offset, int length, int skip, int times)
Starting with BitStream bs's offset offset do the following for times number of times :

write length bytes into the file f from the BitStream bs.
skip skip number of bytes in the BitStream
Return the total number of bytes written into the channel.
For example, the following fragment writes bytes 100-119 and 130-149 of bs1 into file1
         BitStreamFileWriteSegments (bs1, file1, 100, 20, 10, 2)
      