base:streaming_1_2_4_8-bit_numbers_without_spanning_bytes
Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revision | |||
base:streaming_1_2_4_8-bit_numbers_without_spanning_bytes [2019-10-06 12:59] – white_flame | base:streaming_1_2_4_8-bit_numbers_without_spanning_bytes [2019-10-06 13:09] (current) – white_flame | ||
---|---|---|---|
Line 5: | Line 5: | ||
As long as you're outputting 1, 2, 4, and 8 bit tokens (not 3 or 5-7 bit), you can keep all the bitfields aligned so they do not span byte boundaries, making the reading simpler. | As long as you're outputting 1, 2, 4, and 8 bit tokens (not 3 or 5-7 bit), you can keep all the bitfields aligned so they do not span byte boundaries, making the reading simpler. | ||
- | The reader will hold a byte buffer each for reading 1, 2, or 4-bit values. | + | The reader will hold a byte-sized |
- | The writer packs only tokens of the same bit length into each byte, creating a new output byte at the end of the stream whenever a new byte buffer for that bit length is required. The writer remembers where in the stream the various buffer bytes for each bit length resides, adding to it in place until it's full. | + | The writer packs only tokens of the same bit length into each byte, creating a new output byte at the end of the stream whenever a new buffer for that bit length is required. The writer remembers where in the stream the various buffer bytes for each bit length resides, adding to it in place until it's full. |
Example of writing this arbitrary token stream: | Example of writing this arbitrary token stream: |
base/streaming_1_2_4_8-bit_numbers_without_spanning_bytes.1570359581.txt.gz · Last modified: 2019-10-06 12:59 by white_flame