• Heikki Linnakangas's avatar
    Clean up the Simple-8b encoder code. · d303122e
    Heikki Linnakangas authored
    Coverity complained that simple8b_encode() might read beyond the end of
    the 'diffs' array, in the loop to encode the integers. That was a false
    positive, because we never get into the loop in modes 0 or 1, and the
    array is large enough for all the other modes. But I admit it's very
    subtle, so it's not surprising that Coverity didn't see it, and it's not
    very obvious to humans either. Refactor it, so that the second loop
    re-computes the differences, instead of carrying them over from the first
    loop in the 'diffs' array. This way, the 'diffs' array is not needed
    anymore. It makes no measurable difference in performance, and seems more
    straightforward this way.
    
    Also, improve the comments in simple8b_encode(): fix the comment about its
    return value that was flat-out wrong, and explain the condition when it
    returns EMPTY_CODEWORD better.
    
    In the passing, move the 'selector' from the codeword's low bits to the
    high bits. It doesn't matter much, but looking at the original paper, and
    googling around for other Simple-8b implementations, that's how it's
    usually done.
    
    Per Coverity, and Tom Lane's report off-list.
    d303122e
integerset.c 28.1 KB