Delta PNG Output Format

The Delta PNG output format can save a lot of latency and bandwidth, and is especially useful in latency & bandwidth-critical scenarios like mobile apps.

It requires that the pixels in the original image are loaded on the client, and then the Delta PNG is applied to the original image to produce the result image.

An example:

Even in this hair-centric example (which is a worst-case scenario for the Delta PNG format), the savings are significant: 73%

Decoding Delta PNG

A Delta PNG is just a regular PNG file and can be read by any software library capable of reading PNGs. The only difference compared to a regular PNG result is in the pixel values themselves. The background is encoded as transparent black 0x00000000 and the foreground as transparent white 0x00FFFFFF. Partially transparent pixels have their actual color values.

This means that when you're decoding the Delta PNG pixel values, you need to pull the actual pixel value from the Original when you encounter transparent white 0x00FFFFFF. The other pixels have the same values as in the regular PNG format.

Here is a TypeScript code example for decoding the Delta PNG format:

export function decodeDeltaPngInPlace(originalPixels: Uint8Array, deltaPngPixels: Uint8Array): Uint8Array {
    const N = originalPixels.length / 4; // Array of RGBA values, div 4 to get number of pixels
    for (let i = 0; i < N; i++) {
        const i4 = i * 4;
        const alpha = deltaPngPixels[i4 + 3]; // JavaScript is RGBA, +3 to get alpha
        if (alpha == 0) {
            const r = deltaPngPixels[i4]; // JavaScript is RGBA, +0 to get red
            if (r == 0xFF) {
                // Transparent white => foreground => take values from original
                deltaPngPixels[i4] = originalPixels[i4];
                deltaPngPixels[i4 + 1] = originalPixels[i4 + 1];
                deltaPngPixels[i4 + 2] = originalPixels[i4 + 2];
                deltaPngPixels[i4 + 3] = originalPixels[i4 + 3];
            } // else transparent black => background => keep values
        } // else partially transparent => keep values
    }
    return deltaPngPixels;
}

To learn more about operating on image and pixel data in JavaScript, see the excellent Pixel manipulation with canvas tutorial on the Mozilla Developer Network.

Caveats

Your image loading library must be able to preserve the pixel values even for fully transparent pixels, which is how it normally works.

However, if you're using e.g. Python and the well known OpenCV library, then you need to use the cv2.IMREAD_UNCHANGED flag and load the image like so: cv2.imread(path, cv2.IMREAD_UNCHANGED). Otherwise OpenCV clobbers the actual pixel values of the fully transparent pixels.

Unfortunately OpenCV doesn't apply any rotation information stored in the image when you use that flag. This is why we return the X-Input-Orientation header so that you can apply the correct orientation to the image in this scenario.

Here is a Python+OpenCV code example for applying the orientation:

def apply_exif_rotation(im: np.ndarray, orientation: int) -> np.ndarray:
    # https://note.nkmk.me/en/python-opencv-numpy-rotate-flip/
    if 1 < orientation <= 8:
        if 2 == orientation:  # TOP-RIGHT, flip left-right, [1, 1] -> [-1, 1]
            im = cv2.flip(im, 1)
        elif 3 == orientation:  # BOTTOM-RIGHT, rotate 180
            im = cv2.rotate(im, cv2.ROTATE_180)
        elif 4 == orientation:  # BOTTOM-LEFT, flip up-down, [1, 1] -> [1, -1]
            im = cv2.flip(im, 0)
        elif 5 == orientation:  # LEFT-TOP, Rotate 90 and flip left-right
            im = cv2.rotate(im, cv2.ROTATE_90_CLOCKWISE)
            im = cv2.flip(im, 1)
        elif 6 == orientation:  # RIGHT-TOP, Rotate 90
            im = cv2.rotate(im, cv2.ROTATE_90_CLOCKWISE)
        elif 7 == orientation:  # RIGHT-BOTTOM,
            im = cv2.rotate(im, cv2.ROTATE_90_CLOCKWISE)
            im = cv2.flip(im, 0)
        else:  # 8 == orientation:  # LEFT-BOTTOM, Rotate 270
            im = cv2.rotate(im, cv2.ROTATE_90_COUNTERCLOCKWISE)
    return im

• Remove Backgrounds
• Face Stickers
• Delta PNG Format
• Migrating