@cloudpss/compress

A set of compression algorithms used by CloudPSS. Auto detect algorithm by magic number. Be carful when handling binary files since their data might be confusing with magic numbers of compressed data.

Algorithms

| Algorithm | Magic number | Filename extension | Mime type | | ----------------------- | ------------------------------------------- | ------------------ | ------------------ | | gzip | ␟ (\x1F) � (\x8B) | .gz | application/gzip | | Zstandard | ( (\x28) � (\xB5) / (\x2F) � (\xFD) | .zst | application/zstd | | brotli (benchmark only) | None (any data might be brotli) | .br | application/brotli |

Benchmarks

> pnpm benchmark

> @cloudpss/[email protected] benchmark C:\Users\lzy\Documents\Sources\cloudpssnet\sdk\packages\01-compress
> node ./benchmark

File: test-data.json    Raw: 9.83 MiB
  gzip -d (wasm)             14.10 ( 714 KiB)      289.62ms      160.11ms
  gzip -d (node)             13.77 ( 731 KiB)       79.17ms       15.55ms
  gzip -d (pako)             13.77 ( 731 KiB)      290.57ms      117.33ms
  gzip -d (fflate)           13.96 ( 721 KiB)      231.52ms       75.93ms
  zstd -3 (Oz,wasm)          18.37 ( 548 KiB)       61.07ms       25.86ms
  zstd -3 (O3,wasm)          18.37 ( 548 KiB)       50.28ms       34.35ms
  zstd -4 (O3,wasm)          18.38 ( 548 KiB)       26.02ms       22.30ms
  zstd -4 (cppzst)           18.26 ( 551 KiB)       22.90ms       12.07ms
  zstd -4 (napi)             18.38 ( 548 KiB)       14.67ms        9.66ms
  zstd -4 (fzstd)            18.38 ( 548 KiB)      --------       82.81ms
  zstd -10 (O3,wasm)         25.97 ( 388 KiB)      140.90ms        7.58ms
  zstd -10 (cppzst)          25.74 ( 391 KiB)       94.25ms       10.79ms
  zstd -10 (napi)            25.97 ( 388 KiB)       67.28ms        5.58ms
  zstd -10 (fzstd)           25.97 ( 388 KiB)      --------       38.86ms
  br -1 (wasm)                4.79 (2.05 MiB)      261.20ms       98.18ms
  br -3 (wasm)               18.53 ( 543 KiB)      277.39ms       23.14ms
  br -3 (node)               20.41 ( 493 KiB)       33.65ms       16.33ms
  br -5 (node)               26.40 ( 381 KiB)       86.91ms       15.09ms
  br -8 (node)               28.10 ( 358 KiB)      147.02ms       14.19ms
File: 四川-PSS独立-直流等值机_Main.cproj        Raw: 12.1 MiB
  gzip -d (wasm)             13.03 ( 953 KiB)      381.01ms      218.08ms
  gzip -d (node)             12.65 ( 981 KiB)      125.49ms       19.95ms
  gzip -d (pako)             12.65 ( 981 KiB)      409.74ms       97.33ms
  gzip -d (fflate)           11.97 (1.01 MiB)      322.22ms       85.90ms
  zstd -3 (Oz,wasm)          17.52 ( 708 KiB)       76.96ms       18.25ms
  zstd -3 (O3,wasm)          17.52 ( 708 KiB)       29.28ms       14.06ms
  zstd -4 (O3,wasm)          17.49 ( 710 KiB)       25.89ms       16.81ms
  zstd -4 (cppzst)           17.37 ( 714 KiB)       28.38ms       19.39ms
  zstd -4 (napi)             17.49 ( 710 KiB)       19.04ms       11.11ms
  zstd -4 (fzstd)            17.49 ( 710 KiB)      --------       51.96ms
  zstd -10 (O3,wasm)         22.00 ( 564 KiB)      305.36ms       12.25ms
  zstd -10 (cppzst)          21.76 ( 570 KiB)      156.59ms       15.04ms
  zstd -10 (napi)            22.00 ( 564 KiB)      164.29ms        6.75ms
  zstd -10 (fzstd)           22.00 ( 564 KiB)      --------       35.26ms
  br -1 (wasm)                3.90 (3.11 MiB)      361.15ms      132.66ms
  br -3 (wasm)               14.74 ( 842 KiB)      409.38ms       34.79ms
  br -3 (node)               18.32 ( 677 KiB)       47.27ms       23.66ms
  br -5 (node)               21.93 ( 566 KiB)      137.89ms       23.17ms
  br -8 (node)               22.42 ( 553 KiB)      330.43ms       17.50ms
File: 电磁暂态方案 1_参数方案 1 (2).cjob        Raw: 8.76 MiB
  gzip -d (wasm)              3.69 (2.37 MiB)      724.77ms      184.32ms
  gzip -d (node)              3.68 (2.38 MiB)      215.39ms       21.58ms
  gzip -d (pako)              3.68 (2.38 MiB)      470.01ms       85.87ms
  gzip -d (fflate)            3.56 (2.46 MiB)      293.85ms       49.24ms
  zstd -3 (Oz,wasm)           3.48 (2.52 MiB)      145.81ms       37.03ms
  zstd -3 (O3,wasm)           3.48 (2.52 MiB)       71.59ms       26.53ms
  zstd -4 (O3,wasm)           3.46 (2.53 MiB)       94.63ms       30.68ms
  zstd -4 (cppzst)            3.45 (2.54 MiB)       84.72ms       20.39ms
  zstd -4 (napi)              3.46 (2.53 MiB)       76.49ms       16.98ms
  zstd -4 (fzstd)             3.46 (2.53 MiB)      --------       78.35ms
  zstd -10 (O3,wasm)          3.78 (2.32 MiB)      758.74ms       24.97ms
  zstd -10 (cppzst)           3.78 (2.32 MiB)      311.69ms       17.79ms
  zstd -10 (napi)             3.78 (2.32 MiB)      280.52ms       13.03ms
  zstd -10 (fzstd)            3.78 (2.32 MiB)      --------       65.44ms
  br -1 (wasm)                2.92 (   3 MiB)      343.00ms      107.80ms
  br -3 (wasm)                3.62 (2.42 MiB)      450.79ms       71.01ms
  br -3 (node)                3.63 (2.41 MiB)      105.14ms       25.24ms
  br -5 (node)                3.86 (2.27 MiB)      396.30ms       23.37ms
  br -8 (node)                3.87 (2.26 MiB)     1177.34ms       25.02ms
File: 电磁暂态方案 1_参数方案 1.cjob    Raw: 24.1 MiB
  gzip -d (wasm)              4.18 (5.77 MiB)     1395.42ms      498.84ms
  gzip -d (node)              4.17 (5.79 MiB)      304.33ms       49.91ms
  gzip -d (pako)              4.17 (5.79 MiB)      748.55ms      179.98ms
  gzip -d (fflate)            4.08 (5.91 MiB)      584.66ms      137.39ms
  zstd -3 (Oz,wasm)           4.27 (5.65 MiB)      491.35ms       89.71ms
  zstd -3 (O3,wasm)           4.27 (5.65 MiB)      354.26ms       63.80ms
  zstd -4 (O3,wasm)           4.30 (5.61 MiB)      289.05ms       46.23ms
  zstd -4 (cppzst)            4.29 (5.62 MiB)      145.05ms       39.46ms
  zstd -4 (napi)              4.30 (5.61 MiB)      112.67ms       25.97ms
  zstd -4 (fzstd)             4.30 (5.61 MiB)      --------      140.66ms
  zstd -10 (O3,wasm)          4.41 (5.47 MiB)     1208.62ms       44.96ms
  zstd -10 (cppzst)           4.41 (5.47 MiB)      563.06ms       44.58ms
  zstd -10 (napi)             4.41 (5.47 MiB)      475.77ms       27.73ms
  zstd -10 (fzstd)            4.41 (5.47 MiB)      --------      137.30ms
  br -1 (wasm)                3.05 (7.91 MiB)      811.77ms      270.13ms
  br -3 (wasm)                4.07 (5.93 MiB)     1031.59ms      168.99ms
  br -3 (node)                4.25 (5.67 MiB)      179.47ms       50.69ms
  br -5 (node)                4.87 (4.95 MiB)      829.20ms       51.89ms
  br -8 (node)                4.84 (4.98 MiB)     3432.08ms       51.79ms