forked from lz4/lz4
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request lz4#102 from Cyan4973/dev
Dev
- Loading branch information
Showing
49 changed files
with
3,393 additions
and
1,674 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,60 +1,80 @@ | ||
| LZ4 - Extremely fast compression | ||
| ================================ | ||
|
|
||
| LZ4 is lossless compression algorithm, providing compression speed at 400 MB/s per core, scalable with multi-cores CPU. It also features an extremely fast decoder, with speed in multiple GB/s per core, typically reaching RAM speed limits on multi-core systems. | ||
| A high compression derivative, called LZ4_HC, is also provided. It trades CPU time for compression ratio. | ||
| LZ4 is lossless compression algorithm, | ||
| providing compression speed at 400 MB/s per core, | ||
| scalable with multi-cores CPU. | ||
| It also features an extremely fast decoder, | ||
| with speed in multiple GB/s per core, | ||
| typically reaching RAM speed limits on multi-core systems. | ||
|
|
||
| Speed can be tuned dynamically, selecting an "acceleration" factor | ||
| which trades compression ratio for more speed up. | ||
| On the other end, a high compression derivative, LZ4_HC, is also provided, | ||
| trading CPU time for improved compression ratio. | ||
| All versions feature the same excellent decompression speed. | ||
|
|
||
|
|
||
| |Branch |Status | | ||
| |------------|---------| | ||
| |master | [](https://travis-ci.org/Cyan4973/lz4) | | ||
| |dev | [](https://travis-ci.org/Cyan4973/lz4) | | ||
| |master | [](https://travis-ci.org/Cyan4973/lz4) [](https://ci.appveyor.com/project/YannCollet/lz4) | | ||
| |dev | [](https://travis-ci.org/Cyan4973/lz4) [](https://ci.appveyor.com/project/YannCollet/lz4) | | ||
|
|
||
|
|
||
|
|
||
|
|
||
| > **Branch Policy:** | ||
| > - The "master" branch is considered stable, at all times. | ||
| > - The "dev" branch is the one where all contributions must be merged before being promoted to master. | ||
| > - If you plan to propose a patch, please commit into the "dev" branch. Direct commit to "master" are not permitted. | ||
| > - Feature branches can also exist, for dedicated testing of larger modifications before merge into "dev" branch. | ||
| > - The "dev" branch is the one where all contributions must be merged | ||
| before being promoted to master. | ||
| > + If you plan to propose a patch, please commit into the "dev" branch, | ||
| or its own feature branch. | ||
| Direct commit to "master" are not permitted. | ||
|
|
||
| Benchmarks | ||
| ------------------------- | ||
|
|
||
| The benchmark uses the [Open-Source Benchmark program by m^2 (v0.14.2)](http://encode.ru/threads/1371-Filesystem-benchmark?p=33548&viewfull=1#post33548) compiled with GCC v4.6.1 on Linux Ubuntu 64-bits v11.10, | ||
| The reference system uses a Core i5-3340M @2.7GHz. | ||
| Benchmark evaluates the compression of reference [Silesia Corpus](http://sun.aei.polsl.pl/~sdeor/index.php?page=silesia) in single-thread mode. | ||
|
|
||
| <table> | ||
| <tr> | ||
| <th>Compressor</th><th>Ratio</th><th>Compression</th><th>Decompression</th> | ||
| </tr> | ||
| <tr> | ||
| <th>LZ4 (r101)</th><th>2.084</th><th>422 MB/s</th><th>1820 MB/s</th> | ||
| </tr> | ||
| <tr> | ||
| <th>LZO 2.06</th><th>2.106</th><th>414 MB/s</th><th>600 MB/s</th> | ||
| </tr> | ||
| <tr> | ||
| <th>QuickLZ 1.5.1b6</th><th>2.237</th><th>373 MB/s</th><th>420 MB/s</th> | ||
| </tr> | ||
| <tr> | ||
| <th>Snappy 1.1.0</th><th>2.091</th><th>323 MB/s</th><th>1070 MB/s</th> | ||
| </tr> | ||
| <tr> | ||
| <th>LZF</th><th>2.077</th><th>270 MB/s</th><th>570 MB/s</th> | ||
| </tr> | ||
| <tr> | ||
| <th>zlib 1.2.8 -1</th><th>2.730</th><th>65 MB/s</th><th>280 MB/s</th> | ||
| </tr> | ||
| <tr> | ||
| <th>LZ4 HC (r101)</th><th>2.720</th><th>25 MB/s</th><th>2080 MB/s</th> | ||
| </tr> | ||
| <tr> | ||
| <th>zlib 1.2.8 -6</th><th>3.099</th><th>21 MB/s</th><th>300 MB/s</th> | ||
| </tr> | ||
| </table> | ||
|
|
||
| The LZ4 block compression format is detailed within [lz4_block_format.txt](lz4_block_format.txt). | ||
|
|
||
| For streaming unknown amount of data and compress files of any size, a frame format has been published, and can be consulted within the file LZ4_Frame_Format.html . | ||
| The benchmark uses the [Open-Source Benchmark program by m^2 (v0.14.3)] | ||
| compiled with GCC v4.8.2 on Linux Mint 64-bits v17. | ||
| The reference system uses a Core i5-4300U @1.9GHz. | ||
| Benchmark evaluates the compression of reference [Silesia Corpus] | ||
| in single-thread mode. | ||
|
|
||
| | Compressor | Ratio | Compression | Decompression | | ||
| | ---------- | ----- | ----------- | ------------- | | ||
| | memcpy | 1.000 | 4200 MB/s | 4200 MB/s | | ||
| |**LZ4 fast 17 (r129)**| 1.607 |**690 MB/s** | **2220 MB/s** | | ||
| |**LZ4 default (r129)**|**2.101**|**385 MB/s** | **1850 MB/s** | | ||
| | LZO 2.06 | 2.108 | 350 MB/s | 510 MB/s | | ||
| | QuickLZ 1.5.1.b6 | 2.238 | 320 MB/s | 380 MB/s | | ||
| | Snappy 1.1.0 | 2.091 | 250 MB/s | 960 MB/s | | ||
| | zlib 1.2.8 -1 | 2.730 | 59 MB/s | 250 MB/s | | ||
| |**LZ4 HC (r129)** |**2.720**| 22 MB/s | **1830 MB/s** | | ||
| | zlib 1.2.8 -6 | 3.099 | 18 MB/s | 270 MB/s | | ||
|
|
||
|
|
||
| Documentation | ||
| ------------------------- | ||
|
|
||
| The raw LZ4 block compression format is detailed within [lz4_Block_format]. | ||
|
|
||
| To compress an arbitrarily long file or data stream, multiple blocks are required. | ||
| Organizing these blocks and providing a common header format to handle their content | ||
| is the purpose of the Frame format, defined into [lz4_Frame_format]. | ||
| Interoperable versions of LZ4 must respect this frame format. | ||
|
|
||
|
|
||
| Other source versions | ||
| ------------------------- | ||
|
|
||
| Beyond the C reference source, | ||
| many contributors have created versions of lz4 in multiple languages. | ||
| A list of these sources is maintained on the [LZ4 Homepage]. | ||
|
|
||
|
|
||
| [Open-Source Benchmark program by m^2 (v0.14.3)]: http://encode.ru/threads/1371-Filesystem-benchmark?p=34029&viewfull=1#post34029 | ||
| [Silesia Corpus]: http://sun.aei.polsl.pl/~sdeor/index.php?page=silesia | ||
| [lz4_Block_format]: lz4_Block_format.md | ||
| [lz4_Frame_format]: lz4_Frame_format.md | ||
| [LZ4 Homepage]: http://www.lz4.info |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.