Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

clean up and export crc32c function #22274

Merged
merged 14 commits into from
Jun 13, 2017
Merged

clean up and export crc32c function #22274

Changes from 1 commit
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
643b1e3
clean up and export crc32c function
stevengj Jun 7, 2017
721bbf5
added PR to NEWS
stevengj Jun 7, 2017
184acd1
use read, not mmap, for crc
stevengj Jun 7, 2017
c3f5d91
slight optimization (don't open cachefile twice for checksum)
stevengj Jun 7, 2017
bd64651
avoid resizing buffer in readbytes
stevengj Jun 7, 2017
62d7dc4
add an assertion
stevengj Jun 7, 2017
ceda720
more crc32c tests
stevengj Jun 7, 2017
67ef143
docs typo
stevengj Jun 8, 2017
f34d08c
crc32c(IOBuffer) bugfix and test
stevengj Jun 8, 2017
38f6fc0
restore crc32 of String, add crc32c(io) to read all of a stream, add …
stevengj Jun 8, 2017
1126d0f
use crc32c block size of 8192*3, matching the underling C library
stevengj Jun 9, 2017
da2fba1
optimized IOBuffer crc32c
stevengj Jun 9, 2017
a1b0431
slight simplification/generalization
stevengj Jun 12, 2017
8e058f3
eliminate theoretical race condition in crc32c if file is being writt…
stevengj Jun 12, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
avoid resizing buffer in readbytes
  • Loading branch information
@stevengj
stevengj committed Jun 7, 2017
commit bd6465154943b42ad139f8faeff1f6735b2a1fd7
12 changes: 7 additions & 5 deletions base/util.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -783,8 +783,10 @@ note that the result is endian-dependent.
"""
function crc32c end

unsafe_crc32c(a, n, crc) = ccall(:jl_crc32c, UInt32, (UInt32, Ptr{UInt8}, Csize_t), crc, a, n)

crc32c(a::Union{Array{UInt8},FastContiguousSubArray{UInt8,N,<:Array{UInt8}} where N}, crc::UInt32=0x00000000) =
ccall(:jl_crc32c, UInt32, (UInt32, Ptr{UInt8}, Csize_t), crc, a, length(a))
unsafe_crc32c(a, length(a), crc)

crc32c(buf::IOBuffer, crc::UInt32=0x00000000) = crc32c(buf.data, crc)

Expand All @@ -796,12 +798,12 @@ mixed with a starting `crc` integer.
"""
function crc32c(f::IO, nb::Integer, crc::UInt32=0x00000000)
buf = Array{UInt8}(min(nb, 16384))
while !eof(f) && nb > 0
n = readbytes!(f, buf, nb)
crc = ccall(:jl_crc32c, UInt32, (UInt32, Ptr{UInt8}, Csize_t), crc, buf, n)
while !eof(f) && nb > 16384
n = readbytes!(f, buf)
crc = unsafe_crc32c(buf, n, crc)
nb -= n
end
return crc
return unsafe_crc32c(buf, readbytes!(f, buf, nb), crc)
end

crc32c(filename::AbstractString, crc::UInt32=0x00000000) =
Expand Down