Near as I can tell, Compression doesn't support LZW, is that right? I'm not sure if the available compression algorithms implicitly know how to decompress LZW.
FB14643019
Compression
RSS for tagEnable your app to provide lossless compression when saving or sharing files and data using Compression.
Posts under Compression tag
4 Posts
Sort by:
Post
Replies
Boosts
Views
Activity
I've got a Data of deflate (zlib)-compressed data that decompresses properly using NSData.decompressed(), but does not decompress properly using compression_decode_buffer(). The working code looks like this:
let compressedData = Data(bytesNoCopy: buf.baseAddress!, count: readCount, deallocator: .none)
let dataWithoutHeader = compressedData[2...]
let ucData = try (dataWithoutHeader as NSData).decompressed(using: .zlib) as Data
The non-working code looks like this:
let samples = try [Float](unsafeUninitializedCapacity: sampleCount)
{ buffer, initializedCount in
print("Count: \(initializedCount)")
try compressedData.withUnsafeBytes<UInt8>
{ (inCompressedBytes: UnsafeRawBufferPointer) -> Void in
let destBufferSize = buffer.count * MemoryLayout<Float>.size
let scratchBuffer = UnsafeMutableRawBufferPointer.allocate(byteCount: compression_decode_scratch_buffer_size(COMPRESSION_ZLIB), alignment: MemoryLayout<Int>.alignment)
defer { scratchBuffer.deallocate() }
let decompressedSize = compression_decode_buffer(buffer.baseAddress!, destBufferSize,
inCompressedBytes.baseAddress!, inCompressedBytes.count,
scratchBuffer.baseAddress!, COMPRESSION_ZLIB)
print("Actual decompressed size: \(decompressedSize), destBufferSize: \(destBufferSize)")
}
initializedCount = sampleCount
}
It ends up printing:
Actual decompressed size: 46510, destBufferSize: 1048576
(1048576 is the correct size. What data is returned does not appear to be correct.)
I have tried it both with and without the first two bytes of the compressed data buffer, and with and without providing a scratch buffer.
I very much love the performance of AppleArchive and how approachable it is, and believe it to be one of the most underrated frameworks in the SDK. In a scenario quite typical, I need to compress files and submit them to a back end, where the server handling the files is not an Apple platform. Obviously, individual files compressed with AA will not be compatible with other systems out of the box, but there are compatible compression algorithms.
ZLIB is recommended for cases where cross-platform compatibility is necessary. As I understand it, AA adds additional headers to files in order to support preservation of file attributes, ownership and other data. Following the steps outlined in the docs, I've written code to compress single files. I can easily compress and decompress using AA without issue.
To create a proof-of-concept, I've written some code in python using its zlib module. In order to get to the compressed data, it's necessary to handle the AA header fields. The first 64 bytes of a compressed file appear as follows:
AA documentation states that ZLIB Level 5 compression is used, and comes in the form of raw DEFLATE data prefixed with two header bytes. In this case, these bytes are 78 5e, which begin at the 28th byte and appear as x^ above. My hope was that seeking to the start of the compressed data, then passing what remains to a decompressor object initialized with the correct WBITS would work.
It works fantastically for files 1MB or less in size. Files which are larger only decompress the first megabyte. The decompressor object is reaching EOF, and I've tried various ways of attempting to seek to and concatenate the other blocks, but to no avail.
Using the older Compression framework and the method specified here, with the same algorithm, yields different results. I can decompress files of any size using python's zlib module. My assumption is that AppleArchive is doing something differently in order to support its multithreading capabilities, perhaps even with asymmetric encoding where the blocks are not ordered.
Is there a solution to this problem? If not, why would one ever use ZLIB versus the much more efficient LZFSE? I could use the older Compression API, but it is significantly slower compressing synchronously, and performance is critical with the application I am adding this feature to.
I am developing a cloud-based application and have integrated the FileProviderExtension. However, files larger than 20 MB are not downloading as it’s throwing a memory limit exception. In this process I have downloaded the file data but after downloaded data need to decompression the data . I am getting memory limit exception during decompression.
I am using below file to decompress the data.
let decompressedData = try? decryptedChunkBytes?.gunzipped()
Data+Gzip.swift